This document describes in example form the core XML-based instructions to achieve payments to Internet accounting systems.
For references and standards information, see the XML-X site. For introductions to other requests not part of the core XML-X set, see Session Management and User Management.
The purpose of XML-X is to enable clients to deliver accounts and payment instructions to servers. There are two major use scenarios for XML-X, being (a) the external merchant site driving a digital value server (a.k.a. spend interface or shopping cart interface for a web based accounting system), and (b) the internal interface between a front-end web server and a back-end accounts server for the aforementioned value server.
This description concentrates on the core payments activity, more the domain of the merchant scenario. For usage (b) above, see the optional requests in User Requests
The mission of this project is
to create a protocol for spend interfaces and front-end/back-end interfaces that can be the standard for all web based accounting systems.
The benefit will be obtained when merchants only have to implement one and only one interface method to access many different web based accounting systems. As the cost of implementing methods is very high, no merchant implements more than a few of the available systems. By implementing XML-X as a standard interface, any given web based accounting system can quickly acquire many merchants, thus ensuring the success of both the system and the merchant.
The Primary Authors of XML-X are Erwin van der Koogh and Ian Grigg.
Contributors and Influences include
The Author of this document is Ian Grigg, with contributions from all the above.
Parts in italics are optional at some level, for implementation, client or server.
The principle model for communications is the Request / Response model. This is taken from the (unwritten) principles of Financial Cryptography (see Financial Cryptography in 7 Layers for a reasonable introduction).
The request / response model is the only reliable way to construct robust transactional systems over a network. It is based on The Coordination Problem, which examines how difficult it is to achieve closure over a network.
Request / response stands alongside other models such as the datagram model (which is a simple packet travelling from one node to another, only) and the connection-oriented model (better known as TCP/IP) used by most other systems. Both of these other models lack full reliability as they are based on different software engineering principles. It is beyond the scope of this document to further clarify these criticisms.
In the request / response model, a client sends a request to the server. A response is returned when and if, and only when and if, the request is fully dealt with. If the request is not able to be dealt with, an error result may be forthcoming, but not if the error is uncertain. For particular example, if there is uncertainty as to whether a request has made it to a server, no error will be returned. In practice, this means that there is no separate concept of reliable delivery, only a concept of reliable execution. There are no intermediate store and forward results, and these are frowned upon to a greater or lesser extent because of their uncertain meaning.
So, how in practice does a client deal with this? A client must expect 3 potential results to any request: success, failure, or unknown. Note that this is the same for all other models, we just have more cases in the latter class to deal with.
In the case of unknown, a strategy needs to be developed. For example, with a payment transaction, a unique identifier must be inserted into the request packet, and the server must guaruntee that this is executed once or none times. Further, in the event of an unknown result, the client may seek to re-do the transaction with a null or zero valued transaction in order to cancel the transaction.
Note that all of these decisions should be taken at the highest level. That is, this protocol does not promise these decisions, and gives relatively little help beyond guidance. Specifically, the unique identifiers to achieve idempotency need to be selected by the higher layers and are unique in semantics to each type of request. Unlike session protocols such as TCP/IP or SSL, a financial cryptography protocol does not attempt to provide protection for ordering, repeats and other issues at the layer seen in XML-X.
It takes a while to wrap ones head around request / response. The payoff occurs when reliable transactions are considered, as it most closely aligns the protocol with the inherent unreliability of the Internet. There are no false signals to deal with, and everything seen is raw and direct.
Implementation Note: request / response still needs to be transmitted and received across the net. In practice this means that these two packets are layered across some other protocol. Thus, request / response is commonly layered over TCP/IP connections or over UDP datagrams. But be careful to understand - and implemnt - the RR model as independent of the underlying transport. Ideally, build your code base to cope with either underlying model (which will stand you in good stead when moving to a later optimisation phase).
In XML-X, there is a separation between the unit of authentication, the unit of account, and the unit of currency. That is, at the level of the protocol, users are considered to be orthogonal entities to accounts, and currencies are orthogonal to both of them. There are many-to-many mappings possible between these entities.
In each of the above three, there is the <Entity>, being the full description of the entity, and there is a unique identifier, <EntityId>. For example, <Currency> fully describes the nature of the value, and <CurrencyId> will uniquely define the entity <Currency>.
This is more fully described in User Management. Follows is a brief description of the parts relevant to the core part of XML-X.
The user is the unit of authentication to accounts. When the user is created, a method of authentication is established. This method (or methods) should be used in support of each request to access the assets of the user, as described below.
The user is uniquely identified in the system by the <UserId> tag, and full details on each user are obtained from the <User>. See User Management.
Each user might access multiple accounts. For example, in a more sophisticated system, a user can add differently named accounts for different purposes (e.g. "household" and "business"). In many systems, there will only be one account per user. The mapping from users to accounts will therefore be hidden from the user themselves, but the linkage will still exist at the level of XML-X requests.
Any given account is different to the user. The precise linkage is undefined in the XMLX specification, and may range from "one-user-one-account" all the way to "many-to-many" linkages that permit users to share accounts.
An account is uniquely identified in the system by the <AccountId> tag. For the details of any such account, a fuller description can be described by the <Account>. See User Management.
A Currency is uniquely identified within the system by its <CurrencyId> tag, and is fully described by the <Currency> tag. This latter tag includes details on displaying the Currency such as how many decimal points pertain, and what symbols to use.
Accounts are further divided into subaccounts where each subaccount holds one currency. The definition of a subaccount is such that a transfer from one account to another is always between the same subaccount (same currency).
That is, when Alice transfers USD to Bob, it comes from her USD subaccount and always goes to Bob's USD subaccount.
To specify a subaccount uniquely, one needs both the <AccountId> and the <CurrencyId> tag.
Most tag content types are loosely defined or undefined. In contrast, the following types are tightly defined:
These are tightly defined as they have a material impact on the meaning of the payment. Confusion with amounts and times can cause disputes and raise costs.
Other content types are not tightly defined within XML-X. This relates to different implementations not agreeing on such things as the format for an account identifier or a memo field.
This section bears much in common with BDG.
The basic process is to send the <Transfer> to the server, wrapped in its <TransferRequest>, then receive back the <Receipt>, wrapped in its <TransferResponse>. The <Receipt> is a wrapper around the <Transfer>, which adds in the server details of time of transaction, and server's transaction id.
A <Transfer> includes all the details needed to complete the transfer, except the authentication requirements (that part is deferred to a higher layer, the request, as many possible mechanisms are possible).
<Transfer> <Payee> E3491 </Payee> <Payer> 34201-543 </Payer> <CurrencyId> USD </CurrencyId> <Amount> 1594 </Amount> <TransferId> P9348235 </TransferId> <Memo> French Roast 1kg </Memo> <FeeHint> add </FeeHint> </Transfer>
The format and content of the Id tags (Payee, Payer, CurrencyId, TransferId) is open, as most systems have different ways of describing such entities as accounts and currencies.
<Amount> is a single integer number, with no decimal points. It is the number of the smallest unit that is expressable in the currency. For example, in a USD dollar currency, <Amount> would probably be in cents, in which case, the above example would refer to $15.94, or fifteen dollars and ninety four cents.
The currency is fully described in a <Currency> tag, including the number of decimal points that apply.
Implementation Strategies. You can request this using an optional <GetCurrency> and extract the number of decimals from there. If your server does not support the optional <GetCurrency> request, it is a good idea to acquire the <Currency> by other means (including typing it in!). Then, interpret it by recovering the object and requesting decimal support from it. Hard-coding the number of decimals is the worst strategy as it will be tricky to add other currencies later on.
The currency is more fully discussed above. Amounts are discussed in a further document, Amounts.
<TransferId> is optional. It indicates an idempotent identifier for this transfer, which means that the server must treat any two transfers sent with the same <TransferId> as the same transfer (within the Payer account).
A client can set <TransferId> to stop replay failures and attacks. When set, the first copy bearing that Id to make it to the server is the one that is used, all following copies are rejected.
The rejection should be signalled by means of an error that signals the "already" status. How race conditions are resolved is not defined.
<Memo> is the main extension point for the Transfer. It allows any information to be stored within this tag. The server does not interpret nor act upon any information kept within the <Memo>. The entire tag is optional.
Implementations MAY accept arbitrary but correct XML in the <Memo>. Any tag in any namespace is supported and SHOULD be passed on within the Transfer, except when the contents are unreasonably large. Any such limitations should be clearly documented.
Note: This proposal may have security ramifications. It is currently unimplemented.
The following are suggested:
<Description>. The textual reminder that the user types in for a payment. This field is generally shown in History displays.
<TargetId>. An additional reminder for the user that records who the recipient user was intended to be at the time of the transaction. This field is generally shown in History displays. (Note that only the <PayeeId> is normally recorded, and many users potentially have access to that account).
<MerchantOrderId>. A token that a merchant might provide to assist in matching up this payment to an outstanding retail trade.
<MyOrderId>. An indexing system for any outgoing payments by the user, generally used in companies where all orders being placed have an order number ascribed to them. (Many suppliers will not act or treat an order as valid until they receive an Order Id from the user.)
As well as supporting any namespace (tags like the above), an implementation SHOULD preserve the whitespace arrangment in the <Memo>.
A likely additions would include the rate of exchange and amount of an alternate currency, for recording a user's intention to settle a debt in one currency but pay in another.
Currently, no implementations conform to the Memo's pass-thru XML requirement. Only non-XML text is accepted by current implementations.
In response to each <TransferRequest>, the accounting server returns a <Receipt> that includes the entire contents of the <Transfer>, along with the authenticating userid, the server's transaction identifier, the date/time of effect, and any signature elements.
<Receipt>
<ReceiptId> R20349143 </ReceiptId>
<Time> 123456789012345678 </Time>
<Transfer> all of the above </Transfer>
<UserId> Erwin </UserId>
<PayeeAmount> 4500 </PayeeAmount>
<PayerAmount> 4597 </PayerAmount>
<Signature type="DSA">
Blob
</Signature>
</Receipt>
The following sections expand on each of the tags that the server adds to the Transfer.
The reasons for returning the entire <Transfer>, encapsulated in the <Receipt> include,
The authenticating user is recorded in <UserId>. As many users may access a given account, it is important to record the one that authenticated this transaction.
<Time> is the timestamp that the server places on the transaction. In general this timestamp will be the definitive time that the transaction has occurred, and this will occasionally become very critical in disputes.
XML-X Time is "win32" time. This gives a handy resolution for the future. Implementations that read and write the XML-X packets should convert local times to this format, and transmit and receive only this one time base.
The format of "win32" time is the number of 100ns ticks since 1601. In base 10 notation, it is 18 digits long.
See Notes on Time for more details on why this timebase is chosen, and the important limitations.
In the simplest case, a client instructs the server to transfer some amount, X, and the server transfers that very amount, X.
But, the real world is rarely that simple. Issuers of money have a business to run, and they normally finance their operations out of fees taken for services delivered. Transactions often attract such fees.
Fee structures are very complex. Rather than attempt to document the algorithms and options of fees, XML-X provides an ability for the server to record the useful and pertinant results of the transfer.
Where it differs from the <Amount> in the <Transfer>, the server has the option of indicating the amount deducted from the paying account in <PayerAmount>. Likewise, if a different amount to that which was instructed ends up in the receiving account, <PayeeAmount> records that final received value.
In the following example, the client can determine that even though an amount of 4523 was instructed, the server made some adjustments. By inspection, the server charged the client's account 74 units as a payer fee, and charged the receiver an additional 23 units as a payee fee.
<Receipt>
. . .
<Transfer>
. . .
<Amount> 4523 </Amount>
. . .
</Transfer>
. . .
<PayeeAmount> 4500 </PayeeAmount>
<PayerAmount> 4597 </PayerAmount>
</Receipt>
Any information concerning how a fee is calculated is booted upstairs to a higher layer. Also, what is not of concern here is whether the server has any right to set fees in arbitrary or flexible amounts. That's a job for the legal department, not the programmer. In XML-X, the server rules on fees, and the client accepts the server's determination.
Having defined the contents of the transaction, we can wrap that up in a request and receive the return message from the server.
<TransferRequest> <Auth> one of the below </Auth> <Transfer> all of the above </Transfer> </TransferRequest> <TransferResponse> <Receipt> All of the above </Receipt> </TransferResponse>
The intent is to achieve a single request - response cycle to achieve a payment. In order to assist matching up a response to a request, the client can add a request identifier in the rid attribute which the server must return in the response; this becomes useful in sessions, discussed below.
It remains to authenticate the request. In order to authenticate a request such as that described in the <Transfer>, above, a request always has a single <Auth> tag as its first tag. This tag supplies authentication to the server. Here are some examples:
<Auth>
<UserId> Erwin </UserId>
<Password>
TestTest
</Password>
</Auth>
The <Auth> always includes a <UserId> and usually a method of authentication. The more normal one might be the password, above, but there are other possibilities, below.
<Auth>
<UserId> Erwin </UserId>
<Signature>
Blob
</Signature>
</Auth>
<Auth>
<UserId> Erwin </UserId>
<Token>
Magic String
</Token>
</Auth>
This document does not really describe them in detail as they can vary amongst implementations. Likewise, it is assumed that the various signature types includes sufficient information to establish the credibility of claims.
Issues such as confidentiality are not dealt with at this layer, it is assumed that SSL or the like is used.
(For access to all the requests described below, you will need to establish the <UserId> and the method of authentication.)
An Auth tag might leave out the authentication method, in order to request non-priviledged information such as statistics or currency information.
In this case, the UserId must still be present as a tag. It is an implementation choice on behalf of the server as to whether an empty UserId is acceptable.
Experimental.
An XML-X system may support token withdrawals and settlements. Tokens are elements of data that can be moved hand-to-hand between different users without any other protocol support.
In the case of the token withdrawal, a <Withdraw> will be included in the <TransferRequest> instead of a <Transfer>. For settlement of the same token, a <Settle> will be included in a <TransferRequest>. Each of these contains a <ValueToken> that contains the actual token data.
The <Receipt> returned will then include the applicable <Withdraw> or <Settle>, and in the case of withdrawal, an additional <ValueToken> will be added to include the token so withdrawn.
There is no guarantee that a <ValueToken> withdrawn from the system can be settled by this or any other operation, nor whether any type of token technology is supported. That is fundamentally up to the rules of the token technology and the implementation. Also, when a token is withdrawn, this moves that value out of the system managed by the server, so accounting may be impacted generally. More specifically, lost tokens are problematic, and need to be considered.
A <Withdraw> includes all the details needed to create a new token and extract the value for it out of the account. This includes a <ValueToken> which shows the type of token required, and may include any proto-tokens (needed for various forms of tokens).
<Withdraw> <Payer> 34201-543 </Payer> <CurrencyId> USD </CurrencyId> <Amount> 1594 </Amount> <ValueToken type="sox3"> </ValueToken> <Payee> sox:123abc456def@ </Payee> <TransferId> P9348235 </TransferId> <Memo> French Roast 1kg </Memo> <FeeHint> add </FeeHint> </Withdraw>
The types available are not strongly by XML-X, excepting transfer which is reserved. These types have been indicated: sox2, sox3, soxbrn, soxchaum, epoints, hashcash, gmtokens . Depending on the technology of the token, the type may be empty as above.
The token comes back in the Receipt within its <ValueToken> tag. Once so returned, it should be settlable in a <TransferRequest> by providing it in the <Settle> below.
<Receipt>
<ReceiptId> R8374543 </ReceiptId>
<Time> 123456789012345678 </Time>
<Withdraw> all of the above </Withdraw>
<UserId> Iang </UserId>
<ValueToken type="soxbrn">
098zyx765wvu543tsr210qpo
</ValueToken>
</Receipt>
A <Settle> includes all the details needed to settle a token into a user's account.
<Settle> <Payee> 34201-543 </Payee> <CurrencyId> USD </CurrencyId> <ValueToken type="sox2"> -----BEGIN SOX MESSAGE----- Comment: SOX 2.5 webfunds.org Version: 2.0.3 mOPkmxA7QJVhu3XEOmusGLVZ5ItqjxO2+hQBxZCTOVh9YBQB xagmMbz9YA== =DDzr -----END SOX MESSAGE----- </ValueToken> <Amount> 1594 </Amount> <TransferId> P9348235 </TransferId> <Memo> French Roast 1kg </Memo> <FeeHint> add </FeeHint> </Settle>
Most token technologies should be self-identifying so many of the normal transfer fields will be optional. If they are supplied, they must match the token details, or the server should reject the transfer.
As a rule (encouraged by XML), each packet has its own request and reply. One exception to this rule is if there is an error.
On error, the server returns an <ErrorResponse> which includes an error number taken from standard list of numbers, and, optionally, a short text description and a longer help definition.
<ErrorResponse errno="666" rid="token">
<Text> Wrong Password </Text>
<Additional> Check the Postit note on your monitor </Additional>
</ErrorResponse>
Thus, the receiver needs to check the first tags coming in and determine if the matching response is being returned, or an <ErrorResponse>.
On examination, the receiver can look up the error number in the standard list at < http://www.xml-x.org/errors.html >. If there is no <Text> tag supplied, the contents can be assumed to be the standard string from the list.
Similarly to the above, to get a Balance, send a <BalanceRequest> and receive back a <BalanceResponse>.
<BalanceRequest>
<Auth> one of the above </Auth>
<AccountId> 1234567 </AccountId>
<CurrencyId> Gold </CurrencyId>
</BalanceRequest>
<BalanceResponse>
<Balance>
<AccountId> 1234567 </AccountId>
<CurrencyId> Gold </CurrencyId>
<Total> 4523 </Total>
<Time> 123456789087654321 </Time>
</Balance>
</BalanceResponse>
In a <BalanceRequest>, one of <AccountId> and zero or one of <CurrencyId> can be specified.
If both are supplied, then one <Balance> is returned, including the balance for that subaccount.
If <AccountId> is supplied only, then several <Balance> records are returned, one for each <CurrencyId> that the Account has.
The <Total> is returned because the quantity may be negative. It is up to the client to deal with this possibility.
<Total negative="true"> 4523 </Total>
The quantity is to be interpreted according to the decimalisation described in the <Currency> tag.
The optional timestamp in <Time> indicates to what time the balance was prepared and is valid to. The server guaruntees that all transactions up to that time are included, if the time is supplied.
Whether the server serialises the request or parallelises it is up to it. It may be that it is faster to launch parallel requests at the client side, if the server is threaded at the request level only.
The optional timestamp in <Time> is intended to give warning of stale balances. If communications with primary accounts servers are lost, an XML-X server may choose to return cached balances.
Similarly to the above, to get a list of transactions, send a <HistoryRequest> and receive back a <HistoryResponse>.
<HistoryRequest>
<Auth> one of the above </Auth>
<AccountId> 1234567 </AccountId>
<CurrencyId> Gold </CurrencyId>
</HistoryRequest>
<HistoryResponse open="432142" close="432142" when="when">
<Receipt> Number 1 </Receipt>
<Receipt> Number 2 </Receipt>
<Receipt> Number 3 </Receipt>
<Receipt> Number 4 </Receipt>
<Receipt> Number 5 </Receipt>
</HistoryResponse>
At its discretion, the server may not provide the original receipt when returning a history response. In this case, it might drop any signature (often large and difficult for the client to check anyway). It might also enhance the Id fields by including the publicly accessible names for those Ids.
<Receipt> <ReceiptId> R20349143 </ReceiptId> <Time> 123456789012345678 </Time> <Transfer> all of the above </Transfer> <UserId name="Erwin van der Koogh"> ew01 </UserId> </Receipt>
This form avoids further lookups on each of the Id fields. The addition of names might also apply to the applicable Id fields in the Transfer (Payee, Payer, TargetId).
The following selection criteria are implemented, included as additional fields within the <HistoryRequest>.
<Search>
<Tag> ReceiptId </Tag>
<Exact> R12-12345 </Exact>
</Search>
<Search>
<Tag> PayeeId </Tag>
<Exact casesensitive="true"> Me/09876 </Exact>
</Search>
<Search>
<Tag> PayerId </Tag>
<Contains> you/1209 </Contains>
</Search>
<Search>
<Tag> Time </Tag>
<From> 123456789009876543 </From>
<Till> 123456789009876543 </Till>
</Search>
One of the above <Search> can be included in the <HistoryRequest>.
For more complex requests, see the optional searching extensions in Optional Selection Criteria.
The exact value can be specified with <Exact> . (This makes less sense with, for example, <Time> but it must be implemented).
<Contains> matches only a subset. (This makes even less sense with, for example, <Time> and the result delivered by the server is undefined.)
The casesensitive attribute specifies how to match the element.
Where case is not an issue, this option is meaningless.
A range can be specified with <From> and <Till>. This may not make sense with for example <PayeeId>, etc, but it must be implemented.
If one of the pair is absent, it is interpreted as the widest possible range. That is, if there is no <From> tag, then all transactions up until the <Till> are returned.
Any tags defined by XML-X within the <Receipt> can be specified in the searching criteria, including those within <Transfer>. Only some of the tags may be actually searched on, as a server decision.
Instead of the above <Search> tag added at the end of the <HistoryRequest>, a <CompositeSearch> may be added:
<CompositeSearch logic="and">
<Search>
<Tag> Time </Tag>
<From> 123456789009876543 </From>
<Till> 123456789009876543 </Till>
</Search>
<CompositeSearch logic="or">
<Search>
<Tag> PayeeId </Tag>
<Exact> Me/09876 </Exact>
</Search>
<Search>
<Tag> PayeeId </Tag>
<Exact> Me/09999 </Exact>
</Search>
</CompositeSearch>
<Search not="true">
<Tag> PayerId </Tag>
<Contains> You/12 </Contains>
</Search>
</CompositeSearch>
A <CompositeSearch> includes a list of <Search> tags and, recursively, <CompositeSearch> tags. Any of the included <Search> can be replaced with a <CompositeSearch>, allowing for complex searches.
NOTE: check out RegEx in Java. PHP has Posix & Perl. Perl?
The logic attribute of a <CompositeSearch> specifies how to group the elements in the list. These possibilities exist:
As well, each <Search> and <CompositeSearch> tag can have an optional attribute not="true" for reversing the effect of the search. The default is not="false".
Following on from the optional <Search> or <CompositeSearch> tag there may be a <Sort> tag added:
<Sort>
<Tag> Time </Tag>
<Tag ascend="false"> PayeeId </Tag>
<Tag> ReceiptId </Tag>
</Sort>
The first tag is the primary sort key, the second is the secondary, and so on.
Each <Tag> can have an optional attribute ascend that defines how the results are returned. The default for all tags is true, and the alternate is to descend by setting ascend="false".
The server defines the default ordering as appropriate to the type. For example, <Time> and <Amount> tags should be sorted numerically, and other tags may be sorted alphabetically.
Note that where alphabetic sorting is chosen by the server, various character sets will also effect the result.
Character sets are further discussed in Implementation Notes.