Slimmer XML is faster XML

Extensible Markup Language (XML) tags do not need to be human-readable. In fact, not titling your XML tags in a manner that would make sense to strangers can save a significant amount of storage space and even speed transmission time, advised John King, head of the training firm King Training.

King spoke at the Independent Oracle Users Group's Collaborate 2009 conference in Orlando, Fla. this week. In his presentation, he offered a number of tips of improving the performance of systems that must transfer XML data—most of them around clipping the size of the XML documents themselves.

Cataloguing data with XML has gone a long way toward easing the movement of data among different systems, thanks to the universal use of character streams. The documents that XML can produce are quite verbose though. A peculiar trait of XML is that it was designed to be human-readable, when, in fact, humans rarely read XML documents.

In XML, every bit of data gets its own tag, and every tag is buried in a hierarchy of additional tags to add context around that data. While all the chattiness may not seem harmful when the amount of data you're sharing and storing is comparatively light, "If you transmit 30,000 messages an hour, it adds up," King said.

One thing that can be done is to shorten the tags, technically called elements. An element has two parts—the start tag and the end tag. Each is basically a string of identical characters enclosed within brackets. A start tag is placed at the beginning of the data being encapsulated, and a stop tag is placed at the end of that data. The only difference between the two is the stop tag has "/" character before the string of characters within the bracket.

Elements can be as long as the developer wants, which means they can be as descriptive as possible, such as "<TodaysSalesItems>." Anyone could read that and knows what the data in between the tags has something to do with today's sale items.

But such description is entirely unnecessary because XML markup is rarely read by humans while it is being processed anyway, King reminded the audience. So a tag such as "<TodaysSalesItems>" could be shortened to "<tsi>" thereby saving 32 characters for every time this data element is used (16 characters for the opening tag and 16 for the closing tag).

Many organizations have long had abbreviation dictionaries, perhaps leftover from the days of punch cards when memory was a scarce resource. Such dictionaries offer previously-agreed upon short names for commonly used titles. They could be used as a basis for short element titles, King advised.

XML documents can be further shortened by using attributes whenever possible, instead of elements. Attributes are added on to elements to further clarify the meaning of a particular piece of data, which can save the need to develop a separate element for just some aspect of data. The only drawbacks to attributes are that they can be only be used for a single element and that they can not be further parsed, King said.

Cutting down on white spaces in a document also saves space. Leave out the new line, tab or blank space commands cuts bytes from the document. The data may not be as easily readable for the humans, as all the data is mushed together in a single line, but it is just as easily read by the machine.

Another place to look to shave bytes is to strip out all the unneeded sections of the Character Data (CDATA) section of the document, King advised. CDATA is the part of the XML document that is not parsed. It usually is used to stash notes about the document itself. While this is helpful for someone trying to understand the information, it makes no sense to include all CDATA information in each transmission of the information.

In order to show the storage and transmission savings that could be enjoyed by these techniques, King took a sample of XML'ed data from Oracle documentation, and substituted in shorter element names. He reduced a set of tags that was 313 characters in length to a set 215 characters in length, which is about two-thirds the original. He also made a set where he substituted attributes for some elements, reducing the size of the set even further, to 179 characters. He then encoded 30,000 records with each set of tags, so that they could be compared in terms of size and how quickly they could be committed to a database.

The regular data set, in a flat-file format, was 39.4 megabytes (MB), while the one with the shorter tags was 28.9 MB. The set with the attributes was 27.3 MB. Using a standard laptop with 4 gigabytes of working memory, it took 29 seconds to store original set of data in a database. It took 23 seconds to store the shortened set, and 21 seconds to store the set with attributes.

The only quirky finding King came across was that once in storage, the data set with attributes took up more space (20 MB) inside the database, than either the dataset with the shortened tags (11 MB) or the one with the original tags (14 MB).

King also noted that, when it comes to storing data in a database, the most efficient method, space-wise, is to shred the data, or pull each data out of its XML-tag encasement and store in a separate database field. Using this approach, King found that all three datasets took exactly the same amount of space in the database—8 MB. It took 11 minutes to shred each data set, however. So shredding saves space but consumes time, King noted.

While the speed of XML processing can be picked up by slimming down the XML data itself, King also offered a number of other ways to speed XML data transfers. Of course, vendors offer a wide range of XML accelerators, or appliances that strip the XML tags as they are sent from the data source and add them back in at the destination. This approach reduces the flexibility that comes with using XML, however. Another approach is using the JavaScript Object Notation (JSON), a lightweight data-interchange format.

One thing to take a look at is if the data is being validated against an XML schema, and, if so, if it is necessary. By validating an XML document, you are checking to see if it is in the appropriate structure, as defined by its schema. While this is a valuable function when receiving data from other parties, checking data when it is being moved about internally, however, consumes uneccessary processing power.

Also, King advised to look at what is being transmitted and question if all the elements need to be transmitted. For instance, a description of some item that is being ordered through a parts requisition system does not need to be included—only the part number might be necessary. In many cases, the complete set of information is sent in order to validate the data-mark-up. XML parsers can not validate XML files unless the entire structure of that schema is in place.

"Being well-formed eats our lunch, in terms of performance," King said.