Exploring E4X with Ruby

by Jack Herrington
08/12/2004

E4X is a new standard for XML access in ECMAScript (JavaScript). Recently, I had the good fortune to meet some folks on the steering committee and to sample a demo. The idea is simple -- XML access using SAX or DOM is too difficult, so make it easier. Take this simple XML as an example:

<transactions>
   <account id="a">
       <transaction amount="500" />
       <transaction amount="1200" />
   </account>
   <account id="b">
       <transaction amount="600" />
       <transaction amount="800" />
       <transaction amount="2000" />
   </account>
</transactions>

With DOM, you would read the tree into memory and then use methods such as getChildren to fetch all of the nodes of the root, and then traverse those nodes using iterators and more methods. Often, you would also need to write code simply to avoid the whitespace nodes inserted into the tree to preserve formatting.

There's nothing inherently wrong with the DOM API. The problem is that the API is heavyweight and often obscures the algorithm being implemented. Take this example, which reads and traverses the example XML file in Ruby using the REXML API:

require 'REXML/Document'
out = {}
doc = REXML::Document.new( File.open( 'test_data.xml' ) )

doc.root.each_element( 'account') { |account|
   out[ account.attributes[ "id" ].to_s ] = 0
   account.each_element( 'transaction' ) { |trans|
       out [ account.attributes[ "id" ].to_s ]
			+= trans.attributes[ "amount" ].to_s.to_i
   }
}

p out

REXML is an excellent XML API that is very easy to use. However, it still obscures the basic algorithm, which in this case sums up all of the amounts by ID and print the result. This program generates this output:

{"a"=>1700, "b"=>3400}

It's not much to look at, but it is the right answer. The sum of account a is 1,700 dollars and account b is 3,400.

If we step back a little bit and look at XML in the large, isn't there a better way to read and write XML?

A Little XPath

One possibility is to use XPath, which make it easier find nodes in the tree. An example use of XPath that is similar to the original code is:

require 'REXML/Document'
doc   = REXML::Document.new( File.open( 'test_data.xml' ) )
total = 0

REXML::XPath.each( doc, '//@amount' ) { |amount| total += amount.to_s.to_i }
print "#{total}\n"

In this case, we are just totaling up all of the amounts into a single number. (We'll talk a little more about XPath later.) Suffice it to say that even with XPath, working with XML doesn't feel like working with data objects.

E4X

The E4X working group obviously thought so, and they figured that ECMAScript was the right language to start with. The resulting standard extends the ECMAScript language with syntax-level extensions for XML, which turn a DOM into a "dot notation"-capable data structure. For example, this code, in E4X ECMAScript:

var id = doc.account[0].id;

sets the id to a, assuming that the doc variable contains the example XML above.

You can use standard iterators such as for each on the XML variables, and you can set a variable to XML as easily as you can read it. For example:

doc.account[0].id  = "foo";

would set the id. This code:

doc.account[0] = <account id="a"><transaction  amount="200" /></account>;

would rewrite that entire first account node with new XML. Notice how you can even write XML as clear text in the code.

The great value of E4X is that it opens up XML to a wider audience by making it simpler to use. For an XML fan like me, that is music to my ears. How can you use it today? E4X is still in-process, and there is only one example implementation on top of the Rhino JavaScript engine.

E4X for Ruby

Perhaps I could use Ruby, my favorite scripting language, to implement something close to E4X. Why Ruby? For several reasons:

• The excellent REXML API for reading and writing XML makes a fine starting point.
• Ruby supports extensive operator overloading.
• Ruby supports overriding the behavior of the language when a method is not found on a class, which makes it easy for a class to mold itself to the structure of the XML data.
• Ruby's latent typing will make it a lot easier to wrap and contain the elements of REXML I will use.
• Ruby is cool.

Of course, it's this last reason I like the best.

Ruby is very similar to Perl and Python. Its major advantage is its readability. It's often been called "executable pseudo-code." To learn more about Ruby, check out the official Ruby site as well as Dave Thomas' and Andrew Hunt's excellent book Programming Ruby. The book is available for free online.

Let's jump in and see what we can do to make reading XML easier with Ruby.

Reading XML the Easy Way

To test how much we have simplified reading, I will use the original amount-totaling example on the test data file and rewrite it to use a new API:

out = {}
doc = NodeWrapper.new( REXML::Document.new( File.open( 'test_data.xml' ) ) )

doc.transactions.account.each { |account|
   amount = 0
   account.transaction.each { |item| amount += item._amount.to_i }
   out[ account._id ] = amount
}

p out

Instead of calling each_element, I can now just use the node names in the XML as if they were attributes of the class. How do I do that? By wrapping REXML nodes with my own class:

Figure 1. Wrapping REXML nodes

This wrapper will then allow access to the node using the dot notation syntax. Here is the Ruby code for the wrapper:

class NodeWrapper
   def method_missing( name, *args ) 
       name = name.to_s
       if ( name =~ /^_/ )
           name.gsub!( /^_/, "" )
           return @node.attributes[ name ].to_s
       else
           out = NodeListWrapper.new()
           @node.each_element( name ) { |elem|
               out.push( NodeWrapper.new( elem ) )
           }
           return out
       end
   end

   def initialize( node )
       @node = node
   end

   def to_s() @node.to_s; end

   def to_i() @node.to_i; end

end

That was easy, but what does it mean? The trick is in the missing_method class. Ruby is a dynamic language, so it doesn't require a class to define all of its methods at compile time. In fact, there is no compile time at all. When Ruby attempts to call a method on an object, it first looks up the method directly. If it's defined, then it calls it. If not, then it looks in the base class, and so on. If there are no methods defined with that name, it calls method_missing with the method name and the arguments.

This is where our code comes into play. We look for the names of child nodes and attributes and make them appear as real methods by overriding missing_method.

Why does that work for attributes? One would think that in order to use this missing_method process, that the client code would have to read:

doc.transactions().account().each

However, in Ruby everything is a method call. Calling the method transactions without the parentheses is the same as calling it with them. That's why the simple dot notation works:

doc.transactions.account.each

Not everything is so simple. Notice that we can run the each method on the account attribute. Why? Because we don't return a node. We actually return a node list that wraps an array. This is an array of nodes:

Figure 2. An array of nodes

The code for the node list looks like this:

class NodeListWrapper < Array
   def method_missing( name, *args )
       name = name.to_s
       self[0].send( name, args )
   end
end

The trick here is to make the default method lookup apply to the first element of the array. This is how:

doc.transactions[0].account[0]._id

is equivalent to:

doc.transactions.account._id

I compromised a little by using the underscore to hint the system as to what is an attribute versus a node, but I think it's a small price to pay.

The next step is to enhance the model to handle writing as well as reading.

Pages: 1, 2

Next Page