Chapter 11. Generating web services layers – Code Generation in Action

Chapter 11. Generating web services layers

11.1 The big picture
11.2 A case study: generating XML-RPC for Java
11.3 Technique: generating SOAP
11.4 Design tips
11.5 Finding a tool to do it for you
11.6 Summary

A web services layer exposes the business logic of your application to the outside world. If your business logic and security layers are well defined, the task of exporting them through web services requires some simple code to interface your API to the RPC layer and to marshal the data between the two. This process involves building a remote-access layer that is a peer of the user interface. You can use several RPC standards for this purpose. In this chapter, we concentrate on XML-RPC and SOAP, both of which are based on XML and provide tools and support across a wide variety of operating systems and programming languages.

This is just the type of work for which code generation was invented. We show you not only how to create the RPC layers for an application using a generator, but also how to build stub code for the client—in Visual Basic, Perl, Java, or C#—that will make your interface easy to use.

11.1. The big picture

The case study in chapter 1 used an RPC generator to build an external interface for the application. RPC layers are, by definition, multicomponent affairs. A server portion sits as a peer to the user interface on the server side. Corresponding to the server portion are one or more client portions, which act as stubs for the procedures implemented in the server portion. You can build multiple client portions—one for each of the possible client languages. For example, you can deliver stub APIs in Visual Basic, C++, and Java—if your customer base requires that type of diversity in language selection—by simply including additional templates in the generator. The generator and the system components it creates are shown in figure 11.1.

Figure 11.1. The RPC-layer generator from the original case study

In this chapter, we present a case study XML-RPC generator that builds both server and client code. Later, in section 11.3, we introduce a similar generator design, this one for the SOAP architecture.

11.1.1. Providing security and authentication

If your server offers weather information, you probably don’t need to worry about authenticating your clients. But more complex services are likely to need a method for determining who is accessing the server and making sure that users are who they claim to be.

Authentication of the user of the web services interface is complicated by the fact that both XML-RPC and SOAP are completely stateless. This means that every call to the interface is its own complete transaction. There is no way to bundle a set of method invocations together into a single transaction.

The two basic approaches to handling authentication in a stateless environment are:

  • Using tickets —In this model, the client first requests a ticket from the server using a specific method. The request contains the user’s authentication information (i.e., the login and password). The client then sends the ticket along with all subsequent requests.
  • Authenticating on each request —In this model, the client sends authentication information (i.e., the login and password) with each request.

Code generation is useful in either approach. The generator that builds the client and server portions of the RPC code can hide from client users both the implementation of the authentication model as well as the target API on the server.

On the client side, the generated API can have a login method that takes and stores the authentication information. Then the API can either request a ticket or send the information on each request. In this way, client code can remain the same even if the authentication scheme changes.

On the server side, the set of authenticated logins can be cached locally within the generated interface. Alternately, the authentication system in the business logic layer or application server can be used, if such a system exists.

11.1.2. Why not use a library?

A library is often a valid approach to exporting a web services layer, so why generate the layer? For C and C++, a library uses a set of structures to define a mapping layer, which in turn defines the functions that are to be exported and their arguments. Other languages, such as Java and C#, support reflection, which is a way of identifying the methods of an object at runtime. RPC libraries make use of reflection by adjusting their interfaces on the fly to match the methods of the exported objects.

Generation provides advantages in both environments:

  • When using a static-mapping library for C or C++, the maps still need to be maintained and synchronized with the application code. This type of maintenance is an ideal task for code generation.
  • A reflection library may not be compatible with your business logic interface. A generator can work in conjunction with the reflection library to create a compatible, intermediate layer.

In either case, the library will not supply you with client-side stub code that is tailored to the web services interface you provide. The generators shown in this chapter represent examples not only of building the marshalling code for the server, but also of building the client-side stubs at the same time so that the two interfaces—client and server—are kept perfectly in sync.

11.2. A case study: generating XML-RPC for Java

XML-RPC is an RPC layer on top of XML. It’s often compared to SOAP. Generally, XML-RPC is considered to be of lighter weight and simpler than SOAP.

The case study generator exports a simple library of functions through XML-RPC using the Java XML-RPC library from Apache. We chose XML-RPC for the case study because the implementation is generally smaller. However, the generation concepts are portable between the two.

Here are some terms specific to this case study and XML-RPC that you should become familiar with:

  • Target —This term applies to both classes and methods. The target class provides the web services that are exported through the XML-RPC layer. The target class is used as input to the generator to create the handler and stub classes.
  • Handler —The handler class wraps the target class and has a set of handler methods that map one to one with the exported methods of the target.
  • Stub —The stub class contains all of the methods exported by the target class. Each of these stub methods calls through the XML-RPC layer to talk to the target class on the server. The value of a stub class is that it looks and acts like a local class even though the implementation of the methods is on the server.
  • Client —The client makes use of the stub class to call target methods on the server.

Our sample target class is fairly rudimentary. The class provides functions to add numbers, subtract numbers, add strings, and perform other transformations that are meant to test all of the types that can be exported through XML-RPC.

None of these methods holds any state. Neither XML-RPC nor SOAP provides a mechanism for managing state across a session with the server. Every procedure call is considered atomic. If your business logic layer requires managing state across several method invocations to accomplish a single task, you will need to create an additional layer that provides a set of methods to accomplish each task in a completely stateless manner.

Listing 11.1 contains the code for the test service.

Listing 11.1. Test.java

  1. Adding the @rpcgen marker to the JavaDoc tells the generator that this method should be exported by the XML-RPC handler.
  2. The methods in this class are the “services” our server will provide.

11.2.1. The XML-RPC message flow

First, the client initiates a call to the server by calling the add method on the stub. The stub in turn calls the test.add method on the server. Next, the XML-RPC server invokes add on the handler object. The handler object creates an instance of the target object and invokes add. The return value is then sent back to the client class as the transaction unrolls. You can see this message flow between client and target in figure 11.2.

Figure 11.2. The XML-RPC flow for the add method

Now let’s look at what you can expect this generator to do.

11.2.2. Roles of the generator

Before you build the generator, you need to specify its responsibilities. You should also define the functions for which the generator is not responsible.

The generator should create the following:

  • The RPC layer, which includes:

    • The server code that will handle the incoming XML-RPC methods and dispatch those requests to the correct application code.
    • Stub client code that will be used by engineers building another system that talks to our system.
  • The technical documentation for the stub client code.
  • Authentication of the client.
  • Session management with the client.

The generator does not need to handle:

  • Design documentation for the application or the external RPC layer.
  • Any functionality in the application layer.

The case study generator does not address authentication and session management directly. These functions are application specific. I mention them here because authentication and session management comprise the role of the RPC layer.

At this point, you can define the architecture of the generator.

11.2.3. Laying out the generator architecture

Your generator will build code for both the client and the server. On the server side, you create handler code that will be used by XML-RPC to handle each message. On the client side, you create stub code that will look like the target class to the engineer but that will internally call to the server for every method.

For this generator, we chose to use a code munger model. The Java class should have JavaDoc markup in it to tell you which methods require export. The generator will read and analyze the Java file and create the appropriate handler and stub code.

Figure 11.3 shows the block architecture for the generator.

Figure 11.3. The XML-RPC generator builds stubs and handlers from Java code with JavaDoc markup.

In this architecture, the Java code acts as both input to the generator and as the final target of the XML-RPC system. The generator uses a set of templates to build the handler and stub Java code.

For this case study, you are only creating Java stubs. Keep in mind, though, that there is no reason your generator couldn’t build stubs in a variety of languages.

Now let’s look at the steps the generator will go through as it runs.

11.2.4. Processing flow

XML-RPC generator

Here is the process flow of the generator, in high-level steps:

  • Reads the specified Java file.
  • Tokenizes the Java text.
  • Parses the Java text.
  • Creates a buffer that will hold both the client and stub methods text.
  • For each method, follows these steps:

    • Inspects the JavaDoc for the @rpcgen tag. If it has the tag, then it follows these steps:

      • Uses the StubMethod template to build the stub method and adds it to the buffer.
      • Uses the HandlerMethod template to build the handler method and adds it to the buffer.
  • Using the stub template and the stub method buffer, builds the Java class for the stubs, and writes the result of that template to a Java file.
  • Using the handler template and the handler method buffer, builds the Java class for the handler and writes that to the appropriate Java file.

11.2.5. Building the code for the XML-RPC generator

Now you’re ready to start writing the code for the generator. Listing 11.2 shows the Ruby code for this generator.

Listing 11.2. rpcgen.rb

  1. You use CTokenizer and JavaLanguageScanner to tokenize the Java file and then parse it to find the JavaDoc comments.
  2. You use ERb to invoke the templates that build the Java code.
  3. read_java_file is the main entry point for the generator. It reads in a Java file and builds the stub and handler code.
  4. This code builds the handler and stub methods. It iterates through each of the methods in the class and checks to see if the JavaDoc contains the @rpcgen tag. If it does, the templates are added for both the handler and the stub.
  5. This code uses Handler.java.template to build the handler Java file. The template requires the class name and the methods.
  6. This code uses Stubs.java.template to build the stubs Java file. The template requires the class name and the methods.
  7. rpc_type returns the XML-RPC wrapper type for the given Java type.

For each method, you want to export a corresponding handler method. To automate this, create the template HandlerMethod.java, which builds a handler method (listing 11.3).

Listing 11.3. HandlerMethod.java

  1. This template needs to create a method of the type that you are wrapping, then call a method on it and return the output value. The out_type is the XML-RPC return type.
  2. In this section, you are creating two sets of arguments. The args array contains the arguments for this method, and call_args is the array of arguments that you will send to the method of the type you are wrapping.
  3. This code builds the method signature.
  4. This code creates the object of the type that you are wrapping, and then calls the target method and sends back the return value.

Once you have all of the handler methods, put them into a class container. Listing 11.4 contains the class container template.

Listing 11.4. Handler.java.template

Listing 11.5 shows the final output of the generator using the handler method and handler container templates.

Listing 11.5. TestHandler.java output

  1. This is the signature that the XML-RPC library looks for when choosing methods to export. If you use Double instead of double, the method will not be exported by the XML-RPC server. The generator handles marshalling between Double and double as well as the other types.
  2. Because you are wrapping a class type, you first have to create an instance of that class.
  3. Now you invoke the corresponding method on the instance of the class.

You need one stub method for each corresponding handler method that calls the server with the appropriate arguments and handles the return value. Listing 11.6 shows the template that generates the stub code.

Listing 11.6. StubsMethod.java

  1. This code builds the marshalling code that uses temporary variables to set up the parameters for the XML-RPC request. type is the type that the user wants, and rtype is the equivalent XML-RPC type. args is the array of incoming arguments to this method. elements is the set of new Java calls that create the marshalling objects.
  2. XML-RPC uses a Vector of arguments to send an XML-RPC message.
  3. This line runs the XML-RPC request.
  4. This code marshals the return value.

The stub methods are placed inside the Stubs container class, shown in listing 11.7.

Listing 11.7. The Stubs.java.template

Listing 11.8 shows the output of the stub method and stub container templates.

Listing 11.8. TestStubs.java output

  1. The stub class holds a reference to XmlRpcClient to make it easy to call the methods once the stub class is created.
  2. The first step in invoking XML-RPC is to create the argument Vector and to populate it with the arguments.
  3. The next step is to execute the specified function on the server.
  4. The final step is to marshal the return value.

To serve up the TestHandler, create a simple server, as shown in listing 11.9.

Listing 11.9. Server.java

The Client.java file is a test file that uses your generated stub class to make calls to the server. Listing 11.10 contains the code for the client test class.

Listing 11.10. Client.java

  1. This code creates the XmlRpcClient code, which connects you to the target server.
  2. The next step is to create the stub object. You give it a reference to the client object so that it can communicate with the server.
  3. Invokes methods on local stubs: from here on, the code tests the various methods on the server.

11.2.6. Performing system tests

To make sure the XML-RPC generator is functioning properly, let’s use the system test utility (see appendix B). Here is the configuration file for the unit test system:

<ut kgdir="kg">
  <test cmd="ruby -I../lp rpcgen.rb examples/Test.java">
    <out>examples/TestHandler.java</out>
    <out>examples/TestStubs.java</out>
  </test>
</ut>

The test will run the generator on Test.java and then inspect the TestHandler.java and TestStubs.java files that are generated.

11.2.7. XML-RPC resources

At the time of this writing, I couldn’t find any generators that build XML-RPC code. Instead, I have included some resources that will provide more information about the XML-RPC protocol:

  • You can find the Apache library used in this case study at http://xml.apache.org/xmlrpc/.
  • www.xmlrpc.org is the official XML-RPC home page.
  • Programming Web Services with XML-RPC, by Simon St. Laurent, Joe Johnston, and Edd Dumbill, (O’Reilly, 2001), is an excellent book on programming both client and server code for the XML-RPC protocol.

11.3. Technique: generating SOAP

XML-RPC and SOAP are remote procedure call layers built on top of XML. XMLRPC is generally considered a simpler approach, while SOAP has wide industry backing from Sun, Microsoft, and IBM.

SOAP has related standards in addition to the RPC standard. One of the most important is the Web Service Definition Language (WSDL). WSDL describes your web service to the outside world. A WSDL description of your service is a valuable tool for the client, but it is a pain to maintain manually. Thus, another output of the SOAP generator should be the WSDL description of the service. In this section, we discuss how the architecture for a generator builds a web services layer based on the SOAP protocol.

11.3.1. Roles of the generator

The roles of the SOAP generator are the same as those of the XML-RPC generator described in 11.2.3.

11.3.2. Laying out the generator architecture

Our SOAP generator follows the tier generator model. As input, the generator takes an interface specification. This specification defines what classes and methods you want to export. Using a set of templates, the generator builds the WSDL file as well as the server code and client stubs. Figure 11.4 shows the architecture of the SOAP generator.

Figure 11.4. A generator that builds SOAP WSDL, stubs, and an interface on top of an API

Now let’s look at the steps the SOAP generator will take as it executes.

11.3.3. Processing flow

SOAP generator

Here are the high-level steps for the process flow of the generator:

  • Reads the interface specification and stores it locally.
  • Builds the WSDL file using the WSDL template.
  • For each class in the interface specification, follows these steps:

    • Builds the SOAP server class using the server template.
    • Builds the stub class using the stub template.

At this point the generator will have built both the server and client code for the SOAP layer.

11.3.4. SOAP resources

We’ve listed a few SOAP resources to help you learn the fundamentals of the SOAP protocol and web services programming:

  • You can learn about Apache SOAP at http://xml.apache.org/soap/index.html.
  • Java and SOAP, by Robert Englander (O’Reilly, 2002), covers both client and server coding for the SOAP protocol.
  • Programming Web Services with SOAP, by James Snell, Doug Tidwell, and Pavel Kulchenko (O’Reilly, 2001), covers both client and server aspects of the SOAP protocol with a number of languages.

11.4. Design tips

Here are some tips that will make it easier for you to generate the web services layer:

  • Keep the API atomic —The “O” in SOAP stands for Object, but neither SOAP nor XML-RPC is in fact object-oriented. Both layers are strictly function based and are completely stateless. No session state is held between calls. If your business logic layer is completely stateless, you should have few issues exporting it as a web service.
  • Design a consistent API —An API that uses consistent method and variable naming is significantly easier to parse with a generator because it reduces the number of special cases that have to be handled.

11.5. Finding a tool to do it for you

Example generators that build SOAP for C++ and Java include:

11.6. Summary

A web services layer is another way of exporting the functions that the business logic layer already provides. This export layer mapping and the stub code for clients can easily get out of sync with each other or with the application layer if they are maintained by hand. Generation provides a solution that creates a consistent, well-maintained web services layer as well as stubs for the client. Using a generator can save you time and money and give your engineers the freedom to work on business-critical features.

In the next chapter, we look at generators that can build the business logic layer for you.