Chapter 13. More generator ideas – Code Generation in Action

Chapter 13. More generator ideas

13.1 Technique: maintaining header files
13.2 Technique: creating DLL wrappers
13.3 Technique: creating wrappers for external languages
13.4 Technique: creating firewall configurations
13.5 Technique: creating lookup functions
13.6 Technique: creating macro lookup tables
13.7 Summary

Generators do not need to be large and elaborate affairs. This final chapter is intended to give you some ideas for creating small generators that can help you on a day-to-day basis. The examples we present take only a day or so to write but can reduce your workload by days or even weeks, depending on the generator’s target.

13.1. Technique: maintaining header files

One reason to choose Java over C++ is its simple maintenance of the header file. Maintaining the header file may seem trivial, but if you’ve ever had to recompile a huge project because you forgot to add one small method, you’ll appreciate this feature. The generator we describe in this section makes life easier on C++ engineers by creating and maintaining C++ header files derived from the implementation files.

Let’s look at what this type of generator does.

13.1.1. Roles of the generator

The header generator is responsible for:

  • Creating headers for classes contained entirely in single implementation files and including some extra markup to define fields.

The generator does not take responsibility for these tasks:

  • Building C++ templates.
  • Building technical or design documentation for the classes.

Now let’s look at the architecture for the generator.

13.1.2. Laying out the generator architecture

The generator reads in the CPP file and then either reads and alters or creates the new HPP file directly. This, among other reasons, makes the tool ideal for integrating into an IDE. When using an IDE, you can run this generator as an external tool whenever you add a method to the CPP file. The generator receives the pathname of the current file automatically, scans the file, and then adds the method to the HPP file. After it alters the HPP file, the generator prints a short report detailing the number of methods that it added to the HPP file; this report appears in the output window. Figure 13.1 is a block-flow diagram of this simple generator.

Figure 13.1. A generator that builds HPP files from CPP files

Now let’s look at the high-level process steps this generator runs through.

13.1.3. Processing flow

C++ header generator—from scratch

Two program flows are possible. The first is when you just rebuild the HPP file from scratch each time. In that case, the generator follows these steps:

  • Reads the CPP file.
  • Tokenizes the code.
  • Parses the code tokens to find the method signatures and the comments for additional information, such as public, private, protected, and default values.
  • Builds the new HPP contents using the HPP template.
  • Backs up the original HPP file.
  • Creates the new HPP with the new contents.
  • Prints the results of the generation cycle.

13.1.4. Processing flow

C++ header generator—with existing methods

If you decide to merge in the new methods with the existing methods, the generator follows these steps:

  • Reads the CPP file.
  • Tokenizes the code.
  • Parses the code tokens to find the method signatures and uses the comments for additional information, such as public, private, protected, and default values.
  • Reads the HPP file.
  • Tokenizes the code.
  • Parses the code tokens to find all of the method signatures.
  • Compares each code signature against the signature in the CPP file to see if any alterations or additions occurred.
  • In the case of alterations, the generator builds the signature, uses it as a search in a regular expression to find the original, and replaces it with the new signature.
  • In the case of additions, the generator finds special comments that denote the public, protected, and private portions of the class definition and adds the method directly after the comments.
C++ header generators

An example generator for C++ header generation and maintenance is ClassCreator, which builds C++ implementation files from headers (http://sourceforge.net/projects/classcreator/).

13.2. Technique: creating DLL wrappers

A classic method of writing reusable code under Windows is to create DLLs with your C++ code. The problem is that the DLL interface is a functional and not an object-oriented interface. This requires the object interface to be flattened into a procedural interface.

For each method on the object you want to export, you create a function in the DLL interface. The first argument to the function is the object pointer; the rest are the arguments for the method. The body of the function casts the object pointer to the object type and invokes the method call; it then sends back the value returned by the method.

13.2.1. Roles of the generator

Before developing the architecture for the DLL wrapper generator, you need to specify the role that the generator will play within that architecture. This generator will be responsible for:

  • Managing the DLL interface for specific classes that are to be exported from the DLL. These classes must follow the class design style required by the generator.

This generator does not take responsibility for:

  • Exporting templates, C functions, or C++ classes that do not follow the class construction guidelines.
  • Merging the DLL interface from the class with any other interfaces.
  • Creating technical or design documentation for the interface.

With these responsibilities in hand, you can define the architecture of the generator.

13.2.2. Laying out the generator architecture

The generator takes the HPP file for the target class as input and generates the DLL wrapper using a wrapper template. Figure 13.2 shows a block input/output flow for the DLL wrapper generator.

Figure 13.2. A generator that builds DLL wrappers

13.2.3. Processing flow

DLL wrapper generator

Here are the process steps for the DLL wrapper generator:

  • Reads the HPP file.
  • Tokenizes the class definition.
  • Parses out the public methods. Optionally, it uses comments to define which constructor to use and what public methods are to be exported.
  • Passes the exported methods on to the DLL template.
  • Uses the output of the DLL template to create the wrapper .c and .h files. Depending on how you design your layer, you may also want to create a DLL descriptor file.
Generators for building DLL wrappers

General-purpose DLL wrapping generators are still somewhat rare. An important one is the Simplified Wrapper and Interface Generator, or SWIG (www.swig.org), which can be used to wrap DLL wrappers around C++ classes under Windows.

In the next section, we look at generators that wrap scripting languages.

13.3. Technique: creating wrappers for external languages

One of the great things about scripting languages is that whenever you get a performance bottleneck you can migrate a portion of functionality into C and then use the functions as an external library to the scripting language. Each scripting language has a different way of doing this, but all are somewhat cumbersome. The generator you’ll build next solves this issue by reading the engineers’ existing interface files and creating the wrappers automatically.

13.3.1. Roles of the generator

The role of the wrapper generator is similar to the DLL interface generator in section 13.2. This generator has one main responsibility: managing the wrapper for specific classes that are to be exported. These classes must follow the class design style required by the generator.

Here’s what the generator doesn’t do:

  • Export templates, C functions, or C++ classes that do not follow the class construction guidelines.
  • Merge the wrapper generated for the classes with any other interface wrappers.
  • Create technical or design documentation for the interface.

Let’s now turn our attention to the architecture.

13.3.2. Laying out the generator architecture

This architecture closely resembles the generator architecture for the DLL wrapper generator. Figure 13.3 shows the input and output flow of the language interface generator.

Figure 13.3. A generator that builds the external wrappers that sit between scripting languages and C++ code

The generator takes the HPP or .h file for the target code as input and, using one or more templates, builds the wrapper for the external language.

13.3.3. Processing flow

External language wrapper generator

The process flow is fairly language-specific. However, we have included a simple skeleton flow here so that you can see the general steps, regardless of your chosen language:

  • Reads the HPP file.
  • Tokenizes the class definition.
  • Parses out the public methods. Optionally, the generator uses comments to define which constructor to use and what public methods are to be exported.
  • Passes the exported methods on to the templates.
  • Uses the output of the templates to create the output files.
Generators for building wrappers for external languages

The Eiffel Wrapper Generator, or EWG (http://sourceforge.net/projects/ewg) is the only example we could find of a generator that builds external language wrappers. The tool builds glue code between Eiffel and C libraries.

13.4. Technique: creating firewall configurations

Firewalls from different vendors have different configuration options and syntax. If you could maintain your firewall rules at an abstract level, then when the hardware changes, you could change your generator to implement the new rules on the new hardware without having to modify the rules. In this section, we build a firewall configuration generator.

13.4.1. Roles of the generator

You begin by defining the roles of the generator. This generator has one main responsibility: creating the complete firewall configuration file.

The generator is not responsible for:

  • Documenting the configuration.
  • Deploying the configuration.
  • Configuring devices outside firewalls.
  • Building non-firewall configuration files within a firewall device.

Next you need to define the architecture of the firewall generator.

13.4.2. Laying out the generator architecture

The firewall generator reads in the rules for the firewall from an XML description and builds setup files using a set of vendor-specific templates. If your network setup is complex, you may want to have another file that defines your network devices so that the generator can determine the templates it should use for the individual rules. Figure 13.4 shows the block architecture for the firewall configuration generator.

Figure 13.4. A generator that builds firewall definition files from a set of rules in XML

Now let’s walk through the steps this generator takes as it does its work.

13.4.3. Processing flow

Firewall configuration generator

Here is the process flow for the generator:

  • Reads in the rules from the XML file and stores them locally.
  • Normalizes the rules internally and replace any defaults with their real values.
  • Invokes the vendor-specific template files and passes them the rules.
  • Stores the output of the generator into the setup text files.

In the next section, you’ll see a simple generator that saves time by creating lookup tables for you.

13.5. Technique: creating lookup functions

There are times when you can improve the performance of your application dramatically by trading space for time. One technique is to create precompiled lookup tables.

For example, suppose you need a sin() lookup table function. You can easily create a function that locates a requested value in a table containing the values of sin() at a specific granularity.

The input C file for this generator, shown in listing 13.1, resembles the input for the system test from appendix B.

Listing 13.1. lookup_function_example.c

The generator looks for the XML comments in the input .c or CPP file and creates a new function for matching the specifications. Here’s an example result:

As you can see, the comment is preserved, but now there is an interior to the comment: the new sin_lookup function, which takes a double as input and returns a double as output. The double is multiplied by the count, and the static array of values, dValues, is indexed to obtain the correct output value.

13.5.1. Roles of the generator

We can distill the active role of the lookup function generator into one main item: decomposing the functions specified in abstract form into executable code. The generator will not take responsibility for these tasks:

  • Technical or design documentation for the functions.
  • Unit tests for the functions.
  • Input validation.
  • Parameter sanity checking for the functions—whether the functions are in the right state, or they have the proper arguments, or they’re called in the correct order, and so forth.

Knowing what the generator will and will not do aids you in defining the architecture, which is the next step.

13.5.2. Laying out the generator architecture

The generator takes the CPP or .c file as input and then uses templates to build the new output for the CPP file (and the HPP file, if the lookup function is to be externally visible). Figure 13.5 shows the block diagram for the lookup function generator.

Figure 13.5. A generator that builds look-up functions for C++

Next let’s look at the process flow for this generator.

13.5.3. Processing flow

Lookup function generator

The processing steps for the lookup function generator are:

  • Reads the CPP file.
  • Each time it finds special comment code, the generator follows these steps:

    • Parses out the attributes of the function.
    • Runs the equation internally to build the lookup table. This is where a self-interpreting language such as Ruby, Perl, or Python can really come in handy.
    • Invokes the lookup function template with the attributes and the lookup values.
    • Builds the new contents for the output file from the results of the template.
  • Inserts the result back into the original text.
  • Backs up the original file.
  • Creates the new .CPP or .c file.

The final example generator in our book is a helpful utility for using precompiler macros in C and C++.

13.6. Technique: creating macro lookup tables

Both C and C++ support constants using precompiler macros. The problem with these macros is that when you are debugging you cannot see the name of the symbolic constant; instead, all you see is the constant value. It would be convenient to have a tool that takes the value of a symbolic precompiler macro and returns its name. In this section, we show you how to build a lookup table generator.

13.6.1. Feeding code to the generator

The code in listing 13.2 includes a few #define macros and a comment that says where the lookup table function should go.

Listing 13.2. lookup_table_example.c

The code would look like this after the generator runs:

13.6.2. Roles of the generator

The lookup table generator is very simple and has just one main responsibility: building the lookup table functions within an implementation file. The generator will not take responsibility for documenting the lookup function, either on a technical or a design level.

13.6.3. Laying out the generator architecture

As input, the generator takes the HPP or CPP files that contain the pre-processor macros. Then, using a set of templates, the generator builds the lookup table function. The output of the template is then stored in a target CPP file. Figure 13.6 shows the block architecture for the lookup table generator.

Figure 13.6. A generator that builds lookup tables for C++

Now let’s look at the steps the generator goes through as it executes.

13.6.4. Processing flow

Lookup table generator

Here is the step-by-step process flow for this generator, which builds the code in the example from listing 13.2:

  • Reads the CPP file.
  • Parses out and creates a catalog of the #define macros.
  • Creates the lookup table function using the lookup table template.
  • Finds and replaces the lookup_table tag comments with the new code.
  • Makes a backup of the CPP file.
  • Replaces the CPP file with the new text.

13.7. Summary

As we stated in the introduction, the generators in this chapter are very simple; with the right tools they should only take a day or so to build, but they will repay you immeasurably in saved time and trouble over the long run. These examples are just the edge of the spectrum of practical code generation techniques.

Throughout this book, you have seen examples of where and how code generation can be applied. It is our hope that you will customize and apply these techniques to your own problems and your specific architecture. We hope we have convinced you that code generation yields benefits in the areas of quality, consistency, and productivity—benefits that you can reap in every level of your application development.