Chapter 15. Beyond compile, test, and run – Grails in Action

Chapter 15. Beyond compile, test, and run


In this chapter

  • Packaging your application
  • Extending the Grails command set
  • Building Grails projects using other tools

So far in this book, we’ve treated the Grails application in isolation. You’ve seen how to write an application, test it, and then run it, but that’s all. What happens when you want to make your application available for use by customers or the general public? What if you want to distribute the application so that others can install it? Maybe this application is one cog in a larger project—what then?

These questions relate to the larger picture of application development. Figure 15.1 highlights the key stages of the development cycle that we’ll cover in this chapter.

Figure 15.1. The typical development cycle for Grails applications. The code, test, and package steps are implemented by the Grails build system; the others must be implemented manually or by a plugin.

In essence, the story doesn’t end when the code is “finished”: the application must still be made available to users, and there may be several steps involved in the release process. You’re undoubtedly used to builds that automate the compilation and testing of an application (when was the last time you invoked the Java compiler directly?), but you can often automate the release process as well, using the same build tool. This automation is important in reducing the likelihood of errors and improving reproducibility. Sadly, humans are prone to making mistakes, particularly when tired.

You may not be aware of it, but Grails comes with its own build system that covers the spectrum of compile, test, run, and package. Every time you run one of the Grails commands, you’re initiating that build. Nor is the build system set in stone: we’ll show you how you can extend it to support a hypothetical release process.


Continuous integration

Many projects, and almost all open source ones, use a technique called continuous integration (CI). The basic idea is that a server continuously checks out the latest source code for your project, builds it, and then runs the tests. If there are any failures, it sends notifications so that the team gets quick feedback when the build is broken.

There are several CI products available, but none of them have direct support for Grails applications. Instead, you have to use a shell script, batch file, Ant build, or something similar to execute the relevant Grails commands and scripts. Which you use depends on the product you work with, but creating an Ant build for CI is quite common. One product we recommend is Hudson (https://hudson.dev.java.net/), which has a plugin for building Grails projects.


Then there are the situations in which the Grails build system isn’t the preferred option, or not even an option at all. Maybe your company enforces a particular build tool, such as Maven, or maybe your Grails application is one cog in a larger project that requires a single multiproject build. Whatever the reason, you can find yourself wondering how to build a Grails application using a different build tool. To help out, we’ll discuss the options available. In particular, you’ll find that building a Grails application with either Apache Ant or Apache Maven isn’t only possible, but quite straightforward.

But first, let’s look at the Grails build system: at what help it provides in packaging your application, how it works, and what you can do to extend and customize it.

15.1. The Grails build system

In a typical project, you manage the phases in the development cycle (shown in figure 15.1) with a build tool, such as Ant or Maven. Grails could have used one of these and relied on the user to install the required build tool, but that would go against the Grails philosophy of providing everything you need out of the box (except Java!). Instead, it comes with its own build system that you interact with via the command line.

This isn’t just a build tool, though. You don’t have to manually set up a build file so that the tool knows how to compile your project or run the tests—everything is done for you. That’s why we call it a build system rather than a build tool. But Grails needs to implement the commands somehow, and for that it uses a dedicated build tool that you may not be familiar with: Gant. We’ll talk more about it in section 15.1.2.

Looking at the development cycle in figure 15.1, you’ll notice that the Grails build system provides nothing for the distribution and deployment stages, so you’ll have to get your hands dirty and learn how to extend it. Don’t worry, we’ll turn you into an expert later in this section.

But first, let’s take a look the packaging phase of the development cycle, which hasn’t had much exposure yet in this book.

15.1.1. Packaging an application

Before we can either distribute or deploy a Grails application, we must first package it into a form that works for both cases. As you probably know, the standard package format for a web application in the Java world is the WAR file (short for web archive), which is a JAR file that conforms to a well-defined directory and file structure. A WAR file can be dropped into any standards-compliant servlet container, such as Apache Tomcat or Jetty, and the application will just work.

Because application deployment is such a fundamental part of the development cycle, Grails comes with a dedicated command for packaging your application:

grails war

This one command will take all your application’s classes, dependencies, and resources, and place them in the correct location within the final WAR file.

Why not try it out? Let’s say we want to deploy Hubbub to a Tomcat instance that we’ve installed. Running the war command creates a hubbub-0.1.war file in the root of the project directory that you can copy to the Tomcat webapps folder. Start Tomcat and point your browser at the application’s URL, which will typically be http://localhost:8080/hubbub-0.1/.


Warning

Grails also has a package command that you’ll see if you run grails help. But this command doesn’t create a WAR file, nor does it do anything else that’s useful to the end user. It will probably be deprecated and removed sometime in the future.


Do we want that version number in the URL? It kind of ruins the look and is a pain for anyone typing the address manually. There are a couple of ways to remove the version, but the easiest is to change the name of the generated WAR file:

grails war hubbub.war

You don’t have to limit yourself to changing the filename either. What about generating the WAR file directly in the Tomcat webapps directory? Pass the full path and filename to the command like so (where $TOMCAT_HOME is an environment variable containing the location of the root Tomcat directory):

grails war $TOMCAT_HOME/webapps/hubbub.war

By default, the WAR file will be created for the production environment, but you can specify an alternative if you wish. For example, to package the application with the development settings, run this command:

grails dev war

The other core environments are test and prod (for “production”).


Under the hood

You might wonder how an application packaged as a WAR file knows what environment to run in. Normally this information is provided at runtime via commands, such as test-app and run-app. But in a servlet container, this isn’t possible. Instead, Grails inserts the environment into the application.properties file, which is then packaged in the WAR file. The deployed application reads the configured environment from there.


We haven’t quite reached the end of the packaging story yet, although what we’ve covered is more than sufficient in the majority of cases. If you need more control over the process, though, Grails offers several configuration options that allow you to influence the WAR file generation. As an example, suppose that your application requires a different version of a library from the one packaged with Grails, such as the latest and greatest version of Hibernate. Ideally you could put the version you require in your application’s lib directory and all would be well, but the generated WAR file would contain both versions of the Hibernate JAR file, which almost certainly would cause problems. How can we keep the Grails version out of the WAR file?

Before the WAR file is generated, all the files that go into it are first copied to a temporary directory called the staging area. We can remove the offending JAR from the staging directory before it’s zipped up into the WAR file—there’s a hook that allows us to do that. Create the file grails-app/conf/BuildConfig.groovy and add these lines to it:

grails.war.resources = { String stagingDir ->
delete(file: "${stagingDir}/WEB-INF/lib/hibernate3-3.2.6.jar")
}

The closure specified will be called just before the WAR file is created, and, as you can see, Grails passes it the location of the staging directory. The delegate is an instance of Groovy’s AntBuilder, which you’ll see more of later, but for now take it on trust that the line in the closure uses the Ant delete task to remove the Hibernate JAR from the staging area.

The BuildConfig.groovy file has the same syntax as the Config.groovy file that you’re already used to, but whereas the latter contains configuration information for the running application, BuildConfig.groovy is concerned with the build. Because affecting the WAR file generation isn’t a function of the application but of the build, you can see why the grails.war.resources setting goes where it does.

The other configuration options that you can use (all of which go into BuildConfig.groovy) are listed in table 15.1, but because they’re pretty specialized, we won’t dwell on them. Just be aware of their existence—you never know when they might come in handy.

Table 15.1. Influencing WAR file generation

Config option

Type

Description

grails.war.destFile

String

Specifies the path to the generated WAR file, including its name. This is overridden by any explicit argument to the grails war command.

grails.war.copyToWebApp

Closure

All Ant filesets defined in this closure are copied to the root of the staging directory. This is ideal for controlling which files under web-app are included in the WAR file. The closure delegate is an Ant-Builder, and it’s passed the grails war argument if one is used.

grails.war.dependencies

Closure or list

If set to a closure, this defines the Ant filesets that are copied to the WEB-INF/lib directory. Its delegate is an AntBuilder, and it has no arguments. If set to a list, the list includes Ant file patterns defining which JARs are copied from Grails’ lib directory.

grails.war.resources

Closure

This is a one- or two-argument closure in which you can do any last-minute processing before the WAR file is generated. The first argument is the location of the staging directory, and the optional second argument is the grails war command argument. The closure’s delegate is an AntBuilder.

With your WAR file in hand, you can now start thinking about the distribution and deployment stages of the development cycle. Which you choose will depend on your situation, but don’t worry—we’ll cover both. The question is, how do we proceed? We seem to have reached the end of the line as far as Grails commands go.

15.1.2. Going it alone: how to create a dist command

Look high and low, and you won’t find any Grails commands that take you beyond the packaging stage. As far as the build system is concerned, you’re on your own once the WAR file is ready. To be fair, once you enter the realm of distribution and deployment, users’ requirements vary considerably, so it would be difficult for Grails to provide anything meaningful in this arena.

Undoubtedly some people would say that dropping a WAR file into a servlet container is hardly rocket science, yet anyone experienced in builds and deployment knows that human error can creep in even here. Nor are all deployments so simple. The same goes for building and publishing distributions. In both cases, automation aids reproducibility and makes life much easier for the person involved. If that person is you, all the better!

The ideal solution would be to extend the Grails build system to include commands for these processes. Let’s do that.

Enter Stage Left: Gant

Each Grails command is implemented as a Groovy script. This isn’t any ordinary script, though: it uses a syntax that can be interpreted by a special build tool called Gant.

If you take a look in the $GRAILS_HOME/scripts directory, you’ll find lots of these scripts, many of which have familiar names (corresponding to the core Grails commands). You can also provide your own scripts in several locations (where $HOME is your home directory and <app> is the directory containing your application):

  • $HOME/.grails/scripts
  • <app>/scripts
  • plugins

Any scripts in your home directory become always-on Grails commands. You can use them from any project or none at all. In contrast, scripts in the application directory are only available when you run Grails in that application’s directory. In the next chapter on plugin development, you’ll also see how you can extend the Grails command set via plugins.

What is Gant? Back in the day, Ant was pretty much the definitive build tool for Java projects. Times have changed, and there are now new kids on the block challenging Ant’s dominance. Some of these are based on Groovy, and one such is Gant. According to its author, it’s principally a means of scripting Ant tasks, but it has some of the trappings of a full-blown build tool, such as targets and dependencies.

Because Grails uses Gant for its build system, you have to learn how to write Gant scripts in order to provide extra commands. Rather than describe these scripts in all their gory detail (well, not so gory really), we’ll kill two birds with one stone and show you how to create a script that builds a distribution package for Hubbub. At the end of this process, we’ll have a brand new command (grails dist) to play with.

Let’s start by looking at what we want this command to do. Not everyone has a servlet container lying around, so it would be nice to include Jetty in the distribution, along with some scripts for running Hubbub inside it. We’ll also add the project documentation for the sake of completeness. All of these should be packaged inside a zip file that can be put on a website or emailed. Here’s what the contents of the final zip file will look like:

+-- bin
| |
| +-- startHubbub
| +-- startHubbub.bat
|
+-- lib
| |
| +-- jetty-6.1.12.jar
|
+-- docs
| |
| +-- ...
|
+-- hubbub.war

With a clear scope of work, we’re now ready to write a script that does the necessary business. It could be argued that the distribution we described is quite generic and could apply to any Grails application, but for the purposes of this example, we’ll create the script in the project.


Of distributions and data sources

The biggest problem with distributing a Grails application is the data source configuration. The database connection settings are compiled into the DataSource class, so the end user can’t change them!

One approach is to specify a JNDI name in the data source configuration. The user can then associate database connection settings with that name.

Alternatively, you can declare a set of external properties files in Config.groovy that Grails will load configuration information from:

grails.config.locations = ["file:.../custom.properties"]

See the Grails user guide for more information on this setting.


The name of the file will be Dist.groovy. Why? Because Grails infers the name of the corresponding command from the name of the script, so the Dist.groovy file becomes the grails dist command. More generally, the command name is formed by lowercasing the initial letter of each “word” and separating them with hyphens. For example, CreateDomainClass.groovy becomes create-domain-class. If in doubt, create the script and run grails help to see what Grails thinks is the corresponding command name.

It’s not just the command that the script name affects: you can also use it to determine whether the script is available as a command at all, or whether it can be run from outside of a project. Prefix the name with an underscore (_), and Grails will treat it as an internal script—no command will be available to run the script. Use the underscore as a suffix instead, and the script can be run from outside of a Grails project, like the create-app command. The underscore doesn’t form part of the command name.

As soon as you save the Dist.groovy file, you can run the dist command, even if you leave the file empty! It won’t do anything, but we’ll soon fix that. Listing 15.1 contains the code for our new script.

Listing 15.1. The dist script

We want to make sure that both the documentation and the WAR file are generated before we try to create a distribution that includes them. It doesn’t make sense to do it ourselves, because there are commands that do both of these things. Commands imply the existence of corresponding scripts, so we can use those. We include the targets from the scripts we’re interested in , and those targets immediately become available to us.

The includeTargets << syntax is generic Gant, but the grailsScript() method is unique to Grails scripts and should only be used to include the core scripts provided by Grails itself. You can include targets from your own scripts (or those provided by plugins), but you’ll need to specify a file on the right side like so:

includeTargets << new File("${basedir}/scripts/Other.groovy")

Note that you have to give the path to the script file itself—the .groovy extension must be included. The extension should not be used with grailsScript().

A Gant target (similar in concept to an Ant target) is defined using target(), and its general form is

target(<name> : <description>) {
...
}

where <name> is a string representing the name of the target and <description> is a brief summary of what the target does (which also happens to be displayed by grails help). The target implementation goes inside the curly braces, as you can probably guess. In this case, the target is called createDist.

Inside the body of the target, we can declare what other targets should be run before this one by using depends() . Note that you don’t have to put quotes around the names of the targets inside depends(), but you can do so if you wish. In our script, this one line will ensure that the docs and WAR file are generated before we create the zip file.

With the dependencies declared, we then initialize a script variable, zipFile, with the location and name of the distribution zip file. This is a contrived example because we could easily hard-code the path in the createZip target, but it’s important to see how you can pass information from one target to another. Anyway, this approach means that another script or target could call createZip and specify a different file path.


Script and local variables

As well as local variables, class fields, and method arguments, Groovy scripts have another type of variable that we call a script variable. These are created whenever a variable is initialized but not declared anywhere. So where

def var = "Test"

creates a local variable,

var = "Test"

creates a script variable if var hasn’t been declared anywhere else. The most significant aspect of script variables is that they’re global. Whereas a local variable follows the regular scope rules, a script variable is available everywhere in a script once it’s created, no matter what scope it was created in.

One final thing: not only are script variables global within a script, but in Gant they’re also available to all included scripts too. This also works in the other direction: script variables created in an included script are available to the one including it. This is one way of passing data between targets, but be aware that this global namespace can easily become polluted with variables, increasing the chances of name clashes.


We’re nearly done with the script—there are just a few more items to cover. Not only can you declare target dependencies via depends(), but you can also call targets directly as if they were methods . One of the great advantages of this technique is that it allows you to control exactly when the specified target is executed. On the other hand, depends() ensures that a target is only ever executed once during a build, even if more than one target depends on it.

Inside our second target, we get the location of the generated WAR file from the warName script variable, and use Ant’s zip task to generate our distribution zip file . The name and path of the zip file come from the zipFile script variable, which we set in the main target. Where does the location of the WAR file come from? The warName variable is set by the war target in the _GrailsWar script. You’ll also notice the ant property: this is an implicit variable provided by Gant, and it’s an instance of Groovy’s AntBuilder. Any Ant task can be executed by invoking a method of the same name on the builder—for example, ant.echo() runs Ant’s echo task. Each named argument maps to an attribute of the task, and nested elements can be added within a closure. For example, the equivalent of in Ant’s XML format is as follows:

<zip destfile="${zipFile}">
<zipfileset dir="${basedir}/src/templates/bin" prefix="bin"/>
<zipfileset dir="${grailsHome}/lib" prefix="lib">
<include name="jetty*.jar"/>
</zipfileset>
<zipfileset dir="${basedir}/docs" prefix="docs"/>
<fileset file="${warName}"/>
</zip>

In fact, the mapping between the XML and Groovy code follows a common pattern for XML, where elements map to method calls, attributes map to named arguments, and nested elements map to closures.

At the end of the script , we set the default target—the one called by Grails when executing the corresponding command. In this case, when we call the grails dist command, Grails invokes the createDist target. The setDefaultTarget() method must come after the named target, or it will throw an error.

Phew! That’s a lot of detail to take in at once. The key features to remember are targets, dependencies, includes, and the AntBuilder. With a little practice, you’ll get the hang of it, and there are plenty of open source plugins that you can mine for ideas. As for AntBuilder, now that you know how to map Ant’s XML syntax to Groovy, the best source of information is the Ant reference manual at http://ant.apache.org/manual/.

What about existing targets, though? How do you know which of the Grails scripts you should include and what targets to call? This depends on the state of the Grails user guide, but to get you started, we list some of the more useful scripts in table 15.2, along with their main targets.

As you get more experienced with writing scripts, there’s no better way of finding out what scripts and targets are available than going to the source: the Grails scripts themselves! If you’ve installed Grails locally, you can find them in $GRAILS_HOME/scripts, unpackaged and unencrypted.

One script that deserves special mention is _GrailsEvents, which provides the build system’s event mechanism.

Table 15.2. Some useful Grails scripts and their targets

Script

Targets

Description

_GrailsSettings

N/A

Sets up useful variables and does some other essential stuff, but has no targets. If you don’t include any other scripts, include this one—almost all other Grails scripts do.

_GrailsArgParsing

parseArguments

Parses any command-line arguments passed to the script—these arguments are accessible using the argsMap variable.

_GrailsClean

cleanAll, clean

Does what it says on the tin. cleanAll removes test reports, whereas clean doesn’t.

_GrailsCompile

compile

Compiles the application source files.

_GrailsTest

compileTests, testApp

Compiles and runs the tests.

_GrailsRun

runApp, runWar

Runs the application.

_GrailsWar

war

Generates the application’s WAR file.

Build System Events

Grails provides an event mechanism for scripts that allows them to trigger and act upon named events. For example, Grails’ compilation step fires an event before all the source files are compiled, and another one afterwards. Every target also fires start and end events as illustrated in figure 15.2.

Figure 15.2. Targets in Grails automatically fire start and end events, and the event() method allows you to fire them manually.

To manually fire events, your script must first include the _GrailsEvents script (either directly or indirectly via some other script) and then call the event() method:

event("DistCreated", [zipFile])

The first argument is the name of the event, and the second argument is a list of parameters that will be passed to any event listeners. Because this is effectively the sole definition of the particular event, you should document somewhere what it means and what parameters it passes. That makes it easier for application writers and plugin authors to hook into your scripts.

Events aren’t much use if code can’t listen to and act upon them, so let’s take a look at how to implement an event listener. First, you need a file called _Events.groovy in the scripts directory. You then define event handlers by creating an appropriately named script variable and setting its value to a closure:

eventDistCreated = { String zip ->
println "Created distribution zip ${zip)"
}

The name of the script variable is the event name prefixed by “event”. Unfortunately, events like this tend to be poorly documented, so you usually have to search through the scripts themselves to find out how many parameters are passed for a given event and what they are. Hopefully this situation will improve in future, but to aid you in the meantime, we’ve listed a few of the more useful ones in table 15.3. Note that although arguments are passed to event() as a list, the event handler closure receives them as separate arguments.

Table 15.3. Some useful script events

Script name

Arguments

Description

StatusUpdate

Status message (string)

Notifies a listener of the current build status; mainly used for logging purposes.

CompileStart, CompileEnd

A string

Fired before/after the application source has been compiled. The argument indicates what is being compiled; for example, source for the main application source, and tests for the test cases.

CreateWarStart, CreateWarEnd

Two arguments: the name of the WAR file (including path) and the location of the staging directory (both strings)

Fired before/after the WAR file for the application has been created.

CleanStart, CleanEnd

None

Fired before/after the project has been cleaned.

Scripts may not be the most exciting part of Grails, but they’re real workhorses and can make life a lot easier for users. If you have experience with Ant, you should have no problems getting to grips with them, and we’re sure you’ll enjoy the freedom of using a real programming language—witness how much easier it is to implement conditions and loops in Gant scripts compared to Ant.

If you aren’t familiar with Ant or don’t see the point of scripts, that’s OK. Grails provides a pretty comprehensive collection of commands already, so there may be no reason for you to write any extra ones. But before you push them out of mind, maybe we can change your mind with another example: automatic deployment of a Grails application to Tomcat.

15.1.3. Deployment

Creating a distribution for Hubbub allowed us to look at Gant scripts without many distractions: we didn’t have to think beyond what we wanted to include in the zip file. Now it’s time to look at the deployment phase of the development cycle, the last remaining box in figure 15.1. Deploying a web application directly to a servlet container or Java application server is far more common than distributing one, so stay with us!

Automating deployment is a good idea because it reduces both the likelihood of error and the stress level of the person doing the deployment. Unfortunately, this isn’t an area that lends itself to the “one size fits all” paradigm, so any script we use as an example will have only limited application (unless you customize it). Still, a script that automates deployment to Tomcat will be helpful to many people while also serving as a useful example, so that’s what we’ll look at now.

Auto-Deployment in Tomcat 5.5

When you download Apache Tomcat, you aren’t just getting a servlet container. It comes with a prepackaged web application called Tomcat Manager that handles remote deployment of user applications to the container. All we have to do is learn how to access the web application and then use its API.

First things first: how do we enable the manager? Although the manager web application is installed and ready for use in the standard Tomcat distribution, no user has the required rights to access it. Therefore, you must start by configuring a Tomcat user that is allowed to use it.

Edit the <tomcat>/conf/tomcat-users.xml file (where <tomcat> is the location of the Tomcat installation) and add the following line:

<tomcat-users>
...
<user username="overlord" password="Kr@kat04" roles="manager"/>
</tomcat>

In general, you should opt for a nonstandard username and as strong a password as possible, because the manager application gives the user a great deal of power. Any Tomcat user assigned the manager role is authorized to access the manager application.

You can try the configuration out by starting the Tomcat server (usually by running either the startup.sh script or the startup.bat Windows batch file) and pointing your browser at http://localhost:8080/manager/html. After entering the username and password you just created, you should see something like the web page shown in figure 15.3.

Feel free to play with this web application, but it isn’t what we’re interested in right now. Remember, we want to automate deployment from a script, so an interactive UI isn’t much use to us. Fortunately, we can send GET and POST requests to specific manager URLs in order to deploy, undeploy, start, or stop an application. You can even do this from the browser. For example, if you point it at this URL, http://localhost:8080/manager/deploy?war=file:/tmp/hubbub.war, the manager will attempt to deploy the WAR file located at /tmp/hubbub.war.

Figure 15.3. The Tomcat Manager application

Now that we know how to use the manager application, it’s time to put that knowledge into action and write a Tomcat deployment script for Hubbub.

Deploy It, Baby!

What does this deployment script need to do? As before, with our Dist script, we have to make sure that there’s a WAR file to deploy. Once we have that, we need to send a GET request to the Tomcat manager application, passing the location of the WAR file in the URL. Easy! The script code is shown in listing 15.2.

Listing 15.2. The tomcat-deploy script

All this script does is generate the WAR file (which it delegates to the war target), send a GET request to the Tomcat Manager, and read the response. There are a few new features that the script introduces, the first of which is the scriptEnv variable .

When you run a Grails command, you don’t normally give the environment explicitly—Grails uses a sensible default. So grails run-app defaults to the development environment, and grails war defaults to production. With custom scripts like this, Grails doesn’t know what the default should be, so it uses development. We don’t want this for our deployment script, though. Just like grails war, we want production to be the default environment, and that’s exactly what the line at does for us.

There are two important things to know about the scriptEnv variable. The first is that you should set its value to the long name of the environment, not the short name used on the command line, so production instead of prod. Second, you must initialize it before including the targets from the _GrailsSettings script. Note that _GrailsSettings is automatically included via almost all the Grails scripts—_GrailsProxy is the only notable exception.

Next up is the buildConfig script variable . You saw earlier that along with the Config.groovy file, users can put configuration information in a BuildConfig.groovy file. The data from both are available to scripts via the config and buildConfig variables respectively. To access a property called tomcat.mgr.username, we use the syntax buildConfig.tomcat.mgr.username. How do you decide which configuration file the data should go into? As a general rule of thumb, any configuration option required by a script should go into BuildConfig.groovy, so our deployment script uses buildConfig.

The other new feature is the return statement . If a target returns an integer value, Gant treats it as an exit code for the build. This is important because tools, particularly continuous integration servers (which we’ll discuss shortly), often use the exit code to determine whether the build succeeded. A 0 value indicates that everything worked OK, whereas a non-0 value indicates an error of some sort. Typically you would use 1 if there were any problems executing the target (as we do in the deployment script), because it acts as a generic error value. It’s only worth using other values if you need or want to distinguish between different problems.

One more thing: that text property on the URL is added by Groovy. It opens a connection to the URL, reads the response, and returns the content of the response as a string.


A final word of advice

What goes into the WAR file is important, which is why we recommend that any WAR file you plan to deploy to production (or even to staging) is generated from a fresh checkout of the project. You should preferably check out a particular label or version that has been tested. Generating WAR files from developer working copies is almost guaranteed to cause problems due to local modifications, extra files, and so on.


What are you waiting for? Try out your new tomcat-deploy command and discover how a little effort on the script front has saved you the burden of manual deployment.

Congratulations, you’re now officially a script master! We’ve shown you how easy it is to write a couple of scripts that do something useful, and we’re sure that the ideas will start flowing freely now. If you’re productive in this area, you could even package the scripts as a plugin—something that we look at in the next chapter.

Although we’ve spent a lot of time on the Grails build system, we’ve only looked at it from the perspective of the Grails command line. For many users, the command line is all they need, but as we said at the beginning of the chapter, sometimes you have to use a different tool to build your Grails applications. Setting up such tools to successfully build a Grails project is no walk in the park, though, and how would you deal with plugins that hook into the build process? What we need is some way to reuse the existing build system from other build tools.

15.2. Build integration—not for the hobbyist

The existing Grails build system is flexible and extensible, but sometimes it isn’t appropriate for your needs. What if your company has standardized on Maven for its projects, or your Grails application is part of a larger project that uses Ant for its build? Telling the people in charge of a build that they have to do something different for your application will go down like a ton of bricks. Be prepared for a lot of shouting!

This issue is well recognized by the Grails development team, so there are already mechanisms to smoothly integrate the Grails build system into both Ant and Maven builds, the two most popular build tools in the Java world. Let’s start with Ant, which is still phenomenally popular.

15.2.1. Ant

This is the forefather of build tools for the Java platform. Although showing its age compared to some of its younger rivals, it’s robust, flexible, and widely used. Perhaps the greatest compliment to it is that several other build tools allow you to run Ant tasks directly. With that kind of background, surely it’s easy to build Grails projects using it? Thankfully, yes.

Imagine that we have a simple Ant build file that has targets for compiling the application classes, compiling the tests, running the tests, and building the WAR file. Normally, these targets would contain the relevant Ant tasks, such as javac and junit, but a Grails application isn’t a simple beast to build, and it’s far preferable to delegate as much work as possible to the Grails build system. Another point to bear in mind is that some plugins hook into the Grails build system in order to provide certain features, so if you manage the entire build yourself in Ant, those features won’t work.

Grails comes with a dedicated Ant task that bootstraps the Grails build system and allows you to execute any Grails script you want. listing 15.3 shows you how to set up the task and then use it.

Listing 15.3. An Ant build for a Grails application

The first trick is to make sure that the grails-bootstrap JAR is on the classpath when defining the task . In the example build file, we explicitly set the classpath in the task definition, specifying the location of the JAR relative to a Grails installation. You could also include the JAR on Ant’s own classpath, but that’s a less common approach. The important point is that the grails.ant.GrailsTask class is contained in the bootstrap JAR.

Once the task is defined, we can start using it. To compile the application’s classes, we invoke the Compile Gant script . The application references plenty of Grails classes, so the compilation needs to know where it can find them. In this case, we pass the location of the Grails installation to the Ant task via the home attribute. The required script attribute contains the name of the script to run, and the args attribute contains a space-delimited list of arguments to pass to the script. Compile doesn’t make use of arguments, but the TestApp script can accept several, and we demonstrate one of them in Listing 15.3 (-unit, telling Grails to run just the unit tests).

This approach works well and is pretty simple, but there’s a drawback: it relies on Grails being installed locally. You may think this isn’t a problem; after all, it has worked well for us so far. But consider this: how many software builds have you come across that require anything other than the build tool to be installed? Probably not many (if any). Certainly most Java web application frameworks can be used without such a requirement. Your team members are unlikely to take kindly to installing yet another piece of software. Fortunately, you can use Grails without installing a distribution, but it does require more configuration.

No Grails?

When you have a Grails distribution installed, all the JARs in its lib and dist directories are added to the classpaths for compilation and testing, which makes life pretty easy for you. But what happens when you don’t have a Grails distribution? Those JARs have to find their way onto a classpath somehow, and it’s your job to get them there.

In the specific case of the Ant task, we can use some nested elements designed specifically for the job. But we first have to look at the question of the task’s dependencies. You only need the grails-bootstrap JAR on the task definition’s classpath, but running the task requires a few more libraries, such as Groovy and Gant. In the previous example, the task got the libraries from the Grails distribution (via that home attribute we specified), but we can also pass their locations to the task either by using the classpathref attribute,

<grails script="..." classpathref="grails.classpath">

or by using a nested classpath element:

<grails scripts="...">
<classpath>
<pathelement location="..."/>
...
</classpath>
</grails>

Both should be familiar from some of the core Ant tasks.

That’s the easy bit. The hard bit is determining what JARs should be added to the classpath and where to store them. You could put the grails-bootstrap JAR plus all the JARs from the Grails lib directory into your application’s lib directory. All you would have to do then is set the classpath to include all the JARs in that directory. The downside is that you end up including quite a few libraries that are completely unnecessary. Is there no other option?


Grails and Ivy

When you create a project using Grails, you’ll find a ready-made Ant build file and some Ivy files in the project’s root directory. You can use this build straight away, and it’s particularly useful for continuous integration.


All the Grails JAR files, including grails-bootstrap, have corresponding Maven Project Object Model (POM) definitions. These include a description of the artifact’s dependencies. Manually parsing POMs isn’t much fun, but you can use Apache Ivy (http://ant.apache.org/ivy/) or the Maven Ant tasks (http://maven.apache.org/ant-tasks/index.html) to fetch the required dependencies for you.

Let’s say you’ve decided to use the Maven Ant tasks and have loaded them into the artifact namespace in the build file. This fragment will download the dependencies required by the Grails Ant task and create a path from them with the ID grails.classpath:

<project ...>
...
<target name="dependencies">
<artifact:dependencies pathId="grails.classpath">
<dependency groupId="org.grails" artifactId="grails-bootstrap"
version="1.1" scope="runtime"/>
<dependency groupId="org.grails" artifactId="grails-scripts"
version="1.1" scope="runtime"/>
</artifact:dependencies>
</target>
...
<target name="compile" depends="dependencies">
<grails script="Compile" classpathref="grails.classpath"/>
</target>
...
</project>

For a full explanation of how to use the Maven Ant tasks, see the website. The most important pieces of information contained in the preceding fragment are the two dependencies: you need both grails-bootstrap and grails-scripts (plus their dependencies) in order to run any of the Grails scripts via the Ant task.

Compile, Test, Run ...

The classpath required to use the Ant task is different from the ones needed for compiling, testing, and running the application. After all, the build system only needs grails-bootstrap and grails-scripts, but the application itself more often than not needs GORM, the controllers and views support, the testing support, and other Grails components. As we mentioned earlier, Grails normally picks up the required libraries from its installation directory, but an alternative approach is necessary when it hasn’t been installed at all.

Although the Grails distribution bundles all its dependencies into the lib directory, the build system internally maintains three different classpaths: one for compiling the application classes, another for compiling and running the tests, and a third for running the application. These are rarely the same, but how do you know which JARs should be in which classpath? It’s fairly obvious for libraries that you use directly from your own code, but the Grails artifacts are a different kettle of fish.

As with the classpath for the Ant task itself, one option is to put all the Grails libraries into your application’s lib directory. You can then use something like this:

<project ...>
<path id="grails.classpath">
<fileset dir="${basedir}/lib" includes="*.jar"/>
</path>
...
<target name="compile">
<grails script="Compile" classpathref="grails.classpath">
<compile-classpath>
<path refid="grails.classpath"/>
</compile-classpath>
<test-classpath>
<path refid="grails.classpath"/>
</test-classpath>
<runtime-classpath>
<path refid="grails.classpath"/>
</runtime-classpath>
</grails>
</target>
...
</project>

Each of the *-classpath elements behaves like a classpath element, so you can embed filesets as well as path references and path elements. As before, there’s a fair bit of redundancy with this approach, and you would be better off using the Maven Ant tasks or Ivy.

That leads us to a build tool that makes dependency management one of its foundations: Maven.

15.2.2. Maven

Love it or loathe it, Maven is widely used and here to stay. Its core principles of build conventions and dependency management can simplify many builds and help with consistency across multiple projects. Manually integrating a Grails application into a Maven build would be an exercise in masochism, though, because both are opinionated but with different opinions! Fortunately, easy integration is provided via a Maven plugin and an archetype.

The simplest way to start with Maven and Grails is to create a new project using the Grails archetype:

mvn archetype:generate
-DarchetypeGroupId=org.grails \
-DarchetypeArtifactId=grails-maven-archetype \
-DarchetypeVersion=1.0 \
-DarchetypeRepository=http://snapshots.repository.codehaus.org \
-DgroupId=example -DartifactId=my-app

This will create a new application called my-app with a group ID of example. Note that you don’t need to install Grails for this to work. All you need is Maven 2.0.9 (or greater).

Let’s take a look at what the command has created for us:

my-app
|
+--- pom.xml
|
+--- src - main - webapp - WEB-INF - web.xml

The pom.xml file is the Maven project descriptor (the POM), which includes everything you need to build a Grails application with Maven. It contains a standard set of Grails dependencies and the configuration for the Grails Maven plugin. The web.xml file is empty and is only there to ensure that the Maven WAR plugin doesn’t generate an error when packaging the application. Leave it well alone.

One thing you’ll notice immediately is that it doesn’t look much like a Grails application. That’s easily rectified with this command:

mvn initialize

Maven will download all the required Grails artifacts (and their dependencies) and then create the standard project structure that you’re familiar with. You’re now good to go.


Under the hood

Although the Grails Maven plugin hooks into the standard build cycle, it uses smoke and mirrors to make the integration look tighter than it is. None of the standard Maven plugins are used to build the Grails application. Instead, each phase delegates to the corresponding Grails script, so compile delegates to the Compile script, test delegates to the TestApp script, and package delegates to War. In effect, the integration relies on the standard plugins not doing anything because the project structure doesn’t match the convention.

Another point to make is that the Grails script dependencies are still taken into account, so not only does mvn package invoke all the previous phases (such as compile), but the War script also invokes the Grails compilation. In other words, the Grails compile target is invoked more than once. It’s not ideal, but it does work.


The Grails Maven plugin hooks into the standard Maven build cycle, so you can invoke the phases you’re used to, such as compile, test, and package. Also, you declare the application’s dependencies in the POM rather than dropping the relevant JARs into the lib directory, as you would with any regular Maven project. The plugin ensures that the libraries are added to the appropriate classpaths, so you don’t have to worry about all those classpath references we looked at in the section on Ant. Everything is taken care of for you.

In addition to the standard build phases, the plugin also adds quite a few goals for you to use, such as run-app and install-templates. In fact, there are equivalent goals for most of the Grails commands—you can find a complete list in the Grails user guide. To run one, use this syntax:

mvn org.grails:grails-maven-plugin:1.0:run-app

That works, but we can’t imagine that you want to type out such a long command on a regular basis. The reason for the verbosity is to avoid conflicts between plugins that provide goals with identical names. Fortunately, you can cut out lots of typing by adding this fragment to your $HOME/.m2/settings.xml file:

<settings>
...
<pluginGroups>
<pluginGroup>org.grails</pluginGroup>
</pluginGroups>
</settings>

This little piece of XML turns that monster of a command into this:

mvn grails:run-app

Far easier on the eyes and the fingers we think you’ll agree.

You now know how to create a new Maven-enabled Grails project, but what if you have an existing one that you want to convert to Maven? Simple:

mvn grails:create-pom

This command will create a POM and some other files for the project so that you can immediately start building it with Maven.

One final point: the nature of the Maven integration means that some Maven plugins won’t work with Grails projects, such as the Jetty plugin. Anything that relies on the standard Maven project structure is unlikely to work unless you can configure it for the Grails project structure. Despite that, the Maven integration should be quite sufficient for most projects, and it has the added benefit of being easy to set up.

If you need to integrate with something other than Ant or Maven, the techniques we’ve discussed won’t help much. Instead, you need to work with the Grails bootstrap classes directly. That’s a topic beyond the scope of this book, but it’s useful to know it’s possible. Hopefully other build tools will feature direct support for Grails projects in the future.

We’ve talked a lot about both build tools and the Grails build system, but there’s at least one other significant aspect of application deployment that we haven’t considered yet. Making it easy to deploy new versions of an application is great, but what about the database schema? As your application grows, the schema is likely to change, and it’s important to manage those changes.

15.3. Coping with a changing data model

Most Grails applications are built upon a data model backed by a database. In an ideal world, your first attempt at designing that model would be perfect, and you’d never have to change it. But as we all know, this ideal is a long, long way from reality. Data models constantly evolve and require changes to the database schema. This often happens after real data has started accumulating in the database.

Managing these changes is an important part of both development and deployment of an application. How can you improve the schema and incorporate new features without messing up the existing data? We’ll start by looking at whether we can develop an approach based on the dbCreate data source setting, and then follow that up with a look at a plugin dedicated to solving the problem.

15.3.1. Schema migration with Hibernate

While developing the Hubbub sample application, you were taking advantage of a Hibernate feature called “automatic schema export.” This feature uses your domain model to create the relevant tables in the backing database. Figure 15.4 illustrates where it fits into the application startup process.

Figure 15.4. Where Hibernate’s schema export fits into application startup and shutdown. The strings on the right highlight which dbCreate settings are active for each phase.

You control the behavior of the automatic schema export via the dbCreate data source setting, which you saw in chapter 1 (section 1.4.1). During development, you typically set this to either create or create-drop. These values ensure that your database remains free of old tables and columns, but at the cost of losing all the data every time you run the application. If you need some reference or test data, you should create it in the application’s BootStrap.

Once the data model starts settling down, this habit of losing the data can become irritating at best, particularly if you’re building up a good set of data while you manually test your application. At that point, you should switch to update as the dbCreate setting. This will preserve your data while keeping the database schema in sync with your data model.

The update setting is fine for development, but there are some serious problems with it that mean you shouldn’t use it in production systems. First, it can’t perform some migrations. The classic example is a column rename: Hibernate will create the new column but it won’t migrate the data to the new column, nor will it remove the old column. That leads us to the second problem, which is that old tables and columns are left in the database gathering dust and causing confusion. Finally, Hibernate doesn’t keep track of schema changes, which means that it’s nearly impossible to roll back to an old version.

The traditional solution for such controlled data migration is to use manually crafted SQL scripts that both update the schema and migrate the data. You can certainly go down this road, but to do so you must remove the dbCreate setting entirely. This prevents Hibernate from doing anything to the database schema.

If you decide to use SQL scripts for this job, you may be interested in the following Grails command:

grails schema-export initial-schema.sql

This generates a SQL Data Definition Language (DDL) script that you can use to recreate the application’s database schema in a fresh database. Rather than relying on Hibernate’s automatic schema export, you can execute the SQL script against an empty database so that it will work with your application. If you don’t pass an argument to the command, it creates the file ddl.sql.

Writing and maintaining SQL scripts sounds like a lot of hard work, though—something that we’re not used to with Grails. Let’s look at a plugin that promises to make life easier when it comes to migrating data.

15.3.2. Intelligent migration with Autobase

The Autobase plugin tries to solve two problems: migrations that Hibernate can’t manage by itself and migrations that have already been applied to a database. Both problems have already been tackled by a Java library called LiquiBase, and it’s this library that the plugin is based on.

The first step on your path to intelligent data migration is to install the plugin:

grails install-plugin autobase

On installation, the plugin will create a new directory, migrations, in the root of your project. This is where the information about schema and data changes is stored.

What happens next? Let’s say we want to rename the userId field in Hubbub’s User domain class to username. If the dbCreate setting is update, we can make the change and restart the application. Hibernate creates the new username column as expected, but the old column is left containing all the old user IDs. This is obviously a problem.

Instead of using update, we should remove the dbCreate setting altogether. This means we have to take responsibility for all changes to the database, but at least we’re in control—that’s important. Once that’s done, we need to define some migrations that Autobase can apply. Creating a new migration script is as simple as running this command,

grails create-migration RenameUserId

which will create the migrations/<login>/RenameUserId.groovy file, where <login> is your login name.

The newly created file is pretty bare, so let’s add a migration to rename the user_id column. Note how we refer to the name of the database column, not the name of the domain class field.

changeSet(id:'RenameUserId', author:'peter') {
renameColumn(tableName:"account",
oldColumnName:"user_id",
newColumnName:"username")
}

We’ve defined a single changeset here that performs the column rename using what LiquiBase calls a “structural migration”; in other words, something that modifies the structure of a table or database view. Figure 15.5 illustrates the effect we’re after.

Figure 15.5. The effect of renaming the account table’s user_id column to username without affecting the data

Where do the changeSet() and rename-Column() methods come from? The above DSL is markup builder syntax that maps to the equivalent LiquiBase XML changelog format. That makes it easy to use the existing LiquiBase documentation, which refers to the XML format, because you already know how to convert between the two forms. You can find out what other refactorings are available here: http://www.liquibase.org/manual/home.

Now when you start Hubbub, Autobase will automatically apply this migration. It’s even clever enough to ignore changesets that have already been applied to the database.


Under the hood

LiquiBase records the last changeset that was applied to a database in its own table. Whenever it performs a data migration, LiquiBase only executes the changesets that come after that last recorded one.

Changesets are applied in the order that they’re defined in a migration script. There’s no guarantee, by default, as to what order migration scripts are applied, though. You can change this by editing migrations/changelog.groovy and replacing the includeAll() method with an include() for each individual script in the order you want them to run. The latter method takes the path of a single script as its argument.


That’s just one migration, but you’ll undoubtedly want to add more in your own applications. You can either add changeSet blocks to the existing migration file or create a new one. Which you do comes down to personal preference, but it’s easier to manage a few files containing multiple changesets than lots of files with a single changeset each.

What should you put in a changeset? This is a good question, but there’s no easy answer. One strategy is to create a new changeset each time you need to make a change to the database schema. If the change is large, you should break it down into multiple changesets. For example, adding a new relationship may involve creating an extra column or two and adding a foreign key constraint, all of which could go in a single changeset.

Although there’s plenty of room for discretion in choosing what goes into a changeset, the one thing you should not do is define one that leaves the database in a “broken” state. For example, removing a column without removing foreign keys that depend on it.

Refactorings like renaming columns aren’t the only changes you can make. Autobase comes with a command that allows you to do some powerful stuff:

grails create-groovy-migration UpdateData

This command creates the usual migration script that you would expect, but it also creates a plain Groovy script in migrations/<login>/.scripts. The code in the script has access to some important objects, such as a groovy.sql.Sql instance (see the generated script for information on others), which you can use to manipulate the data in the database directly.

That’s all there is to it! Just remember that once you remove the dbCreate setting, you have to create a migration for every change to the domain model that also requires a change to the database schema.


Warning

Autobase can’t currently create the database schema from scratch, so it won’t work with a fresh, empty database. Hopefully this problem will be resolved in the near future. In the meantime, consider running the schema-export command before you apply any migrations. You can then run the generated SQL against a fresh database before running the application (with Autobase) against it.


Controlled data migration using a tool like Autobase may seem inconvenient at first, compared to Hibernate’s automatic schema export, but in the long run it saves a lot of grief. You know what migrations are applied, and it’s easy to associate them with feature requests and bug reports. You can also easily roll back to previous database states. These features ensure that when the crunch comes and the pressure is on, you have the information and tools to deploy new versions of an application against an existing database.

15.4. Summary and best practices

As you’ve seen in this chapter, software development involves more than writing code. You have to be able to run the tests, run and package the application, and probably deploy it too. Grails provides a comprehensive build system that takes the burden of creating a system off your shoulders, while also allowing you to both hook into and extend it. Yet it doesn’t provide support for the full development cycle, so you may have to write your own Grails scripts.

Then you have the situations where you can’t use the Grails build system directly, but must use an alternative build tool. You’ve seen that Grails provides Ant and Maven integration out of the box, so you won’t have any worries there. In fact, it raises the question of whether you should use one of these tools out of choice rather than necessity.

Here are some pointers to help you out with all this build and development cycle stuff:

  • Write scripts for any repetitive task that you can. Automation is important in reducing errors and improving reproducibility. On top of that, a little investment of time early on can pay big dividends later. When crunch time comes, you’ll be glad you just have to type in a single command for any task.
  • Use continuous integration. This is almost mandatory for any team bigger than one to catch those bugs that appear when changes from different people are merged together. Even for sole developers, continuous integration is useful for running your full suite of tests, in case you forget to run them locally or they take too long.
  • Use the build tool you’re comfortable with. At the end of the day, you should use the build tool that meets all your requirements and that you’re most comfortable with. If you’re happy with the Grails build system, use it if you can—it saves you a fair bit of effort. Consider the other build tools if your application is just one part of a larger project.
  • Start using controlled data migration. Although Hibernate’s schema export feature is convenient, its usefulness is limited in the long run. Start using a tool like Autobase as soon as the data model has settled down or just after the first deployment at the latest.

Whatever option you eventually choose, you can rest assured that Grails will support that decision and allow you to use your chosen build tool with a minimum of fuss.