A quick journey through Grails searchable plugin

Grails searchable plugin is a great Compass based tool to implement search in Grails. As a part of my server inventory app, I used it to implement Ajax based search where search results update as you type. The relevant part of my domain model looks like this:

class BaseDomain {
    Date dateCreated
    Date lastUpdated
    Boolean active = true
}

class Host extends BaseDomain {
    String ipAddress
    String hostName
    Site site
}

class Site extends BaseDomain {
    String name
    String description
}

Objective: user searches for matching hosts by either IP address, host name, site name, as well as filter by active/inactive field.


Defining searchable fields.

class Host extends BaseDomain {
    static searchable = {
        only: ["ipAddress", "hostName", "site", "active"]
        ipAddress boost: 2.0
        hostName boost: 2.0 
        site component: true
    }

    String ipAddress
    String hostName
    Site site
}

class Site extends BaseDomain {
    static searchable = { 
        only: ["name", "active"]
    }

    String name
    String description
}

"ipAddress" and "hostName" get a boost over "active" and Site.name. Site is defined as searchable component, meaning if a match is found on a site name, the Host that has a Site whose name matched the query is returned in the search result.


Working the query

User submitted query string should be a wildcard match on Host.ipAddress and Host.hostName fields, so if we have following data:

ip address	127.0.0.1
host name	localhost
active	true
site name	yahoo

queries like '127.0' or '0.0.1', 'local', 'calho' , 'yahoo', should all produce the above Host object as a match. Since there is only one search field, there is no way to know if what a user submitted is an ip address, a host name, or a site, therefore, all fields need to be checked. So, if a user submits "127.0", the query string for search should look like this:

ipAddress:*127.0* OR hostName:*127.0*

Since Site is a component, its matches are to be handled separately inside the search closure:

def wildcardQuery = "*" + params.query + "*"
def searchQuery = "ipAddress:" + wildcardQuery + " OR hostName:" + wildcardQuery 
return Host.search({
    must {
        queryString(searchQuery)
        wildcard('$/Host/site/name', wildcardQuery)
    }
}, params)

Then we add search for only active Hosts unless 'inactive' flag is set:

return Host.search({
    must {
        queryString(searchQuery)
        wildcard('$/Host/site/name', wildcardQuery)
    }
    if (!params.inactive) {
        must(term('$/Host/active', "true"))
    }
}, params) 

Now, let's add sort to the query. Technically, nothing needs to be added to the above snippet as long as there is a params.sort in the request. It works fine for fields like "ipAddress" and "hostName". However, if sorting by Site.name or 'active', actual "sort" request parameter must contain '$/Host/site/name' and '$/Host/active', which is not very convenient. It is more practical to have "site" and "active" as the value for params.sort, clone params into a separate map, and then overwrite sort param with the right value. This is how entire search closure looks:

def search = { 
    def searchParams = [:]
    params.each {
        searchParams."$it.key" = it.value
    }

    switch (params.sort) {
        case "site" :
            searchParams.sort = '$/Host/site/name'
            break
        case "active" :
            searchParams.sort = '$/Host/active'
    }

    def wildcardQuery = "*" + searchParams.query + "*"
    def searchQuery = "ipAddress:" + wildcardQuery + " OR hostName:" + wildcardQuery 
    return Host.search({
        must {
            queryString(searchQuery)
            wildcard('$/Host/site/name', wildcardQuery)
        }
        if (!params.inactive) {
            must(term('$/Host/active', "true"))
        }
    }, searchParams)
}

Sorting with an outer join using GORM

In a situation where a GORM query is to return a sorted result set, but the sort field belongs to the associated object, which may or may not be null, a regular Criteria query will not cut it, because of the following limitation:

Host.createCriteria().list {
    createAlias("site", "s")
    order("s.name", "asc")
}

Without "createAlias", the query would be an outer join on "site", but because we are trying to sort by site.name, "createAlias" makes the query an inner join, so all results where the site is null are lost. Using regular HQL would help us in this situation:

Host.findAll("from Host as h \
               left outer join h.site as s order by s.name asc") 

Reverse sort dropdown values in a GSP page

In a GSP page in a Grails application, it is easy to sort values in a dropdown in ascending order. Just pass a closure to sort() call with the name of the attribute to sort by.

${i.name}

It is just as easy to sort in descending order. Just provide a sort closure with 2 parameters and return 1 or -1 as in a Comparator.compare() method.

${i.name}

... or see Tom's post below.

Extracting IP addresses from DiG output for a bunch of hosts

In my Grails server inventory app, I had to account for future IP space refactoring, where a bunch of hosts would change their IP addresses. All users of the system will expect to see new IPs in all runbooks, so something had to be done to keep them in sync.

So, I decided to add a Quartz job that would

1. Query all existing hosts with a valid host name
2. Call a 'dig' on that host
3. Parse an IP address from dig response

So, let's say we have a Host object that looks something like this:

class Host {
    ...
    String hostName
    String ipAddress
    ...
}

Adding a Quartz job is a trivial task that can be easily accomplished using grails Quartz plugin. One advice: use the Quartz plugin instead of manually adding a TimerTask bean using resources.groovy. Apparently, Grails application's most vital aspects, such as autowiring of services and GORM Hibernate session are not available at construction time for beans defined in resources.groovy, because those beans are read before anything in grails-app directory. So, here's how my Quartz job looks in a Grails 1.3.4 application:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class IpSynchJob {
    def hostService
    def startDelay
    def timeout

    public IpSynchJob() {
        startDelay = 0
        timeout = ApplicationHolder.application.config.timer.ipsynch.interval
    }


    def execute() {
        def hosts = Host.createCriteria().list {
            isNotNull("hostName")
        }

        hosts.each { host ->
            log.info "synching IP for hostname $host.hostName"
            synchIp(host)
        }
    }

    void synchIp(host) {
        def dig = digServer(host.hostName)
        def ipAddress = parseIpFromDigOutput(dig)
        host.ipAddress = ipAddress
        host.save()
    }

    String parseIpFromDigOutput(digOutput) {

        // looking for IP address in dig's answer section using positive lookahead
        def ipInDigPtn = /(ANSWER SECTION\:)(?=\n.*IN\s+A\s+(\d+\.\d+\.\d+\.\d+))/
        def ip
        digOutput.eachMatch(ipInDigPtn) { match ->
            ip = match[2]
        }
        return ip
    }

    String digServer(hostName) {
        Process p = "dig $hostName".execute()
        int initCapacity = 4096
        StringBuffer out = new StringBuffer(initCapacity)
        p.consumeProcessOutput(out, out)
        // maxing out wait at 20 seconds
        p.waitForOrKill(20000)

        return out.toString()
    }
}

functional-test plugin is broken in Grails 1.3.x versions

Got around to write some functional tests for the Grails app i'm working on. "Grails in Action" book recommends functional-test plugin, and it looks like it's pretty popular in the community, but it appears that it doesn't work for Grails 1.3.x versions. And since grails-plugins code hasn't been touched since July, it only makes sense to switch to something else at the moment.

Curried closures and static methods

If the same static utility method is called from multiple places in Groovy code, and some of the parameters to the method are always the same, curried closure is a nice way to shorten the syntax. For instance, the following static method retrieves all the entries from a list that match a certain pattern:

class MatchHelper {
    static List getMatches(pattern, list) {
        list.grep(pattern)
    }
}

Method getMatches is then called from several places in the code:

def pattern = ~/^[Tt].*/
...
List cars = ["Murano", "Touareg", "Taurus"]
List matchedCars = MatchHelper.getMatches(pattern, cars)
assert  ["Touareg", "Taurus"] == matchedCars
...
List veggies = ["Tomato", "Broccoli", "Sprout"]
List matchedVeggies = MatchHelper.getMatches(pattern, veggies)
assert ["Tomato"] == matchedVeggies
...
List cities = ["Torino", "Sydney", "Tegucigalpa"]
List matchedCities = MatchHelper.getMatches(pattern, cities)
assert ["Torino", "Tegucigalpa"] == matchedCities

Not only does each code snippet calls MatchHelper.getMatches(), but also ~/^[Tt].*/ is passed as the first parameter each time. Instead the code can be shorted this way:

def getTMatches = MatchHelper.&getMatches.curry(~/^[Tt].*/)
...
List cars = ["Murano", "Touareg", "Taurus"]
List matchedCars = getTMatches(cars)
assert  ["Touareg", "Taurus"] == matchedCars
...
List veggies = ["Tomato", "Broccoli", "Sprout"]
List matchedVeggies = getTMatches(veggies)
assert ["Tomato"] == matchedVeggies
...
List cities = ["Torino", "Sydney", "Tegucigalpa"]
List matchedCities = getTMatches(cities)
assert ["Torino", "Tegucigalpa"] == matchedCities

A little less code and a little more elegance. Typical Groovy.

Properly defining integration tests in Maven POM

By default, Maven executes all tests during test phase, no matter if the class name ends with 'Test' or 'IntegrationTest'. So, unless integration tests are placed in a separate module (which is not a bad idea at all), some POM crafting is required to make sure that unit tests execute during test phase and integration tests execute during integration-test phase. There are 2 ways to do that:

1. Maven surefire plugin

Maven uses this plugin by default for all test runs. So no special configuration is needed to run unit tests. This is what needs to be added for integration tests:

<build>
    <plugins>
         ...
        <plugin>
             <groupId>org.apache.maven.plugin</groupId>
             <artifactId>maven-surefire-plugin</artifactId>
             <configuration>

                  <!-- at first, all integration tests need to be explicitly excluded 
                         from all test executions, be it test phase 
                         or any other phase where maven-surefire-plugin is invoked 
                         on a custom basis.
                     -->
                  <excludes>
                      <exclude>**/*IntegrationTest.java</exclude>
                  </excludes>

              </configuration>
              <executions>
                  <execution>
                      <id>integration-test</id>
                      <goals>
                          <goal>test</goal>
                      </goals>
                      <phase>integration-test</phase>
                      <configuration>

                          <!-- Here integration tests are included 
                                 at the integration-test phase separately.
                              -->
                          <includes>
                              <include>**/*IntegrationTest.java</include>
                          </includes>
                          <!-- a parameter called 'skipIntegration' is
                                 the same as 'skipTests' except only
                                 for integration tests -->
                          <skipTests>${skipIntegration}</skipTests>
                      </configuration>
                  </execution>
              </executions>
           </plugin>
           ...
      </plugins>
</build>

2. Maven failsafe plugin

This maven plugin includes integration test classes whose names fit '*IT' or '*ITCase' patterns and executes them during integration-test phase. If classes are named that way, then it's enough to simply throw in default maven-failsafe-plugin configuration. However, if classes are named using '*IntegrationTest' pattern, then they have to be excluded from test phase in maven-surefire-plugin configuration, and that pattern has to be defined in maven-failsafe-plugin configuration:

<build>
    <plugins>
         ...
        <plugin>
             <groupId>org.apache.maven.plugin</groupId>
             <artifactId>maven-surefire-plugin</artifactId>
             <configuration>
                  <excludes>
                      <exclude>**/*IntegrationTest.java</exclude>
                  </excludes>
             </configuration>
         </plugin>

         <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-failsafe-plugin</artifactId>
             <configuration>
                 <includes>
                     <include>**/*IntegrationTest.java</include>
                  </includes>

                  <skipTests>${skipIntegration}</skipTests>  
              </configuration>
              <executions>
                  <execution>
                      <id>integration-test</id>
                      <phase>integration-test</phase>
                      <goals>
                          <goal>integration-test</goal>
                      </goals>
                  </execution>
                  <execution>
                      <id>verify</id>
                      <phase>verify</phase>
                      <goals>
                          <goal>verify</goal>
                      </goals>
                  </execution>
              </executions> 
          </plugin>    
         ...
     </plugins>
</build>

Maven versions plugin breaks when version number is inherited from parent

In release management cycle, it is common to force a version number on to maven POM before the artifact is deployed to maven repository. This type of situation arises when, for instance, releases are managed by Continuous Integration tools, such as Cruise Control, Hudson, or Anthill Pro, and it feels redundant to have to go into source code, manually update version number in the POM and commit it. If managing entire release is not done via maven release plugin, then maven versions plugin can come in handy to just set the version number:

mvn versions:set -DnewVersion=1.0.0

There is one nuance though. In some cases, project version is inherited from parent, and project's POM uses parent POM, like this:

<parent>
    <groupId>com.company</groupId>
    <artifactId>company-parent</artifactId>
    <version>1.0</version>
    <relativePath>../company-parent/pom.xml</relativePath>
</parent> 

In this case for a local maven build, version does not have to be defined in POM as it will be 2.0, the version number of parent. However, the mvn versions:set command will result in error with the following message:

[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Project version is inherited from parent.

Thus, some version has to be added to the project. If project still has to have parent version for some reason, then just copy version from inside parent tag. Otherwise, just go with any valid version number.

Using your application classes inside Grails Gant script

So you have a Grails application, and want to script some cool things on top of it, like calling your RESTful service, using dynamic finders on domain classes, or adding new records using Hibernate. No surprise, there are some caveats that await for you on the way. Let's see what they are:

1. Loading your application context

This is an obvious first step, and this is what Grails expects you to do:

includeTargets << grailsScript("_GrailsBootstrap")   
target(main: "My Gant script") {       
    depends(configureProxy, packageApp, classpath, loadApp, configureApp, compile)       
    ...  
}   
setDefaultTarget("main")   

Yep, all these parts of depends() clause are needed, although "compile" part is needed only if domain objects are directly used, i.e., MyDomainClass.findByAttr().

2. Setting target environment

Two ways to do that. Standard grails way:

grails prod my-script    - for environments pre-defined in Grails, such as 'dev' or 'prod'
grails -Dgrails.env=stage my-script     - for custom environments

or place this at the beginning of your script:

scriptEnv = "envName"
and then simply run
grails my-script

3. Accessing your application classes and calling methods on them

The following example shows how to call a service method in your Gant script.

def serviceClass = grailsApp.getClassForName("com.killerapp.AwesomeService")
def serviceClassMethod = serviceClass.metaClass.getMetaMethod("blazingMethod")
def service = appCtx.getBean("awesomeService")
serviceClassMethod.invoke(service, [arg1, arg2] as Object[])

As seen in the example, there are 2 implicit variables that the script provides: "grailsApp" and "appCtx". First one is an instance of org.codehaus.groovy.grails.commons.GrailsApplication, and the second one is an instance of org.springframework.context.ApplicationContext. To call a service method, both are needed: grailsApp helps retrieve an instance of MetaMethod wrapper of the method you want to call. However, MetaMethod has to be called on a specific instance, which is Spring context singleton that Grails application creates when it initializes. appCtx helps retrieve that singleton. 
However, this is not enough. Your service method has to be defined as an actual method, like this:

void blazingMethod(arg1, arg2) {
      ...
}

, NOT a closure:

def blazingMethod { arg1, arg2 ->
      ...
}

That is because groovy closures are compiled into non-anonymous inner classes, not regular Java methods. Therefore, accessing closures by name will not work.


4. Running queries

To run dynamic finders, nothing extra needs to be done:

MyDomainClass.findByName("someName")

To run updates, an instance of org.springframework.orm.hibernate3.HibernateTemplate is needed. It can be obtained using appCtx:

def sessionFactory = appCtx.getBean("sessionFactory")
def template = new HibernateTemplate(sessionFactory)
def newRecord = new MyDomainClass(name: "someName")
template.saveOrUpdate(newRecord)

HibernateTemplate can also be used for queries, like this:

def domainClass = grailsApp.getClassForName("com.killerapp.MyDomainClass")
def allRecords = template.loadAll(domainClass) 

5. Cleaning script cache

As of Grails 1.3.4, using depends(compile) with accessing domain classes in a Gant script generates the following error simply running a script once, then making a change and re-running it:

java.lang.NoClassDefFoundError: com.killerapp.MyDomainClass

This is because scriptCache needs to be cleaned. Simply delete ~/.grails/1.3.4/projects/killerApp/scriptCache/ directory, and rerun the script.

passing String array argument to a Groovy method

Let's say we have a Groovy method that takes a String array as an argument:

void myMethod(String[] arr) { ... }

When passing a string array to this method, the most intuitive way to do it is this:

myMethod(["a", "b", "c"])

However, this produces a NoSuchMethodError, because GDK converts ["a", "b", "c"] to an instance of ArrayList instead of String[]. Next thing you may try is this:

myMethod(new String[]{"a", "b", "c"})

, which is standard Java one-line syntax for creating a String array. This, however, also confuses GDK, and creates exception with this message:

No expression for the array constructor call

Groovy notices opening and closing braces and starts compiling it as a closure, which of course it isn't. This can be worked around in 2 ways:

myMethod(["a", "b", "c"] as String[])

or

myMethod((String[]) ["a", "b", "c"])

Replacing a persistent collection in Grails domain object

If I have a domain model set up the following way:

class Site {

    static hasMany = [instances: Instance]

}

class Instance {

    static belongsTo = [Site]

}

and have to perform a task of wiping out a collection of Instances in a Site, including the deletion of all records of Instance, this would be the right way to do it:

site.instances*.delete()

site.instances.clear()

site.save(flush: true)

This may be a bit non-intuitive for such a trivial task, however, if you omit the first delete() call, you'll end up with orphan Instance records. If you omit the clear() call, you'll get this error:

deleted object would be re-saved by cascade (remove deleted object from associations): [Instance]

And of course, site.removeFromInstances() does not take a Collection, and if you loop through it and delete each individual Instance, you'll get a ConcurrentModificationException. 

Grails UrlMappings - mutually exclusive patterns

Let's say, I have URL patterns that are mutually exclusive and have to map to 2 different controllers. For instance, if I have an enum Vals with the following values: ROCK, PAPER, SCISSORS, and I want URLs like

/root/group/rock (or ROCK)

/root/group/paper (or PAPER) 

/root/group/scissors (or SCISSORS) to map to YouWinController

and

/root/group/(everything-else-but-ROCK|rock|PAPER|paper|SCISSORS|scissors) to map to YouLoseController

Both URL patterns are very similar, because each one consists of 3 strings separated by '/'. We can start by defining some constraints, but at the same time we have to define the exclusive pattern first. In this case, exclusive pattern is the one mapping to YouLoseController.

class UrlMappings {
    static mappings = {
        "/root/group/$loseParam"(controller: "youLose", action: "/loseAction")
        "/root/group/$winParam"(controller: "youWin", action: "/winAction") {
            constraints {
                  // get all string values from the enum
                  def list = stringList(Vals) 
                  // make sure lowercase values are also included
                  list.addAll(list*.toLowerCase()) 
                  $winParam(inList: list)
            }
        }
    }
}

Criteria queries don't work properly with Grails Enums

Looks like Grails (as of version 1.3.1) still didn't fix all the issues with Enums.

I'm using a criteria query like this:

c = MyDomainClass.createCriteria()
def list = c.list {
    and {
        classAttr {
            eq ("attr1", param1)
        }
        eq ("enumAttr", param2)
    }
} 

In this case, MyDomainClass.enumAttr represents an enum. It looks something like this:

enum CountryEnum {
    AUSTRIA("austria")
    BRAZIL("brazil")
    CANADA("canada")

    String name

    CountryEnum(String name) {
        this.name = name
    }
}

Grails, quite wisely, stores 'AUSTRIA', 'BRAZIL', and 'CANADA' in the DB.
However, the above criteria query generates the following error:

java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Enum

There a little section about Enums in Grails 1.1 release notes, which mentions that the value attribute should now be named 'id'. However switching 'name' to 'id' in my CountryEnum simply got rid of the exception, but the 'list' was still empty after the query completed. Extracting SQL from Hibernate debug output produced valid results, which means the problem lies somewhere in the Hibernate layer. Also DB values became 'austria', 'brazil', and 'canada', which are the values of 'id'.

Dynamic finders seem to handle Enums pretty well, but evidently the problem still exists when using Criteria or HQL.

Grails: using Groovy MarkupBuilder to render properly indented XML

When using Grails, the easiest way to render XML is to use 

 render obj as XML

command. However, that returns the entire XML on one line.

If you want XML to look pretty, you can use groovy.xml.MarkupBuilder

def showXml = {
    def writer = new StringWriter() 
    new MarkupBuilder(writer).root {
        child1 {
            child2('someValue')
        }       
    }

    render(text: writer.toString(), contentType: "text/xml")
}

This will produce a neat output where each tag is on a new line and properly indented.