A quick journey through Grails searchable plugin

Grails searchable plugin is a great Compass based tool to implement search in Grails. As a part of my server inventory app, I used it to implement Ajax based search where search results update as you type. The relevant part of my domain model looks like this:

class BaseDomain {
    Date dateCreated
    Date lastUpdated
    Boolean active = true
}

class Host extends BaseDomain {
    String ipAddress
    String hostName
    Site site
}

class Site extends BaseDomain {
    String name
    String description
}

Objective: user searches for matching hosts by either IP address, host name, site name, as well as filter by active/inactive field.


Defining searchable fields.

class Host extends BaseDomain {
    static searchable = {
        only: ["ipAddress", "hostName", "site", "active"]
        ipAddress boost: 2.0
        hostName boost: 2.0 
        site component: true
    }

    String ipAddress
    String hostName
    Site site
}

class Site extends BaseDomain {
    static searchable = { 
        only: ["name", "active"]
    }

    String name
    String description
}

"ipAddress" and "hostName" get a boost over "active" and Site.name. Site is defined as searchable component, meaning if a match is found on a site name, the Host that has a Site whose name matched the query is returned in the search result.


Working the query

User submitted query string should be a wildcard match on Host.ipAddress and Host.hostName fields, so if we have following data:

ip address	127.0.0.1
host name	localhost
active	true
site name	yahoo

queries like '127.0' or '0.0.1', 'local', 'calho' , 'yahoo', should all produce the above Host object as a match. Since there is only one search field, there is no way to know if what a user submitted is an ip address, a host name, or a site, therefore, all fields need to be checked. So, if a user submits "127.0", the query string for search should look like this:

ipAddress:*127.0* OR hostName:*127.0*

Since Site is a component, its matches are to be handled separately inside the search closure:

def wildcardQuery = "*" + params.query + "*"
def searchQuery = "ipAddress:" + wildcardQuery + " OR hostName:" + wildcardQuery 
return Host.search({
    must {
        queryString(searchQuery)
        wildcard('$/Host/site/name', wildcardQuery)
    }
}, params)

Then we add search for only active Hosts unless 'inactive' flag is set:

return Host.search({
    must {
        queryString(searchQuery)
        wildcard('$/Host/site/name', wildcardQuery)
    }
    if (!params.inactive) {
        must(term('$/Host/active', "true"))
    }
}, params) 

Now, let's add sort to the query. Technically, nothing needs to be added to the above snippet as long as there is a params.sort in the request. It works fine for fields like "ipAddress" and "hostName". However, if sorting by Site.name or 'active', actual "sort" request parameter must contain '$/Host/site/name' and '$/Host/active', which is not very convenient. It is more practical to have "site" and "active" as the value for params.sort, clone params into a separate map, and then overwrite sort param with the right value. This is how entire search closure looks:

def search = { 
    def searchParams = [:]
    params.each {
        searchParams."$it.key" = it.value
    }

    switch (params.sort) {
        case "site" :
            searchParams.sort = '$/Host/site/name'
            break
        case "active" :
            searchParams.sort = '$/Host/active'
    }

    def wildcardQuery = "*" + searchParams.query + "*"
    def searchQuery = "ipAddress:" + wildcardQuery + " OR hostName:" + wildcardQuery 
    return Host.search({
        must {
            queryString(searchQuery)
            wildcard('$/Host/site/name', wildcardQuery)
        }
        if (!params.inactive) {
            must(term('$/Host/active', "true"))
        }
    }, searchParams)
}

Sorting with an outer join using GORM

In a situation where a GORM query is to return a sorted result set, but the sort field belongs to the associated object, which may or may not be null, a regular Criteria query will not cut it, because of the following limitation:

Host.createCriteria().list {
    createAlias("site", "s")
    order("s.name", "asc")
}

Without "createAlias", the query would be an outer join on "site", but because we are trying to sort by site.name, "createAlias" makes the query an inner join, so all results where the site is null are lost. Using regular HQL would help us in this situation:

Host.findAll("from Host as h \
               left outer join h.site as s order by s.name asc") 

Reverse sort dropdown values in a GSP page

In a GSP page in a Grails application, it is easy to sort values in a dropdown in ascending order. Just pass a closure to sort() call with the name of the attribute to sort by.

${i.name}

It is just as easy to sort in descending order. Just provide a sort closure with 2 parameters and return 1 or -1 as in a Comparator.compare() method.

${i.name}

... or see Tom's post below.

Extracting IP addresses from DiG output for a bunch of hosts

In my Grails server inventory app, I had to account for future IP space refactoring, where a bunch of hosts would change their IP addresses. All users of the system will expect to see new IPs in all runbooks, so something had to be done to keep them in sync.

So, I decided to add a Quartz job that would

1. Query all existing hosts with a valid host name
2. Call a 'dig' on that host
3. Parse an IP address from dig response

So, let's say we have a Host object that looks something like this:

class Host {
    ...
    String hostName
    String ipAddress
    ...
}

Adding a Quartz job is a trivial task that can be easily accomplished using grails Quartz plugin. One advice: use the Quartz plugin instead of manually adding a TimerTask bean using resources.groovy. Apparently, Grails application's most vital aspects, such as autowiring of services and GORM Hibernate session are not available at construction time for beans defined in resources.groovy, because those beans are read before anything in grails-app directory. So, here's how my Quartz job looks in a Grails 1.3.4 application:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class IpSynchJob {
    def hostService
    def startDelay
    def timeout

    public IpSynchJob() {
        startDelay = 0
        timeout = ApplicationHolder.application.config.timer.ipsynch.interval
    }


    def execute() {
        def hosts = Host.createCriteria().list {
            isNotNull("hostName")
        }

        hosts.each { host ->
            log.info "synching IP for hostname $host.hostName"
            synchIp(host)
        }
    }

    void synchIp(host) {
        def dig = digServer(host.hostName)
        def ipAddress = parseIpFromDigOutput(dig)
        host.ipAddress = ipAddress
        host.save()
    }

    String parseIpFromDigOutput(digOutput) {

        // looking for IP address in dig's answer section using positive lookahead
        def ipInDigPtn = /(ANSWER SECTION\:)(?=\n.*IN\s+A\s+(\d+\.\d+\.\d+\.\d+))/
        def ip
        digOutput.eachMatch(ipInDigPtn) { match ->
            ip = match[2]
        }
        return ip
    }

    String digServer(hostName) {
        Process p = "dig $hostName".execute()
        int initCapacity = 4096
        StringBuffer out = new StringBuffer(initCapacity)
        p.consumeProcessOutput(out, out)
        // maxing out wait at 20 seconds
        p.waitForOrKill(20000)

        return out.toString()
    }
}

functional-test plugin is broken in Grails 1.3.x versions

Got around to write some functional tests for the Grails app i'm working on. "Grails in Action" book recommends functional-test plugin, and it looks like it's pretty popular in the community, but it appears that it doesn't work for Grails 1.3.x versions. And since grails-plugins code hasn't been touched since July, it only makes sense to switch to something else at the moment.

Curried closures and static methods

If the same static utility method is called from multiple places in Groovy code, and some of the parameters to the method are always the same, curried closure is a nice way to shorten the syntax. For instance, the following static method retrieves all the entries from a list that match a certain pattern:

class MatchHelper {
    static List getMatches(pattern, list) {
        list.grep(pattern)
    }
}

Method getMatches is then called from several places in the code:

def pattern = ~/^[Tt].*/
...
List cars = ["Murano", "Touareg", "Taurus"]
List matchedCars = MatchHelper.getMatches(pattern, cars)
assert  ["Touareg", "Taurus"] == matchedCars
...
List veggies = ["Tomato", "Broccoli", "Sprout"]
List matchedVeggies = MatchHelper.getMatches(pattern, veggies)
assert ["Tomato"] == matchedVeggies
...
List cities = ["Torino", "Sydney", "Tegucigalpa"]
List matchedCities = MatchHelper.getMatches(pattern, cities)
assert ["Torino", "Tegucigalpa"] == matchedCities

Not only does each code snippet calls MatchHelper.getMatches(), but also ~/^[Tt].*/ is passed as the first parameter each time. Instead the code can be shorted this way:

def getTMatches = MatchHelper.&getMatches.curry(~/^[Tt].*/)
...
List cars = ["Murano", "Touareg", "Taurus"]
List matchedCars = getTMatches(cars)
assert  ["Touareg", "Taurus"] == matchedCars
...
List veggies = ["Tomato", "Broccoli", "Sprout"]
List matchedVeggies = getTMatches(veggies)
assert ["Tomato"] == matchedVeggies
...
List cities = ["Torino", "Sydney", "Tegucigalpa"]
List matchedCities = getTMatches(cities)
assert ["Torino", "Tegucigalpa"] == matchedCities

A little less code and a little more elegance. Typical Groovy.

Properly defining integration tests in Maven POM

By default, Maven executes all tests during test phase, no matter if the class name ends with 'Test' or 'IntegrationTest'. So, unless integration tests are placed in a separate module (which is not a bad idea at all), some POM crafting is required to make sure that unit tests execute during test phase and integration tests execute during integration-test phase. There are 2 ways to do that:

1. Maven surefire plugin

Maven uses this plugin by default for all test runs. So no special configuration is needed to run unit tests. This is what needs to be added for integration tests:

<build>
    <plugins>
         ...
        <plugin>
             <groupId>org.apache.maven.plugin</groupId>
             <artifactId>maven-surefire-plugin</artifactId>
             <configuration>

                  <!-- at first, all integration tests need to be explicitly excluded 
                         from all test executions, be it test phase 
                         or any other phase where maven-surefire-plugin is invoked 
                         on a custom basis.
                     -->
                  <excludes>
                      <exclude>**/*IntegrationTest.java</exclude>
                  </excludes>

              </configuration>
              <executions>
                  <execution>
                      <id>integration-test</id>
                      <goals>
                          <goal>test</goal>
                      </goals>
                      <phase>integration-test</phase>
                      <configuration>

                          <!-- Here integration tests are included 
                                 at the integration-test phase separately.
                              -->
                          <includes>
                              <include>**/*IntegrationTest.java</include>
                          </includes>
                          <!-- a parameter called 'skipIntegration' is
                                 the same as 'skipTests' except only
                                 for integration tests -->
                          <skipTests>${skipIntegration}</skipTests>
                      </configuration>
                  </execution>
              </executions>
           </plugin>
           ...
      </plugins>
</build>

2. Maven failsafe plugin

This maven plugin includes integration test classes whose names fit '*IT' or '*ITCase' patterns and executes them during integration-test phase. If classes are named that way, then it's enough to simply throw in default maven-failsafe-plugin configuration. However, if classes are named using '*IntegrationTest' pattern, then they have to be excluded from test phase in maven-surefire-plugin configuration, and that pattern has to be defined in maven-failsafe-plugin configuration:

<build>
    <plugins>
         ...
        <plugin>
             <groupId>org.apache.maven.plugin</groupId>
             <artifactId>maven-surefire-plugin</artifactId>
             <configuration>
                  <excludes>
                      <exclude>**/*IntegrationTest.java</exclude>
                  </excludes>
             </configuration>
         </plugin>

         <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-failsafe-plugin</artifactId>
             <configuration>
                 <includes>
                     <include>**/*IntegrationTest.java</include>
                  </includes>

                  <skipTests>${skipIntegration}</skipTests>  
              </configuration>
              <executions>
                  <execution>
                      <id>integration-test</id>
                      <phase>integration-test</phase>
                      <goals>
                          <goal>integration-test</goal>
                      </goals>
                  </execution>
                  <execution>
                      <id>verify</id>
                      <phase>verify</phase>
                      <goals>
                          <goal>verify</goal>
                      </goals>
                  </execution>
              </executions> 
          </plugin>    
         ...
     </plugins>
</build>