Sunday, March 23, 2014

Building XPages with Jenkins video

Few people told me that all this automated building is definitely nice, but they don't get why they should bother. I decided to record my test Jenkins machine, so it might show them what is this all good for - it's doing work for them (so you can show this to your boss and tell him/her you will be more efficient if you use such tools).

Anyway, this video just runs project from previous post using Jenkins.
  • At first I committed my code with an error that is later detected using Selenium test (it returns 300 instead of 200)
  • Then I fixed it (using notepad) and committed again

 
I did few cuts in the video, because often not much happens. I hope this will bring more XPages projects to Jenkins or any other CI tool.

final note - I really haven't touched that machine. No keyboard shortcuts :)

Gradle Domino testing

I mentioned in previous post that once you have db on server, you can easily test your app using Selenium. Here is proof for that:

And let the magic happen.

What it does?
  1. Creates local nsf from on disk project
  2. Copies nsf to server
  3. Runs Selenium test using TestNG on this app
App is really simple. Just one calculation using viewScope, but good enough for demonstration.

More Gradle goodness for Domino

I created new release of plugin for integration of headless Domino Designer into Gradle builds. Biggest changes are:
  • Rename to Gradle Domino Plugin (most projects around are named after Domino not Notes, so I wanted to keep it aligned)
  • Added basic error checking and output from HEADLESS.log
  • Task for NSF copying
  • build-in tasks in plugin for building nsf and copying to server
 All changes follow Gradle convention over configuration paradigm, so usage is now even easier.

Since I renamed the project I also renamed repostiory to https://bitbucket.org/pradnik/gradledominoplugin (sorry for any troubles) .

 

Access to Domino API

One of most important improvements is introduction of DominoAPITask that can wrap any other task that call Domino Java API. For these operation you need to add Notes.jar to your configuration, so it can call required libs. Also you may need to add Notes programm directory to system PATH to find required DLLs.

In future I'll probably add OpenNTF Domino API too, because it's much easier to use when doing more complex stuff (now there is just one createCopy call) .

 

Simplest usage

 I also added basic sample and I'll probably add more samples later, instead of more complex documentation. So you might just check Samples/Basic directory

If you have gradle installed on your machine, you need just 2 files to add to any NSF on disk project to get this running.

First is gradle.properties to set path to libs directory. In previous post I set properties from command-line, but then the command is to long to type, so you might want to add those properties somewhere else. You can add this file to the project, or your home directory. There is resolution mechanism in gradle that will take care of it.

Content of the file is simple:
dominoLibDir=C:\\gradle-lib
It is just used in build to tell Gradle where to look for our jars. This folder should contain Notes.jar and GradleDominoPlugin jar

Now to main build file. Since we now full obey convention over configuration, our build file is really simple.
 buildscript {

    dependencies {
        classpath fileTree(dominoLibDir)
       
    }
}

apply plugin: 'domino'
That's all. Just tell Gradle that it should use our plugin.

Now from command-line run gradle buildNSF and you should get NSF in your local Notes data directory. Name of the NSF is taken from name of Gradle project, which is name of the directory.

Customization

If you don't like any of defaults, you can change them. buildNSF tasks has following input parameters:
  • odpPath - change path to On Disk Project
  • nsfName - name of target NSF
If you check sources, there are more parameters to change wait time, location of data directory (which is read form notes.ini), location of notes program directory (which is C:\IBM\Notes by default)

If you want just create nsf called "myCool.nsf" add following lines to your build.gradle
buildNSF {
    nsfName='myCool.nsf'
}

Copying to a server

If you want to get your nsf to a server, you have 2 options. First is to use provided task buildToServer, sencond is to use CopyNSFTask and configure it.

Domino plugin adds extension that currently only contains name of server that you plan to work with. You can set it using domino.dominoServer name, e.g. domino.dominoServer = 'dev.pradny.com'

So your complete build file would look:
buildscript {

    dependencies {
        classpath fileTree(dominoLibDir)
       
    }
}

apply plugin: 'domino'
domino.dominoServer = 'dev.pradny.com'
Now just run gradle buildToServer  and you should get first created nsf locally and then copied to your server.

What's next

There are two features that need to be added. Integration with test framework and filling nsf with test data. UI testing can be added now using standard Gradle tools, data manipulation may require some DXL processing or OpenNTF Domino API.

Notice

Sometimes Designer get stuck during load. I don't know why and since it did that even when I didn't change build logic it's probably some bug that is caused by being just Tech-preview. Usually killing designer.exe and nlnotes.exe helped. If this gets too common, I might introduce mechanism in build file that would do that automatically

Saturday, March 8, 2014

Build XPages app from Git with Jenkins

Since introduction of Headless designer tech-preview in Domino Designer 9.0.1 many people started to think about build automation. Now is time to try it with Jenkins - popular free build automation server.

I've used Gradle as a build tool, which uses Groovy language. So some G-stuff which all the cool kids use in their projects (I always wonder why so many project names start with G even when they are not in direct relation to Google).

*disclaimer*
Most of these tools I haven't used in production, so I might break some best practices and take other shortcuts.
Also all code is written just as a proof of concept, nothing more.
*end of disclaimer*

Required tools:

  • Windows machine
  • Java JDK
  • GIT
  • Domino Designer 9.0.1
  • Jenkins
  • Git repository - bitbucket in my case

Jenkins setup

I did use Jenkins installation with GIT and Gradle plugin. If you are new to Jenkins, there are many tutorials online how to setup Jenkins. Recent tutorials for Android app builds use Gradle too, so you might want to check them.

If you just want to get it up and running, following steps did it for me:
  1. Install Jenkins
  2. In windows configure Jenkins service, so it can interact with desktop

    This might not be necessary, but egmar had some problems when running as a service in his TeamCity integration and I had problem when my ID file was stored on VMware shared folder mapped as local drive, which couldn't be accessed by services, and without this I didn't know why the build was stuck.
  3.  In Manage Jenkins set-up security (it should ask you to do it using small banner on top). I just used settings that allowed all logged-in users do anything.
    This is probably necessary only if you want to access private Git repositores.
  4. Install plugins - in Manage Jenkins / Manage Plugins / Available plugins just check Gradle and Git plugin and let Jenkins install them
  5. If your GIT and Java installation are not in system PATH, you might need to add whose in Manage Jenkins / Configure System
 Now your  Jenkins should be ready.

Create job to fetch project

I hope you have all your projects in GIT (or any other SCM) repositories, so you can pick any to try. I recommend small ones first, since for example IBM Teamroom template took about 8 minutes to build in my test VM. And this is not good for experiments.

If you just want to follow this article, repository I used is public, so you can get it from Bitbucket too.

  1. Click on New Item and create your first job
  2. Give it a name (ideally without spaces to avoid troubles later with my GradleNotesPlugin)
  3. Select a free-style build and click OK
  4. Configure Source Code Management to fetch you repository

    I used my repository bitbucket.org/pradnik/testing.git and develop branch. It contains one NSF project and one Java project for tests that I want to integrate later.
  5. Optionaly configure build triggers.
    My Jenkins is not publicly available, which is why I couldn't trigger the build from Bitbucket, which is recommended way, so I used just Poll SCM option.
Now you should be ready to start your build and fetch the repository to local file system.

Save it and click Build Now. When done, you should find your repository locally under Jenkins/jobs/ProjectName/workspace/


Build the Notes app with Gradle

You might notice that my project contains files like settings.gradle, build.gradle and gradlew.bat. These are used for Gradle build.

Gradlew.bat together with gradle directory is just a wrapper arround gradle. It allows anyone to run Gradle build on machines even when they don't have Gradle installed, because it takes care of everything.

This is good for our testing, as we can try to launch the build from our on-disk-project without using Jenkins and then get back to Jenkins with configuration that works.

Gradle Notes Build plugin

To keep our builds flexible I decided to create plugin that hides all the calls to Domino Designer. You may as well write it in build.gradle file, but this is better for reuse.
Originally I wanted to write the plugin in Java, but I had problems with task parameters, so I had to switch to Groovy (this is first piece of code I have ever written in Groovy, so don't blame me for any mess, please).

Again, plugin can be build with Gradle. But if you don't want to do it, just download it here as a jar.

Repository
Project for this plugin is publicly available on Bitbucket  https://bitbucket.org/pradnik/gradlenotesplugin

I won't cover details how to build a Gradle plugin here as many tutorials exist. I just want to show main code of task that we will use



All it does is same routine as in batch file form my previous posts. It has 3 parameters
  • filePath - path to NSFproject (actually .project file)
  • fileName - name of NSF file created
  • notesDir - path to Notes directory
 It creates command in syntax from Headless designer wiki runs it and then waits in loop for notes to finish.

You can see that there is currently no error handling and also the time limit can be low (I probably almost hit it with Teamroom). But this will be improved in future releases :-) . Also when you run the build, it doesn't show output from designer, which might be usefull.

If you want to modify it and build it yourself just run gradle uploadArchives and it will save jar in locally created maven repository.

Build structure

In my repository I have currently 2 projects, so I need to structure my build in some way. This is quite common in Gradle world and Gradle can do magic with it. For now we can forget TestingTestTests directory as it is not used.

Our root project Gradle file just tells gradle to build what needed and setting.gradle tells what subproject is should check.

build.gradle
apply plugin: 'java'




dependencies {
    compile gradleApi()
    compile localGroovy()
}

settings.gradle
include ':testingTest'

Main work is done in build.gradle in testingTest directory (which is my NSF project)

build.grade
buildscript {
     dependencies {
        classpath files(gradleNotesPlugin)
    }
}
apply plugin: 'notes-plugin'

import com.pradny.gradle.NotesBuildTask

task createNSF(type: NotesBuildTask) {
    filePath=file('.project')
    fileName=nsfName
  
} 

All lines are important in this case. Gradle, unlike Maven and Ant, doesn't require huge ammount of lines just to get basic stuff done.

  1. buildscript block - tells us where to find our plugin. We will set this as a command line parameter.
  2. apply-plugin - tells Gradle to use our plugin
  3. import - imports class of our Task (might not be necessary if you use full name on next line)
  4. task - runs our task with parameters (filePath is taken from current project location and fileName is passed as a command line parameter)
In real-world scenario you will configure those parameters in more sophisticated manner, but this also shows that Gradle can do simple things in very simple way.

And this is all.
Now just run:

gradlew createNSF -PgradleNotesPlugin=c:/gradleNotesPlugin.jar -PnsfName=blogTest.nsf

note: parametres are passed using -Pname=value syntax

When done, you should have blogTest.nsf in your Notes data directory.

Calling Gradle build from Jenkins

Last piece in our puzzle is hooking Gradle into Jenkins build. So return to Jenkins web page and open configuration of your job.

  1. At bottom click Add build step button and select Invoke Gradle script
  2. Select Use Gradle Wrapper
  3. Add parameters that we used during test launch
  4. As Task use createNSF
  5. Save it
Run your build and enjoy power of automation.

You can experiment with  build triggers and other Jenkins options, but from Notes - Jenkins integration it is all for today. In future I want to and task to move created nsf to server (for now you can use batch file method from my previous posts, run test including Selenium, etc.

------------------------------------------------------------------------------------------------------------------------

Troubleshooting note - sometimes Designer didn't want to start. Currently I don't know if it was caused by calling it from local service and as a user during experiments, or was it something else. After reboot it was fine again.