Thursday, January 15, 2015

Fun with POI4XPages and Maven

It's been long since I wrote my last post. For part of that period I completely left world of Domino and XPages. During that time also many interesting XPages technologies and project appeared or got more popular. One of them is build automation using Maven, so my plan for holidays was to check recent posts from Christian and Jesse about mavenization of XPages libraries.

Before I got to them, a friend of mine called with a request for PDF creation on Domino. It was not for an XPages app, but anyway it seemed like a good fit for POI4XPages, which could be built by Maven. I agreed to create small demo for him. Main goal was to generate an order document which also contains table of items. It will be called by a button on classic notes form, so the plan was to use  URLOpen to download it from the server. Problem that comes with this is that users won't be authenticated, so custom security and session as signer is needed.

I had some prior experience with POI4XPages and FOP, which is used for PDF conversion. But not big. Next chapters are more about fun of learning and experiments rather than instructions how to code. Goal was just a proof of concept anyway

First attempt

So I went ahead and downloaded latest 1.2.6 release of POI4XPages for Domino and put it on my dev server and designer. Easy as usual. Then I had to check the sample database to see how a PDF can be generated. Since I needed some flexibility of getting my data using session as signer, my first attempt was with PoiBean. But problem of PoiBean is that it can't generate PDF.

Second attempt

Now I focused just whether I can just generate the document, no matter how. So I took docx sample and started to fill some fields. It was quite easy, but as the document contained some tables and a logo, it didn't go as easy as I would expect. 

First problem was that sometimes the export worked OK, sometimes not. It took me some digging in docx file itself, before I got to a "pro tip" that is in POI4XPages documentations. DISABLE SPELLCHECKER. If you don't do this, your text is sometimes split by spell-checking marks and search and replace feature of POI4XPages won't find your marks. It took most of my time in this phase.

Once this was working, I just had to get my document using sessionAsSigner and fill wgpoi:document component like this:

I know it's quick and dirty :)

Next problem was a table of items, which are saved in a text field and separated using #.  Here I just resorted to Java and created a class with a helper method that returns List<MyEntry>. With this in place, table generation was easy:

There was one more problem with the table. First two data rows got mixed together, so I had to create fake empty second row.

With all this in place, I could generate my document. But the result didn't look good (btw. original logo is replaced with openntf).

Problem was that my dev server is on Linux. When I created docx and not pdf, it was OK. Problem is caused by FOP font mechanism. It couldn't find some Czech characters, so it just replaced them with #. I used to deal with such problems is past, but luckily customer is running on Windows and there it was fine.

 (If you need to do similar platform test and don't have Domino on Windows, just run your app in Notes as XPINC. I assume you have Notes on Windows if you do development and have Designer. If not, please let me know, I would love to have designer on different platform)

Last hack I did was that I called generateDocument in afterPageLoad event to get immediate download from the URL. It was really a hack since it breaks standard processing and returns an error to Domino console. But basic flow for a demo was working and I was quite happy with that. In just few hours I got a good looking PDF with possibility of easy changes in the template (alternative was to write it by hand in iText, but this would have to been done by someone else).

And then it came.

Image challenge

After demo to the customer, friend called with 2 bad news. First was easy, he forgot one column in the table, which is probably addition of 8 lines in my code, fine. Second one was not. Customer need to add images of signatures to the order. And those images are stored in RichText fields.

OK. I knew it was too easy till this point.

After brief twitter and skype conversation with Christian I decided to continue further. He already did some progress with RichText processing, but his code is not integrated into POI4XPages yet. So I had to do it myself. Also his goal is much bigger. He wants full RichText processing. I just needed 2 images.

It's opensource

I needed to find a way how to hook my code to exiting process of POI4XPages, so I cloned the repo from GitHub . Also Christian did some bug fixing, so I could start on development branch. One of the fixes apparently fixed my issue with table noted earlier.

At this point I finally could check Maven configuration for building of XPages libraries and update sites. It worked well and I was able to build it in few minutes.

After some digging I found that wgpoi:document has an attribute for postGenerationProcess. It allows exactly what I needed - do some processing after the POI document is generated and before it is handed to FOP. Awesome.

Getting an image from RichText

It's not that hard to get image embedded in RichText. I used DXL, where it is Base64 encoded. Now I just have to hope it comes out in form that I can use with POI.

Fail one

I created simple code that took the image and called addPicture() on XWPFRun and put it in my NSF. It failed. It just raised a SecurityException, so I knew I hit a dead end here. Changing java.policy on production server was not an option. 

Clean solution is to put my code into own plugin that runs in XPages runtime, so it has higher permissions. 

Even cleaner solution would be to use POIPowerAction from POI4XPages, but Christian told me about it few days after it got here. But anyway, it wouldn't work because of Fail two later.

One of my goals was to play with OSGi plugin development and Maven anyway, so it wasn't big deal as it was on my learning list for holidays anyway.

 Small problem was that I couldn't find my plugin dev VM and I had to install new one. This didn't go as easy as I would expect. Domino and Notes installation were fine, but fixpacks failed. I use Win2k3 as base of my VMs, so it is unsupported, so this could be the reason. Anyway, if I remember correctly, what helped was to revert from JRE7 to JRE6 before Notes fixpack installation and increasing JVM memory using JAVA_OPTS for Domino fixpack installation.  

Image Exporter plugin

Goal for the plugin was really simple. Just get an image from passed Domino document and add it using POI. I decided that I need to get this working first and then I will play with Maven.
Using XPages SDK for Eclipse I got my plugin easily deployed to Domino, so I was ready to test.

Fail two

Everything seemed OK, except the document was broken. Even when I tried just to generate docx document, word complained that it was corrupted. I did some googling and found a bug report for addPicture. It's not resolved and also POI can't be easily upgraded on Domino anyway. But between comments was a post with workaround that did manual XML inserts.

This manipulations required access to schema classes that were already in place in poi.library plugin. But as we are on OSGi and those classes were not exported by the plugin, I had no access to them. Even worse, since POI used them I couldn't just add them to my plugin. This resulted in an error that classes were already loaded by the classloader. Only solution I found was to add required packages to export list of poi.library plugin. This means I need to have custom built library on the server, but I can live with that for now as I assume my plugin won't be required with future release that will support RichText.

It still didn't work. Even in the plugin I got Access denied exception. I went back to code of POI4XPages code and found some AccessController.doPriviledged magic. Yes, it is in to PowerPOIAction that I didn't know about earlier. It did the trick.

Last request was to align numbers in the table to right. I didn't want to mess with code in POI4XPages because of this, so I used another hack. I set default font for the document to Courier and filled the strings with unbreakable spaces. It looks quite nice. Here is final prototype (signatures are fake :-) ):


Great, I got my prototype, so now was time for some clean-up. First I finished "Mavenization" of my library, so it can be built from command line. Since I started with pom files from Christian, it wasn't that hard. Only problem was with BASE64Decoder. It is an internal JVM class and shouldn't be used. I tried to replace it with Base64.decode(string).getBytes(), but even when I played with character sets, I couldn't get same result. Luckily there was another option with existing classes - DatatypeConverter.parseBase64Binary. It worked as expected.

Second I needed to clean my way of triggering file download. I again found inspiration in POI4XPages demo database and moved it to a REST service. It was quite easy

Now when I call  "objpdf.xsp/docdownload?docid=B3980D1E0D88E207C1257DBC004E8181"  I get back my PDF.


It was both fun and educational, so it fulfilled my needs. I hit some blockers, but since all the code was open source, I could help myself. Also Christian helped when I needed, so many thanks to him for his help and for POI4XPages in general. 

If you want to check my code, my plugin is available on BitBucket . It's not clean, but works for me. Feel free to use it and if you need me to add there some license files, let me know. 

XPages app is quite simple. As it contains customer related info, I didn't share it. But all that is added is:

public void addSignatures(Document doc, XWPFDocument xdoc){
ImageExporter ie=new ImageExporter(doc);
  HashMap<String,String> signatures=new HashMap<String, String>();
  signatures.put("SIG01", "sig01");
  signatures.put("SIG02", "sig02");

<wgpoi:document id="docContract" downloadFileName="order.pdf"
pathInfo="download" pdfOutput="true"

I still need to add some kind of security mechanism to my app, but it will probably just pass some insensitive info from the document to the URL, maybe even just noteid, and compare it while opening.