Thursday, November 2, 2017

Using JAX-RS inside NSF

Last week Christian G├╝demann published new release of SmartNSF on OpenNTF that contains cool new feature that Christian tweeted before. With new CUSTOM strategy it allows direct execution
of Java code from REST API defined in router configuration. It's even better than it sounds as it initializes facesContext and XPages application if needed, so even access to beans works.

I needed to start to build new REST APIs for few databases, so I decided to test new SmartNSF option and also other available options for REST APIs on Domino (there are several, check references at the end for more info). Since CUSTOM strategy requires dependency on SmatNSF in NSF project and also implementation of CustomRestHandler interface, it'd force me to do more changes in my code that I wanted to. If I need to change my code, why not adjust it for JAX-RS spec anyway.

Existing Domino JAX-RS options had to packaged as plugins, which make it hard to call code that is currently in NSF. I could make it work using pieces from SmartNSF, more classloading hacks and java reflection, but calling all code using reflection isn't best for development productivity. This way I was able to build JAX-RS API on top of existing NSF without changing anything in NSF itself, which would be perfect for my current use case, but I didn't like it.

I decided to take different approach. Use code from SmartNSF to take care of facesContext initialization and create XSP library that would allow direct usage of JAX-RS in NSF. ExtLib already contains AbstractRestServlet that wraps Wink implementation that is available on Domino for long time,  so at the end this solution required far less code than I expected.

Using com.ibm.xsp.adpater.servletFactory I was able to register own factory that takes care of new servlet creation for each servlet/nsf.

This factory also passes properties map to the servlet, so it knows where to look for Wink configuration and JAX-RS resources/providers.

Those paths are evaluated using module classloader, so it looks for files inside NSF.

CustomWinkServlet then just overrides getContextClassLoader method to point it to module classloader and takes care of facesContext and app initialization if needed.
I experienced issues with SessionAsSigner access, which I need for some configuration loading, so I had to move super.doInit() call to first doService call. Problem is that if something is loaded form NSF using module classloader when NotesContext doesn't have current session assigned or first resource is not a class (at least I think), it's not possible to get sessionAsSigner after that point. I got around by loading plugin.Activator first, which I know that should exist in any NSF.

With this infrastructure in place, wrapped as XSP library and installed on my server and Designer, I'm able to call existing XPages/Java code, including existing beans. For example:

For automated mapping of JSON data I included Jackson mapper in my plugin, so it can be just registered in wink.application along with resources.


In wink.properties I kept wink.defaultUrisRelative=false which was mentioned od Jesse's blog, but I really don't know if it has to be there. I'll have to test it. 

That's all what needs to be done. Now I can just access my new REST services: e.g.

or
If you want to try it yourself, sample database also contains showCounter.xsp that displays value from appBean, which is a proof that there is no secret double-life, which may happen when you use different approach for class loading. 


Now I can just wrap existing model and controller classes with JAX-RS annotations or thin wrapper layer when needed, so I can easily use same code from XPages and REST API. 

Source code is available on Bitbucket https://bitbucket.org/pradnik/pristo_rest .

I can imagine that combining this approach with OpenNTF API can make creation of JAX-RS services quite easy as ODA wrappers already take care for mapping proprietary Notes structures to standard Java classes. This is something to try next. 

Happy coding

Resources
Many good articles, series, sagas and samples were published around Domino and REST topics. Here is list of those that I know of:
Extension Library REST Services - https://www.openntf.org/Projects/pmt.nsf/36B7CD129ED7357A86257AC6005523E7/$file/Extension%20Library%20REST%20Services.pdf

Wednesday, November 16, 2016

My presentation from SUTOL 2016 - Automation is developer's friend

Last week I had a session at Sutol conference about automation for developers. It covered several samples, where I started with Reat.js front-end app, stored it in Domino nsf database and later did also complete build on Jenkins server with Selenium tests.

It was a lot of fun to put this together and even more fun, as always, was to meet all those great people from ICS (or should I call it Watson Workplace now?) community.



Sutol 2016 - Automation is developer's friend from mpradny

Repository that I used for the demo is at Bitbucket.org - https://bitbucket.org/pradnik/todoapp_full/branch/develop . If you want to try it, you need to adjust some hardcoded values as I didn't make this build parameterized and it also depends on my Jenkins configuration. Job itself that I used was simple Pipeline from SCM, just with repo address.

Let me know if you want to try it and get stuck.

Many thanks to all sponsors,  organizers and attendees, who made SUTOL 2016 great event. 

Friday, September 23, 2016

SUTOL conference 2016


It became a tradition that every autumn SUTOL organize technical conference that focuses on topics around IBM ICS portfolio. After successful switch to English as primary language last year, this year the event will be extended to 1.5 days.

Event will take place at PARKHOTEL Praha on 10th and 11th November.

More details are at conference site 8th SUTOL Technical Conference. Registration will open soon.

Two day program of conference should give more room for attendee interaction and community discussion. We hope that more international attendees will find their way to Prague as autumn is really nice time to visit. Weather is usually still warm and it's less crowded than during summer season.

If you need any reason, why you should attend, let me know. Call for abstracts is open till 9th October and sponsors are also welcomed.

See you in Prague.

Monday, August 15, 2016

XPages ${} risk of code injection possible workaround

I wasn't happy with findings in my previous post, because it can lead to security issues, but also can have performance hit when you actually need dynamic evaluation of injected code (I use it for app localization and few other use cases). After some digging I came to conclusion that it can't be easily changed/overridden because getBindingValue simply turns into createValueBinding when value is evaluated to a String with #{} inside.

Only solution I see is to wrap binding with code that checks possible injections or runs the evaluation in case I really need it. Another benefit is that I can easily log/notify when possible unwanted injection happens.

For the most simple use case that I used in demo I added two beans that implement DataObject to the app, so I can use following syntax ssan[..]/seval[..] (it's not possible to pass arguments in EL method calls in XPages, so this is a bit hacky way of doing this).



Now when I repeat my test I get:
Partial refresh to text2 doesn't update time in second text, because it was pre-calculated using seval bean.

Here is code of those beans
ssan - StringSanitizer:

seval - StringEvaluator:

Now you can have complete control. It'd be nicer if ExpressionEvaluatorImpl could be somehow replaced with custom implementation, so developers can get this level of control without such wrappers, but I haven't found any way doing so.

Friday, August 12, 2016

XPages ${} risk of code injection

While working on app optimization I experimented a bit more with 'Compute on page load' vs. 'Compute dynamically' behavior. There have been several discussions in past about possible combination of ${} and #{}, for example posts from Marky RodenSven Hasselbach and Paul Withers . What struck me today was risk of code injection.

In this app many elements are read from configuration documents that are loaded into beans and later used using ${} binding. This is recommended way as it is static information, so it's efficient. It works nicely until you insert expressions into your data. This way I realized that a lot of code is prone to code injection that can be contained either in configuration documents or any string that is stored and later read this way.

To simulate the issue I created simple page with one field, one button and one text:


All it does is saving entered value into applicationScope and then displaying it. Since the text uses ${} Compute on page load, I have to reload whole page to see the result immediately.

So now to the results. Normal user would probably enter something like 'Hi there'
But more advanced user can try 'Now is #{javascript:new Date()}'
(Also note that if you do partial refresh to the text, date gets updated as it's computed every time)

If he stops being nice to your app, he will switch to
 'You #{javascript:database.getAllDocuments().removeAll(true);'had'} data'
Once he gets bored, he can finish his job with
'You had #{javascript:sessionAsSigner.getDatabase('','names.nsf').getAllDocuments().getCount()} documents in address book, but 
now you have 0.  #{javascript:sessionAsSigner.getEffectiveUserName()} did that.'
(dev/pradny is name of the server as I signed the db with server ID as many admins do)

(one part is missing in previous example as I wasn't brave enough to run full version on my server to take a screenshot, you get the idea...).

The problem is not how you get the data, but how you use it. It can come from field, configuration or computation. With great power comes great responsibility. Just be aware that ugly things can happen, which reminds me of a question. Is your son's name really Robert';) Drop Tables?

Update (15.8.2016)
 Possible workaround in next post

Tuesday, August 2, 2016

XPagesPreloadDB more evil than good

While doing optimization of application load time I found that XPagesPreloadDB notes.ini parameter didn't work in way I expected. With quick google search I realized that I'm not the first one to hit this problem as John Dalgaard wrote about the issue few years ago https://www.dalsgaard-data.eu/blog/caching-in-xpages-not-as-straightforward-as-you-would-believe/. My goal was similar. Just preload configuration as it's loaded from several places and even worse it's loaded using SessionAsSigner.

First of my issues was caused by my stupid mistake. I copied parameter in syntax for Notes client, so it contained server name. It worked, kind of. So if you want to try it, just check the URL from request that's processed by XPages and you get:

With Notes.ini setting:
XPagesPreloadDB=dev/pradny!!test/appload.nsf/entry.xsp
result was:
http://localhost:80/dev/pradny!!test/appload.nsf/entry.xsp

Which is different context than you'd normally use, so it's actually completely different instance of your application.

So next step was to test correct path to application:
XPagesPreloadDB=test/appload.nsf/entry.xsp
result was OK:
http://localhost:80/test/appload.nsf/entry.xsp

Application scope was correctly initialized during preload and stayed for first access. This was what I needed.

I was happy for few minutes, until I started to see strange things happening. It looked like that the app sometimes stopped working. After few tests I added ApplicationListener to the nsf and found the reason. No matter what I did, the application was killed after 30 seconds, so new request after this time hit clean application, which resulted in strange behavior I observed.

I tried to change xsp.application.timeout parameter, but with no luck. It was correctly obeyed when I started the app using normal HTTP request, but the pre-loaded instance was always killed even when requests were hitting the app.

Conclusion is simple. Don't use XPagesPreloadDB as it can cause you troubles. It didn't matter if I loaded just the nsf or an XPage. Application was killed in both cases.

[26853:00002-1723520800] 08/02/2016 04:05:07 PM  HTTP Server: Started
[26853:00013-402384640] 08/02/2016 04:05:17 PM  HTTP JVM: applicationCreated()
[26853:00013-402384640] 08/02/2016 04:05:17 PM  HTTP JVM: ID: 1
[26853:00002-1723520800] 08/02/2016 04:05:37 PM  HTTP JVM: applicationDestroyed()
[26853:00002-1723520800] 08/02/2016 04:05:37 PM  HTTP JVM: ID: 1

Problem probably won't have much impact on production environment as users probably won't be using the app 30 seconds after restart, but it makes this feature useless.

Also note that when XPages are processed during preload, session user name is anonymous with lowercase a, instead of normal Anonymous. So if you have some conditions in your code, make sure you use equalsIgnoreCase.

(problem was tested on 9.0.1FP4 on Windows and 9.0.1 without FP/FP6 on Linux)

Wednesday, February 24, 2016

Domino classic Java elements encoding problems

Recently I encountered a problem, where Notes Java agent contained strange characters. My Domino Designer was set to UTF-8 and everything looked fine when I changed those characters to correct ones, but only until I reopened this design element again.

I tried to reproduce the issue in different VM, but the problem didn't occur and since I have never seen it before, I started to blame Windows 8 for that, since other VM was Windows 2003 and both were English versions with default region for non-Unicode programs set to Czech.

Only option I had was to switch back to non-Eclipse editors, but when I tried to open the element, it looked even worse.

For example this code (check the line with a comment):
When reopened:

When reopened in classic editor:
This can get really bad when such characters occur in string constants.

Only solution was to fix the element in classic editor and leave it this way. Not really good for development.

Today, I was testing the agent again and when I decided to try to print the code to console, the output wasn't correct either (which was just fine in old VM).

Hopefully this lead me to the solution. When you hit 'p' in Java console, you can get some basic information about the environment. And to my surprise it mentioned file.encoding=Cp1252, which is default for English windows, but I really want to deal with UTF-8 only.

To change this option, you can set this option on JVM startup, or use environment variable JAVA_TOOL_OPTIONS. I didn't want to experiment with eclipse.ini for Notes client, so I just created environment variable JAVA_TOOL_OPTIONS : -Dfile.encoding=UTF8


After I restarted my Notes client everything seems to work just fine. Console outputs all characters and I'm able to edit same code from both old and Eclipse based editors. Also non-binary export to ODP seems to contain valid UTF-8 version of the code.

I'm not sure if this setting can break something else, but I really try to default encoding to UTF-8, so I'm willing take the risk.