Monday, August 15, 2016

XPages ${} risk of code injection possible workaround

I wasn't happy with findings in my previous post, because it can lead to security issues, but also can have performance hit when you actually need dynamic evaluation of injected code (I use it for app localization and few other use cases). After some digging I came to conclusion that it can't be easily changed/overridden because getBindingValue simply turns into createValueBinding when value is evaluated to a String with #{} inside.

Only solution I see is to wrap binding with code that checks possible injections or runs the evaluation in case I really need it. Another benefit is that I can easily log/notify when possible unwanted injection happens.

For the most simple use case that I used in demo I added two beans that implement DataObject to the app, so I can use following syntax ssan[..]/seval[..] (it's not possible to pass arguments in EL method calls in XPages, so this is a bit hacky way of doing this).



Now when I repeat my test I get:
Partial refresh to text2 doesn't update time in second text, because it was pre-calculated using seval bean.

Here is code of those beans
ssan - StringSanitizer:

seval - StringEvaluator:

Now you can have complete control. It'd be nicer if ExpressionEvaluatorImpl could be somehow replaced with custom implementation, so developers can get this level of control without such wrappers, but I haven't found any way doing so.

Friday, August 12, 2016

XPages ${} risk of code injection

While working on app optimization I experimented a bit more with 'Compute on page load' vs. 'Compute dynamically' behavior. There have been several discussions in past about possible combination of ${} and #{}, for example posts from Marky RodenSven Hasselbach and Paul Withers . What struck me today was risk of code injection.

In this app many elements are read from configuration documents that are loaded into beans and later used using ${} binding. This is recommended way as it is static information, so it's efficient. It works nicely until you insert expressions into your data. This way I realized that a lot of code is prone to code injection that can be contained either in configuration documents or any string that is stored and later read this way.

To simulate the issue I created simple page with one field, one button and one text:


All it does is saving entered value into applicationScope and then displaying it. Since the text uses ${} Compute on page load, I have to reload whole page to see the result immediately.

So now to the results. Normal user would probably enter something like 'Hi there'
But more advanced user can try 'Now is #{javascript:new Date()}'
(Also note that if you do partial refresh to the text, date gets updated as it's computed every time)

If he stops being nice to your app, he will switch to
 'You #{javascript:database.getAllDocuments().removeAll(true);'had'} data'
Once he gets bored, he can finish his job with
'You had #{javascript:sessionAsSigner.getDatabase('','names.nsf').getAllDocuments().getCount()} documents in address book, but 
now you have 0.  #{javascript:sessionAsSigner.getEffectiveUserName()} did that.'
(dev/pradny is name of the server as I signed the db with server ID as many admins do)

(one part is missing in previous example as I wasn't brave enough to run full version on my server to take a screenshot, you get the idea...).

The problem is not how you get the data, but how you use it. It can come from field, configuration or computation. With great power comes great responsibility. Just be aware that ugly things can happen, which reminds me of a question. Is your son's name really Robert';) Drop Tables?

Update (15.8.2016)
 Possible workaround in next post

Tuesday, August 2, 2016

XPagesPreloadDB more evil than good

While doing optimization of application load time I found that XPagesPreloadDB notes.ini parameter didn't work in way I expected. With quick google search I realized that I'm not the first one to hit this problem as John Dalgaard wrote about the issue few years ago https://www.dalsgaard-data.eu/blog/caching-in-xpages-not-as-straightforward-as-you-would-believe/. My goal was similar. Just preload configuration as it's loaded from several places and even worse it's loaded using SessionAsSigner.

First of my issues was caused by my stupid mistake. I copied parameter in syntax for Notes client, so it contained server name. It worked, kind of. So if you want to try it, just check the URL from request that's processed by XPages and you get:

With Notes.ini setting:
XPagesPreloadDB=dev/pradny!!test/appload.nsf/entry.xsp
result was:
http://localhost:80/dev/pradny!!test/appload.nsf/entry.xsp

Which is different context than you'd normally use, so it's actually completely different instance of your application.

So next step was to test correct path to application:
XPagesPreloadDB=test/appload.nsf/entry.xsp
result was OK:
http://localhost:80/test/appload.nsf/entry.xsp

Application scope was correctly initialized during preload and stayed for first access. This was what I needed.

I was happy for few minutes, until I started to see strange things happening. It looked like that the app sometimes stopped working. After few tests I added ApplicationListener to the nsf and found the reason. No matter what I did, the application was killed after 30 seconds, so new request after this time hit clean application, which resulted in strange behavior I observed.

I tried to change xsp.application.timeout parameter, but with no luck. It was correctly obeyed when I started the app using normal HTTP request, but the pre-loaded instance was always killed even when requests were hitting the app.

Conclusion is simple. Don't use XPagesPreloadDB as it can cause you troubles. It didn't matter if I loaded just the nsf or an XPage. Application was killed in both cases.

[26853:00002-1723520800] 08/02/2016 04:05:07 PM  HTTP Server: Started
[26853:00013-402384640] 08/02/2016 04:05:17 PM  HTTP JVM: applicationCreated()
[26853:00013-402384640] 08/02/2016 04:05:17 PM  HTTP JVM: ID: 1
[26853:00002-1723520800] 08/02/2016 04:05:37 PM  HTTP JVM: applicationDestroyed()
[26853:00002-1723520800] 08/02/2016 04:05:37 PM  HTTP JVM: ID: 1

Problem probably won't have much impact on production environment as users probably won't be using the app 30 seconds after restart, but it makes this feature useless.

Also note that when XPages are processed during preload, session user name is anonymous with lowercase a, instead of normal Anonymous. So if you have some conditions in your code, make sure you use equalsIgnoreCase.

(problem was tested on 9.0.1FP4 on Windows and 9.0.1 without FP/FP6 on Linux)

Wednesday, February 24, 2016

Domino classic Java elements encoding problems

Recently I encountered a problem, where Notes Java agent contained strange characters. My Domino Designer was set to UTF-8 and everything looked fine when I changed those characters to correct ones, but only until I reopened this design element again.

I tried to reproduce the issue in different VM, but the problem didn't occur and since I have never seen it before, I started to blame Windows 8 for that, since other VM was Windows 2003 and both were English versions with default region for non-Unicode programs set to Czech.

Only option I had was to switch back to non-Eclipse editors, but when I tried to open the element, it looked even worse.

For example this code (check the line with a comment):
When reopened:

When reopened in classic editor:
This can get really bad when such characters occur in string constants.

Only solution was to fix the element in classic editor and leave it this way. Not really good for development.

Today, I was testing the agent again and when I decided to try to print the code to console, the output wasn't correct either (which was just fine in old VM).

Hopefully this lead me to the solution. When you hit 'p' in Java console, you can get some basic information about the environment. And to my surprise it mentioned file.encoding=Cp1252, which is default for English windows, but I really want to deal with UTF-8 only.

To change this option, you can set this option on JVM startup, or use environment variable JAVA_TOOL_OPTIONS. I didn't want to experiment with eclipse.ini for Notes client, so I just created environment variable JAVA_TOOL_OPTIONS : -Dfile.encoding=UTF8


After I restarted my Notes client everything seems to work just fine. Console outputs all characters and I'm able to edit same code from both old and Eclipse based editors. Also non-binary export to ODP seems to contain valid UTF-8 version of the code.

I'm not sure if this setting can break something else, but I really try to default encoding to UTF-8, so I'm willing take the risk.

Monday, May 25, 2015

XPages and Java security troubles

This is more a rant than classic post, but I hope it may help someone or someone could prove me wrong, which would be even better.

In recent app I use  XPages Scaffolding from Jesse Gallagher. He does some magic in his code that relies on Java reflection to access classes that you can define in your NSF as your model. As my dev server had Java AllPermission grant in global block of java.policy everything worked just fine. But just until I moved the app to production server, where this setting is not possible.

I started to get Exceptions like:
java.lang.SecurityException: not allowed to access members in class 
class model.Problem
lotus.notes.AgentSecurityManager.checkMemberAccess(Unknown Source)
java.lang.Class.checkMemberAccess(Class.java:123)
java.lang.Class.getDeclaredFields(Class.java:601)
frostillicus.xsp.model.AbstractModelObject.getGenericType(AbstractMo
delObject.java:215)
frostillicus.xsp.model.domino.AbstractDominoModel.setValueImmediate
(AbstractDominoModel.java:271)
frostillicus.xsp.model.domino.AbstractDominoModel.initFromDatabase
(AbstractDominoModel.java:64)
model.Problem.initFromDatabase(Problem.java:56)

The stack trace shows that  the exception is thrown from internal Notes class that tries to enforce security.

I still thought that is is not a big deal, since I'm allowed to modify java.policy on this server, so I would just grant correct permission to my app using grant codeBase "xspnsf://" syntax. But it just don't work. I checked XPages Portable Command Guide and other sources to see if I spelled it correctly. I tried to debug the code and validate the permission in code, it just looked OK. But it had no impact on the exception. Than I found a note in release notes of Threads and Jobs project.
Note that the following does not work since the Java code is put as class in NSF as opposed to a jar file in the/lib directory:
grant codeBase "xspnsf://server:0/threadsjob.nsf/-" {
   permission java.security.AllPermission;
};
I'm not willing to put my code into a jar as it would make development much harder, so I had to look for another solution.

The problem is caused by using some reflection calls between classes that are loaded by different classloaders. To kill this check you have to play games with SecurityManager and currentClassLoader() . So I decided to use AccessController.doPriviledges and wrap all code that caused my troubles into PrivilegedActions.

It is not the nicest solution for code readability, but it solved my problem and I can continue to focus on my app and not fighting with the platform. 

It took me couple hours to solve this issue and if anyone uses Scaffolding and has similar problem, you can try to use my fork, where these changes are implemented - https://github.com/mpradny/XPages-Scaffolding/tree/feature/priviledged (code probably needs some refactoring, since this is still first version that worked)

If anyone can show me that java.policy setting for a nsf could solve this problems, let me know. I have seen many posts on Stackoverflow where people couldn't get it to work and probably just ended up with AllPermission in global block.

Monday, April 20, 2015

My slides and video form ICS.UG

Last month I had opportunity to speak at ICS.UG user group together with Martin Jinoch. It was great event as I mentioned in previous post. Our slides are available on ICS.UG session list, but I also want to publish demo video and GIT repository that I used for the demo.

I wanted to do some cleanup first, but as time didn't permit me to do so, I publish it just as it was for the conference. It really just a proof-of-concept that put together pieces that I would use separately otherwise.

Slides:


Video:

I had to move mymouse all the time to get correct recording, so please ignore it. It's no voodoo. 

Repository:
https://bitbucket.org/pradnik/ics.ug-demo 

Next plans are to all of this on current project that also involves Open NTF Domino API and XPages Scaffolding, which probably will make some parts more complicated. 
 

Friday, April 3, 2015

ICS.UG and Engage recap



Last week I had an opportunity to visit and speak at ICS.UG in Bremen and then continue seamlessly to Engage user group in Ghent. Both events were amazing and transport between them organized by We4IT was more than special, a beer-bus with actual bar inside. I still wonder whether it is legal, definitely would not be in US.

I’ve been to several user groups in past, so I can compare them to recent events. Here are my thoughts:

Venues and organization
Every year events are better organized and take place in nicer venues. This is especially true for Engage where Theo is raising the bar for all conferences every year. It’s going to be hard to keep this pace, but everyone is doing their best.

Sponsors
This year there were many new names I never heard of, some of the classic ones were missing. It’s still good sign about the market as there are definitely people interested in IBM collaboration space. Also giveaways from sponsors are getting better every year.

People
People are what these events are all about from my point of view. It’s amazing to meet most of the best people from the community at one place. This time even some people I never met before like Nathan Freeman came to Europe. It was nice to talk to anybody at both events.

Focus
All these events started as Lotus user groups, but as product strategy, market and also experience of people who organize those events change, focus of these events change too. And it is good. It keeps us updated what real world needs and not just our yellow bubble.

Technology
I was able to attend most of development track sessions and the message is clear. It shifted a bit from last year, where we were trying to bring modern frameworks to Domino world like Bootstrap4Xpages, to integrating Domino into general software ecosystem. With IBM focusing on Bluemix it will be more clear in future that everyone has to find own way to build systems and solutions according to their needs. Systems are getting disassembled into services that can be integrated with any other third-party services in similar fashion. Biggest hints about future are:

  •  IBM is separating XPages runtime and Notes data store for Bluemix
  • From Bluemix XPages runtime you can connect to on premises Domino data
  • You can consume Domino data from almost any tools using e.g. REST, or even directly from for example Websphere Liberty using CrossWorlds
  • Not everyone is comfortable with JavaScript

Whole container based approach to application management has many similar characteristics with Domino running classic Notes apps. It was also easy to bring an app to a customer and just ‘push’ it to existing server where defined set of services were running. Nothing new for us, right?

Everyone should just find tools they feel comfortable with and deliver best possible experience for end-users. It’s probably not going to be still the best tool in 10 years, but if we only keep looking for the best solution, we probably never create any solution. The experience we have with Notes being stable ecosystem for 25 years is probably not going to return with any tool. We should be grateful for what we had, but that’s all.

Future
There were 2 special sessions at Engage. Session from Paul Mooney and Gab Davis about life of a consultant summarized thoughts I also noted in previous paragraph to people who still don’t get it. It was nice to see Paul again as he is one of the first Lotus UG organizers. As I’ve written already, those events are about people and as we all enjoy just to meet even when career path have taken us elsewhere. But this brings also external knowledge to the community.
Second session was about future of ‘Lotusphere’ or similar user group events as Engage. Everyone would love to share their social experience with as many a people as possible, doing many similar events in every country or region. But we all know it is not possible since we all have to do billable/real work some time and we have limited time for such social events. I’m not saying that there is only place for one Lotusphere, but there always will be high profile events like this Engage and then rest of more local events focused on information delivery. All the speakers are also attendees of the conferences and they come too enjoy them as well, remember they are not paid for the presentations, they go to these events to have fun.

Thank you
I want to thank to all the organizers, speakers and attendees, many of them I consider friends. It’s always nice to meet everyone in person. I also know that the work is not done the minute conference ends. There is always huge paperwork to get done which seamlessly goes into planning for next event. So I thank everyone for the work they do for the community.

See you all at another event somewhere around the globe.