Wednesday, December 23, 2009

Google App Engine Datastore "IN" operator

This is the first Desert Island-related technical post. In this article, I am outlining a couple of gotchas regarding the use of the "IN" operator in GQL.

In http://desert-island.appspot.com, I created a class DesertIslandList, which represents a user's list of his/her 8 desert island albums. It, amongst other properties, contains user_id and id_source. For a user who's logged in via Facebook Connect, the user_id of his/her created list would simply be his Facebook profile id and id_source would be "f" (for Facebook).

In order to display lists created by his/her friends, I, first of all, get his/her friend's id list using friends.get method in the Facebook RESTful API with a version of minifb hacked up for Google App Engine. I then created a GQL query for all DesertIslandList WHERE user_id IN :1 AND id_source = :2 with :1 being the Python list containing all the friend ids as strings, and :2 being "f". Straightforward enough? Well it all seemed so until I stumbled across of a couple of gotchas.

Only 30 items allowed

First of all, I soon got a "too many subqueries (max: 30..." exception. As it turns out, only 30 items are allowed in the list:

(from http://code.google.com/appengine/docs/python/datastore/gqlreference.html) Note: The IN and != operators use multiple queries behind the scenes. For example, the IN operator executes a separate underlying datastore query for every item in the list. The entities returned are a result of the cross-product of all the underlying datastore queries and are de-duplicated. A maximum of 30 datastore queries are allowed for any single GQL query.

So, my bad, not reading the documentation properly. To get around this, you could either somehow limit the list to 30 items or chop your list up into 30-item chunks and union the results (sorting offline if necessary).

What if the "IN" list is empty?

What if he or she has no friends?

Convention SQL wisdom would suggest that the query should return no DesertIslandList objects. However, in the world of Google App Engine datastore, "IN" means cross-product of all the separate individual queries, so in my GQL query above, an empty "IN" list effectively means get all DesertIslandList WHERE id_source = "f" i.e. not filtering on the user_id field at all!

Not only is this slight counter-intuitive, this can also throw a surprise "no matching index found" exception. The fact that an empty "IN" list does not filter on the user_id field means my GQL instance with an empty "friend ids" list has to access an additional DesertIslandList index with only the id_source field, and not the usual index with both the user_id and the id_source fields. Imagine this, if you have not run a GQL instance with an empty "IN" list in your local dev environment for this particular query, no index (with only the id_source field) will be added to your index.yaml. Up in the cloud, if a person happens to have no Facebook friends, he or she is going to see an ugly stacktrace complaining the the index does not exist!

Of course, the simple solution to this problem is to always check the list to see if it is empty before running the query.

Just another thing to bear in mind!

First time film processing...

Finally got round to getting my hands dirty (not literally) and processing my first film rolls (Ilford HP5 Plus ISO 400) with the assistance of a kind colleague of mine.

Using a changing bag, I tranferred the two rolls of film from the canisters onto the plastic reel, which was then placed into the developing tank. After 30 mins or so of pouring various fluids (developer, fixer, warm water) in and out of the tank, the images finally appeared on the negative films. With the films being hung up by the door to dry overnight, I was extremely eager to scan them into Photoshop to get a proper look at them.

The following day, I gingerly placed the negs onto the glass of my Canon flatbed scanner (using the film adapters that came with the scanner), started up Photoshop and got my first thumbnail previews of the shots. They looked OK, so I went ahead and scanned a few of them at maximum (3200 dpi) resolution. Much to my surprise, they came out extremely grainy, bordering on being rubbish. I love film grains, but what I was seeing was definitely not the level of grains that you would expect from an ISO 400 film. Worse still, one of the rolls had a vertical streak running down every single frame, and I am still not sure whether it was due to a mistake during processing or from the camera itself (I used different cameras for the two different rolls). Since the other roll does not exhibit this defect, I suspect it is more likely to be the latter.

Anyway three of the "better shots" (which I actually quite like) that came out - the first two I took using the my Canonet 19QL (£15 from eBay!) and the last one using my Cosina 35E (£2 from eBay...) - notice the vertical streak in this last one:

Aunt Sally

Welcome to Kat's party!

Notice the vertical streak in the following photo:

Music to walk by

Friday, December 11, 2009

Druid Street Cycles - best LBS in London?

Update: Thor and co. have recently set up an official Facebook group. And he is promising 10% discount for all group members :-)

Their address: 18 Druid Street, London SE1 2EY gmap
Phone: 07551016380

Vintage Sun Solo RacerHaving recently bought an old Sun Solo racer from Gumtree (80s Raleigh "gaspipe" steel frame, 27" x 1 1/4 rims, Weinmann single-pivot brakes, Sturmey Archer stem, Sturmey Archer hub, 5-speed), I took it to the little known Druid Street Cycles near London Bridge for a quick service.

Not a lot has been mentioned on the web about this social business project, apart from a couple of youtube videos, a user comment in a Timeout article and an LCC article (they themselves don't have a website):



Thor, who used to repair tanks in his native Germany, started this project to provide affordable (and yes, really affordable) bike repair service for the local community in Southwark. To say that he, along with the rest of Druid Street Cycles, is passionate about building/fixing bikes is an understatement.

I have been there twice now, and on both occasions, the jobs they did on the bike far exceeded my expectation. The first time round, they did a full checkup of the bike, fixed the rear brake, and replaced the dodgy tyres. He ended up only charging me 30 pounds (the tyres along cost 20 quid!) - he even crimped the cable ends for me, such is his attention to details! Second time round, I went in to see if they have any mudguard that would fit over my 27" wheel, which I knew was very hard to come by. In typical Thor's fashion, he said he would find something, and surely he did, somehow he managed to fit a full-length mudguard beautifully on the 27" wheel frame, which has minimal clearance. Initially he refused to take any money from me, but in the end I managed to convince him to take the money, which no doubt would be put to good use to benefit the local biking community.

I really cannot recommend them enough, and I hope more people will get to know about this secret little place hidden underneath one of London Bridge's dark arches.

Ah how can I forget the mention the fact that they provide free courtesy bikes to customers, so you can continue your commute to work or your cycle home while Thor and co. are busying fixing yours! Beat that!

Looking for a commute route, or wanting to share your route with other cyclists? Visit my cycle route database: http://zoomaroundtown.appspot.com!

Tuesday, December 08, 2009

Desert Island Discs for the non-celebs

I have just finished my second Google App Engine project: "Desert Island Discs for the non-celebs" http://desert-island.appspot.com/

This is just a little fun project to test out the powerful Facebook Connect API, and to further explore Google App Engine (to find yet another Datastore oddity/feature, which I will blog about soon).

Try it out and add you list!

Thx

Tuesday, November 24, 2009

Django template urlencode unicode characters (Google App Engine)

I have just started work on my next Google App Engine project, and I have been using the Django Template urlencode filter to urlencode strings before displaying them as components inside an URL like this: <a href="/?{{ unsafe_string|urlencode }}" title="..">..</a>, to prevent XSS (Cross Site Scripting) attack, etc. This is all good as long as unsafe_string does not contain unicode characters. Underneath, the Django urlencode filter calls urllib.quote method to do the encoding. Unfortunately, this method does not like unicode characters, and a string containing unicode (utf-8) characters has to be utf-8 encoded before it can be put through urllib.quote (and hence the Django urlencode filter).

So I created a custom Django template filter to circumvent this issue:

Note: I have only tested it in Google App Engine (GAE) using its Django 0.96 template support. Also, the registration mechanism described below is specific to Google App Engine's own templating engine. However, there is no reason why this custom filter would not work in non-GAE Python apps using Django 0.96 or 1.0 templates. Please follow instructions here to configure your app if you are not using GAE.

The custom tag (which handles both unicode and "non-unicode" characters) itself is very straightforward to create.

I created a module customtags.xss (you can name it something else) as follows:

1) under the root folder of my GAE application, I created a folder "customtags"

2) inside customtags, I created two empty files xss.py and __init__.py (__init__.py will remain empty)

3) inside xss.py, I put
import types
import urllib
from django import template

register = template.Library()

@register.filter
def unicode_urlencode(value):
if type(value) is types.UnicodeType:
return urllib.quote(value.encode("utf-8"))
else:
return urllib.quote(value)
So, the filter utf-8 encodes the string first if it is of type UnicodeType. (You may want to handle exceptions more gracefully though - at the moment, I am not catching exceptions...)

Let's look at how to register this template library so that you can use the filter inside your Django templates.

In your application script, for example, YOUR_APP_NAME.py, simply insert the line:
webapp.template.register_template_library("customtags.xss")
after the import statements at the top of the script. Obviously, if you are calling your module something else, you have to change the line above accordingly.

Now you can write <a href="/?{{ unsafe_string|unicode_urlencode }}" title="..">..</a> in your template!

Friday, November 13, 2009

GPolyline decoding in Python

I am currently working on providing GPX data for routes on http://zoomaroundtown.appspot.com. As the routes are stored as Google Map GPolyline encoded strings in the datastore, I need a way to decode them back into latitude and longitude points before I can create the respective GPX files. Mark McClure has an excellent page on this topic. Apart from his Javascript implementation, I have also found a PHP port of the decoder function. However, there does not seem to the any Python port, so I created a straightforward port:

You can donwload the Python file here.



def decode_line(encoded):

"""Decodes a polyline that was encoded using the Google Maps method.

See http://code.google.com/apis/maps/documentation/polylinealgorithm.html

This is a straightforward Python port of Mark McClure's JavaScript polyline decoder
(http://facstaff.unca.edu/mcmcclur/GoogleMaps/EncodePolyline/decode.js)
and Peter Chng's PHP polyline decode
(http://unitstep.net/blog/2008/08/02/decoding-google-maps-encoded-polylines-using-php/)
"""

encoded_len = len(encoded)
index = 0
array = []
lat = 0
lng = 0

while index < encoded_len:

b = 0
shift = 0
result = 0

while True:
b = ord(encoded[index]) - 63
index = index + 1
result |= (b & 0x1f) << shift
shift += 5
if b < 0x20:
break

dlat = ~(result >> 1) if result & 1 else result >> 1
lat += dlat

shift = 0
result = 0

while True:
b = ord(encoded[index]) - 63
index = index + 1
result |= (b & 0x1f) << shift
shift += 5
if b < 0x20:
break

dlng = ~(result >> 1) if result & 1 else result >> 1
lng += dlng

array.append((lat * 1e-5, lng * 1e-5))

return array

if __name__ == "__main__":
latlngs = decode_line("grkyHhpc@B[[_IYiLiEgj@a@q@yEoAGi@bEyH_@aHj@m@^qAB{@IkHi@cHcAkPSiMJqEj@s@CkFp@sDfB}Ex@iBj@S_AyIkCcUWgAaA_JUyAFk@{D_]~KiLwAeCsHqJmBlAmFuXe@{DcByIZIYiBxBwAc@eCcAl@y@aEdCcBVJpHsEyAeE")
for latlng in latlngs:
print str(latlng[0]) + "," + str(latlng[1])


Hope you Python GMap coders out there will find this useful!

Wednesday, November 04, 2009

Javascript Tester iGoogle gadget

Have been looking at iGoogle gadgets for work (I know they are so 2008, but still a good idea...) Anyway I created a simple Javascript Tester gadget for myself, which some of you may find useful: (See this gadget in the iGoogle directory)

Tip: Handy for testing (parsing) JSON objects!

Being an iGoogle gadget, you can add it to you iGoogle homepage, but you can always just use the embedded gadget right here!

Monday, November 02, 2009

Twitter Search API and Google App Engine

So in my previous post, I talked about using Twitter as a commenting engine in http://zoomaroundtown.appspot.com/, and, towards the end, I briefly mentioned that the Twitter Search API (http://apiwiki.twitter.com/Twitter-API-Documentation) does not really work on Google App Engine (and most other cloud environments).

The reason why making Twitter Search API requests on Google App Engine often fails is simple. Twitter rate-limits requests per IP, and on Google App Engine, you share IPs with loads of other apps, a lot of which are probably trying to do the same thing as you i.e. making Twitter Search requests. Unfortunately Twitter Search API is still not incorporated into the Twitter REST API, which means there is no way of identifying requests via your username, so Twitter can only really rate-limit according to IPs.

To circumvent this, I created a simple PHP proxy on my other website (hosted on a "real" machine") :

$url = $_GET['url'];
$session = curl_init($url);
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
$json = curl_exec($session);
header("Content-Type: application/json");
echo $json;
curl_close($session);


and in my GAE Python app, instead of sending the requests straight to Twitter using URL Fetch, I send the requests to this proxy, which does a simple straightforward relay job for me.

Of course, this is less than ideal, as this goes in the face of the philosophy of deploying in a scalable cloud environment such as Google App Engine. But until Twitter allows username-based identification in Twitter Search requests, or Google sorts something out with Twitter (and other service API providers, for that matter), this seems to be the best way to get round this problem!

(I have heard that you are get your own IP on Amazon AWS, but I don't know enough about Amazon AWS to comment on it. Anyway no doubt more and more cloud infrastructures relying on shared IPs will be developed, and I can only see it heading that way...)

Using Twitter as a commenting engine for Zoom around Town

So with the number of routes in http://zoomaroundtown.appspot.com/ increasing daily, the time has come for me to start considering implementing some kind of commenting functionality for the added cycle routes. I have not had the time to implement a user login system (I am still hoping to see more routes in the database first), let alone a commenting system. However one day, a simple idea came to me. What about getting people to comment on routes by creating tweets and then all I have to do is to display these tweets using Twitter Search. I was quite keen on exploring this idea, and I implemented the following in Zoom around Town:

For each route, I programmatically create a bit.ly link for its unique page url e.g. (http://bit.ly/39oF3z for http://zoomaroundtown.appspot.com/findRoutes?id=24001). On the page, I then display a message telling people to comment on the route by sending a tweet starting with @zoomaroundtown http://bit.ly/39oF3z (I'm even including a link that automatically populates the starting text for the user: http://twitter.com/home?status=%40zoomaroundtown%20http%3A%2F%2Fbit.ly%2F39oF3z%20).

To display comments, I use the Search API (http://apiwiki.twitter.com/Twitter-API-Documentation) to search for all tweets sent to zoomaroundtown containing the string http://bit.ly/39oF3z i.e. using the search string to:zoomaroundtown http://bit.ly/39oF3z.

Whether this is a viable way of enabling commenting Zoom around Town, only time can tell. However there are obvious pros and cons:

PROs:

Extremely easy to implement (especially when I don't have a user login system yet)

Increased site exposure on Twitter

CONs:

Twitter Search only seems to return tweets no more than 2 weeks old

I have no control over content (but since these are public tweets, do I really care?)

Last but not least, something unfortunately I only managed to discover after deployment to the live cloud environment - Twitter Search's rate limiter does not seem to like Google App Engine, and it is this point that will lead us to my next post, but for now it suffices to say that Twitter Search rate-limits requests per IP, so it is not surprising that in a cloud environment where one is sharing IPs with loads of other apps, some requests are bound to get rejected!

Wednesday, October 07, 2009

Circle overlay on Google Map

This is my first "Zoom around Town"-related technical post. This first post looks at the use of GGroundOverlay for rendering circles on Google Maps.

To represent the current search areas in Zoom around Town, I decided to overlay two radar-style circles, on the start point map and on the end point map. These circles have to increase and decrease in diameter depending on the user's range selections. See this page in Zoom around Town for a demonstration. When you change the values of the +/- distance dropdowns, and update the maps, you should see the two circles change in size. The radius of the circle corresponds to the selected +/- value.

While the Google Map API (v2) provides GPolyline and GPolygon for drawing lines and polygons on Google Map, it does not provide an overlay class for drawing circles.

One solution is to create a multi-point GPolygon to approximate a circle. Check out this article for an explanation on how this can be done.

A better (in my opinion) solution is to instead use a GGroundOverlay. GGroundOverlay provides a means to overlay an image on top of the map. The image will be scaled automatically as the user zooms the map, so that the image will always cover the same geography area on the map. So, to create a radar-style circle, one only has to find a transparent circle png file, locate the point on the map where the circle should be centred upon, and specify the southwest corner coordinate and the northeast corner coordinate of the imaginery square that bounds this circle (i.e. the GLatLngBounds). Sounds simple, does'nt it? Well almost, to work out the GLatLngBounds coordinates, we need a bit of good old trigonometry.

Pythagoras told us that, for a 2km-radius circle, the four corners of this bounding square will be sqrt(2*2^2) km from the centre of the circle. So, to draw a 2km-radius circle centred upon a specific point, we need the southwest corner coordinate, which is sqrt(2*2^2) at 225 degrees from the original point and the northeast corner coordinate, which is the same distance at 45 degrees from the original point.

Coordinate of a destination point given distance and bearing from a start point

Unfortunately, as the globe is a sphere, working out a destination point coordinate given the distance and the bearing from a start point is no straightforward maths. Fortunately, the good people at Movable Type Ltd. (not to be confused with http://www.movabletype.org) published this informative page. So based on the formula and the Javascript given on the page, I created the Javascript function:
function getDestLatLng(latLng, bearing, distance) {
var lat1 = latLng.latRadians();
var lng1 = latLng.lngRadians();
var brng = bearing*Math.PI/180;
var dDivR = distance/EARTH_RADIUS;
var lat2 = Math.asin( Math.sin(lat1)*Math.cos(dDivR) + Math.cos(lat1)*Math.sin(dDivR)*Math.cos(brng) );
var lng2 = lng1 + Math.atan2(Math.sin(brng)*Math.sin(dDivR)*Math.cos(lat1), Math.cos(dDivR)-Math.sin(lat1)*Math.sin(lat2));
return new GLatLng(lat2/ Math.PI * 180, lng2/ Math.PI * 180);
}

Now, the rest is easy:
var EARTH_RADIUS = 6378.137; //in kilometres

function drawCircle(map, centrePt, rangeValue) {
var boundaries = getBoundaries(centrePt, rangeValue);
var circle = new GGroundOverlay("/images/map_overlays/circle.png", boundaries);
map.addOverlay(circle);
}

function getBoundaries(centrePt, radius) {
var hypotenuse = Math.sqrt(2 * radius * radius);
var sw = getDestLatLng(centrePt, 225, hypotenuse);
var ne = getDestLatLng(centrePt, 45, hypotenuse);
return new GLatLngBounds(sw, ne);
}

Sunday, October 04, 2009

Zoom around Town is live!

Hi, just been spending many late evenings working on my pet project Zoom around Town. It is now live on http://zoomaroundtown.appspot.com/!

Zoom around Town is a global cycle route database, based on user contribution, to help city cyclists get around town.

Please help by adding your commuting routes and by spreading the word!

This project makes heavy use of the Google Map API and has been developed in Python on the excellent and "free" Google App Engine platform. I will no doubt be blogging about some of the interesting technical encounters during the project in the near future.

Wednesday, August 05, 2009

Datastore Bulk Upload + ReferenceProperty in Google App Engine

I have been working on my first Google App Engine app (a vintage camera database), and I have just got to the point when I need to upload some real data to the Datastore. Essentially, I have an CSV containing all the camera information, including the manufacturer's information and I want to use google.appengine.tools.bulkloader.Loader to upload all this information to Datastore. I have two separate entities representing "camera" and "manufacturer", and I would like to normalize the data as such.

Did someone shout, "Normalization + Bigtable = Bad"? Well, I have yet to make up my mind on this, and for now, I have decided to go with a more traditional relational database model. This may change in the future, but I decided I should still share my experience with uploading relational data as I feel that many people are probably trying to do the same thing.

Before I carry on, these are the two (simplified) classes:


class Manufacturer(db.Model):
name = db.StringProperty(required=True)

class Camera(db.Model):
name = db.StringProperty(required=True)
weight = db.IntegerProperty()
introduction_year = db.IntegerProperty()
max_aperture_value = db.FloatProperty()
e_mode_p = db.BooleanProperty(default=False)
e_mode_sp = db.BooleanProperty(default=False)
e_mode_ap = db.BooleanProperty(default=False)
e_mode_m = db.BooleanProperty(default=False)
note = db.TextProperty()
manufacturer = db.ReferenceProperty(Manufacturer)


and the CSV:


"Canonet G-III QL17",Canon,620,1972,1.7,FALSE,TRUE,FALSE,TRUE,
"Canonet G-III QL19",Canon,620,1972,1.9,FALSE,TRUE,FALSE,TRUE,
Demi,Canon,380,1963,2.8,FALSE,TRUE,FALSE,TRUE,"Demi is a half-frame camera"
"TRIP 35",Olympus,390,1967,2.8,TRUE,FALSE,FALSE,FALSE,


Effectively, I want four instances of camera and two instances of manufacturer created, with the cameras correctly referencing the manufacturers.

Google App Engine comes with the google.appengine.tools.bulkloader.Loader class which facilitates the process of bulk uploading data to both the development server (local) datastore and the real (cloud) datastore. It supports basic mapping of CSV fields to object attributes. However, dealing with ReferenceProperty turns out to be not so straightforward. A quick Google search led to a few articles suggesting overridding the HandleEntity (or handle_entity?) function when subclassing google.appengine.tools.bulkloader.Loader. Unfortunately, for some reason, I did not manage to get that to work. After some experimentation, I stumbled across my own solution, which actually is quite simple :-)

DISCLAIMER: I HAVE ONLY TRIED UPLOADING DATA TO MY LOCAL DEVELOPMENT SERVER (GOOGLE APP ENGINE 1.2.3) USING THIS METHOD, AND NOT TO THE REAL CLOUD YET...

Instead of injecting the reference in the HandleEntity function, I create a function called get_manufacturer which looks in the datastore for an existing manufacturer based on the manufacturer's name, and returns its key (or the instance itself) if found, or creates a new instance in the datastore before returning the newly assigned key otherwise. I then make that the transformation function for the manufacturer ReferenceProperty.

The following is my loader class with the magic function:


from google.appengine.ext import db
from google.appengine.tools import bulkloader
import model


def get_manufacturer(name):
manufacturers = db.GqlQuery("select * from Manufacturer where name = :1", name)
if manufacturers.count() == 0:
newManufacturer = model.Manufacturer(name=name)
db.put(newManufacturer)
return newManufacturer
else:
return manufacturers[0]

class CameraLoader(bulkloader.Loader):
def __init__(self):
bulkloader.Loader.__init__(self, "Camera",
[("name", str),
("manufacturer", get_manufacturer),
("weight", int),
("introduction_year", int),
("max_aperture_value", float),
("e_mode_p", bool),
("e_mode_sp", bool),
("e_mode_ap", bool),
("e_mode_m", bool),
("note", str)])


loaders = [CameraLoader]
if __name__ == '__main__':
bulkload.main(CameraLoader)


Please refer to http://code.google.com/appengine/docs/python/tools/uploadingdata.html for details on how to do the actual bulk upload.

Saturday, June 06, 2009

GWT Tips 3 - Popups and IE8

If you are like me and find that your GWT popups start killing your beautiful application in IE8, do not despair. I found this really useful article: http://www.mooreds.com/wordpress/archives/000513

Many thanks to Dan!

Friday, June 05, 2009

GWT and Spring Security - sample demo download

As requested, I have put together a sample app to demonstrate the GWT and Spring Security (formerly ACEGI) integration technique discussed in http://seewah.blogspot.com/2009/02/gwt-spring-integration-using.html and http://seewah.blogspot.com/2009/02/gwt-and-spring-security.html.

You can download the demo project here: http://www.fotowuj.com/gwt_spring_security_demo.zip

Essentially this app demonstrates how a GWT client can access a Spring-managed authentication service (via AuthenticationServiceServlet) to authenticate against Spring Security, and subsequently access a secured operation (getNumOfPrivatePublications) in another Spring-managed service (via DocumentServiceServlet). Please refer to the aforementioned blog articles for detailed discussion on how to integrate GWT and Spring Security.

You may also want to download a version of GWT.

NOTE THAT I HAVE ONLY TESTED THE DEMO WITH VERSION 1.5.3 OF GWT.

The zip contains a complete eclipse project consisting of:

1) source (both client and server)
2) all dependent jars
3) WebContent/WEB-INF/web.xml, which gets added to the final WAR
4) tomcat/webapps/ROOT/WEB-INF/web.xml, which is required to run in hosted mode
5) build.xml for running app in hosted mode (gwt-shell) and creating war (war)
6) .launch file for running app in hosted mode inside eclipse

You can run the demo in a number of ways:

1) WITH ECLIPSE - having imported the project into eclipse, right-click the .launch file inside eclipse and select "run as...", which will start the app in hosted mode. Note that you will probably have to change the gwt-dev-windows.jar location specified in the .launch file first.

2) RUNNING "gwt-shell" USING ANT - simply run "ant gwt-shell", which will start the app in hosted mode. Note that you will probably will have change the gwt-dev-windows.jar location specified in the build.xml file first.

3) RUNNING "war" USING ANT AND DEPLOYING IN TOMCAT - simply run "ant war", and deploy in Tomcat. Note that you will probably will have change the gwt-dev-windows.jar location specified in the build.xml file first.

Thanks

PS, a reader recently asked the question, "How come that you declare the services (DocumentService for example) both in the client and in the server package? Why not having the DocumentServiceImpl in the server package implement the DocumentService in the client package?"

And this is my anwser:

1) my design is based on the premises that the Spring layer and the GWT layer are very loosely-coupled e.g. in your company, you may already have all the Spring service beans already developed and you are just bolting a GWT frontend on top of this service layer. There are other reasons why you may want your GWT service interface to be slightly different from your Spring service interface as well.

2) The two interfaces ARE DIFFERENT! The GWT service interface getNumberOfPrivatePublications() method throws a checked ServiceSecurityException while the corresponding method in the Spring service interface does not. There may be alternatives that can potentially circumvent this, but to be honest, I have not really thought hard about this :-)

Anyway as I mentioned before, this is by no means the definitive way to integrate Spring Security in GWT. This method works great for me, but depending on your particular circumstances, this may not work so well. I am just hoping to give you guys some ideas :-)

Wednesday, May 13, 2009

OSGi - the next big thing? (a beginner's view)

Eclipse has been using it for years, and a number of open-source and non open-source Java application servers, such as SpringSource dm server and JBoss, are now providing server-side OSGi support.

I have been hearing the term OSGi for quite some time now. I have always known it to be some kind of modularization framework in Java. It is often mentioned alongside with the other wonderful buzzword that is SOA (Service Oriented Architecture). But what is it really? I decided that I should get a full understanding straight from the horse's mouth. So I went to http://www.osgi.org/About/HowOSGi, read the articles and tried out some examples. After a couple of hours, I learnt (and saw for myself) how using OSGi:

- you can divide up your ugly monolithic application into self-contained, loosely-coupled deployment units / modules known as bundles.

- a single bundle exists as a single JAR file.

- a bundle can access functionality provided by other bundles, but each bundle inherent provides a strict level of encapsulation. The creator of a bundle decides what functionality to expose. Everything else inside the bundle is private otherwise.

- bundles can be installed, updated, uninstalled without restarting the application.

- different versions of a single bundle can be deployed concurrently.

- the current OSGi model assumes that all bundles run in the same JVM, so no distributed SOA ("Don’t distribute your objects." - Martin Fowler’s “First Rule of Distributed Object Design" :-) )

The idea of modularization of course has existed since the dawn of software engineering, and OSGi is not the first framework to provide modularization in Java. Yes, this idea of being able to hotswap/upgrade bundles during runtime sounds cool. But does an average IT manager REALLY want to be able to swap in a FTSE-lookup service for a DOW JONES-lookup service during runtime? (Oops I think I have just sparked controvery) My point is while OSGi provides an excellent framework for dynamically linking bundles, developers still has to write extra code to handle when a bundle goes offline, for example. For most projects, this kind of flexibility probably spells more trouble than it is worth. And if non-runtime-configurable dependency injection is what you are after, then just stick with simple Spring!

So I was beginning to think whether OSGi is just yet another "grand" idea.

It then suddenly dawned on me that, forgetting about all this hotswappable bundle magic, the basic idea of self-contained deployment units is the answer to the single biggest cause of Java developers' stress - jar hell. I am sure a lot of you have used or are using Maven, Ivy to manage your hundreds of jars. While they do a decent job of managing dependencies and jar versioning, the simple fact remains - without writing custom classloading code, in Java, you cannot have two versions of a same jar in the same classpath and expect one bit of your application to use the 1st version and another bit of your application (maybe you are using some third-party classes) to use the 2nd version. Now, implemented as OSGi bundles, your different "bits" of the application can be packaged with the required versions of the jars (inside their respective OSGi bundle jar) without interfering with each other. Essentially each bundle has its own classpath/classloader. Is that not great?

OK nothing in this world is free. There is obviously the overhead in having to create these bundles, which is something I have yet to try in a real project. Nonetheless, knowing that there is a chance of forever emancipation from jar hell, there is more of a reason now why I should at least consider trying out OSGi for real! Umm it may be a long road, but perhaps a road worth travelling. Well, at least I know it is possible to embed Equinox in Tomcat. That is a start...

Don't get me wrong. I am not knocking OSGi's ability to dynamically manage and link bundles. I am just saying that the more immediate benefit of adopting OSGi may lie in the fact that you can have many versions of the same jar in your project! Yes, a simple problem, but a real problem.

Any comments welcome!

Thursday, March 05, 2009

Bombay Dreams



"13.7 million people call Mumbai home, but not everyone in Mumbai is lucky enough to have a home."

Recently I made a 36-hour stopover in Mumbai on my way from London to Hong Kong.

28th February 2009

Following advice in the guide book, I ordered a "pre-paid" taxi inside the Chhatrapati Shivaji International Airport. In just under 40 minutes, I was dropped off at the YMCA near Mumbai Central train station. The YMCA is an excellent hotel if you don't mind the fact that it is not located in the touristy South Mumbai. In fact, it turned out that the location was pretty convenient for visiting a number of interesting sites in the city. And how can I not mention the tasty buffet dinner, which is included in the very reasonable room price.

Having filled up my water bottle, I was ready to brave the 37C heat. Initially, it did not feel as scorching as I had expected. However, the combination of heat and constant car-horning was quickly doing my head in. Instead of wandering in the general direction which I had expected to take me through the Kalbadevi neighbourhood, past Crawford Market and onto Chhatrapati Shivaji Terminus, formerly known as Victoria Terminus (CST), I decided that I should take out my map and follow a more precise route. Having previously been to Delhi and several Rajasthan cities, I had an idea of how tough life can be in cities for the less privileged. Walking down the side streets, I soon realised that life in Mumbai for the marginalized was no better than what I had witnessed before, if not worse. The clusters of corrugated iron-roof huts (for the more lucky ones) hiding timidly behind the main thoroughfares were a constant reminder of the millions of slum dwellers in Mumbai, who ironically serve as the unsung powerhouse of the city's economy.

I soon arrived at the first stop, Mumbaidevi Temple in Kalbadevi. Mumbaidevi is the patron goddess after which the city took its name. The entrance to the temple is notoriously hard to find, but the guidebook clearly said there would be a long queue of faithfuls, which I could join to enter the temple. People were busy running around in all directions, but no queue of any description was to be found. I referred to the guidebook again, it also clearly said it is closed between noon and 4pm. Of course I had just managed to overlook this minor detail and had arrived during their siesta. Doh.

Finding Crawford Market was easy enough as it stood right by an unsightly flyover. A lot of the stores were closed when I arrived, but the cover market provided a much needed shelter from the increasingly tormenting sun. After bidding farewell to puppies housed in tiny cages (which presumably would one day be sold to someone with a stronger belief in animal wellfare), I carried on walking along the flyover to reach Chhatrapati Shivaji Terminus. I had once read that the train station was home to hundreds of street kids. I did not see any when I was there. Part of the exterior of the station was undergoing renovation, but the grandeur of the place still made it a magnificent sight to behold. From the grandeur of Chhatrapati Shivaji Terminus, I crossed a busy road Indian-style to reach the convenience that is universally known as McDonald's. In an attempt to not feel completely shameful visiting McD in India, I ordered a McAloo Tikki. Sadly, it tasted not much different from any of the other McInventions. It did the job nonetheless.

With a full stomach and a pair of less tired legs, I ventured further south to visit the Horniman Circle and the Ballard Estate / Dockyard districts. The broad tarmac roads provided perfect venues for many impromptu cricket games. Luckily for all those aging colonial buildings, tennis balls were used. And of course passing cars have to give way to the cricketers. I was disappointed to find that the whole of the waterfront was a restricted area due to national security, so no glorious view of the Gateway of India from across the water.

It was time to make my way back to the YMCA. I passed Mohammed Ali Road on the way. The religious divide (or harmony, depending on how you look at it) in India cannot be better illustrated than by the two surrounding neighbourhoods: the Islamic neighbourhood to the east and the Hindu neighbourhood to the west.

1st March 2009

After a hearty breakfast at the YMCA, I decided to walk to Malabar Hill and to the Hanging Garden. The garden was not all that, but any excuse to sit around in the shade was a good one. Visiting the excellent Jain Temple on the way, I descended the hill taking Walkeshwar Road to reach Chowpatty Beach. Looking out onto the Manhattan of Mumbai across the bay, the beach does not only provide a weekend escape for millions of Mumbai residents but also another plot of land for the marginalized to sleep under the stars.

I finally gave up walking and took the train from Charni Road to Churchgate. The guidebook did not paint a very appealing picture of this mode of transport, which is an integral part of the daily lives of the millions of Mumbai commuters. The journey was painless enough and in no time I arrived at Churchgate. With students playing crickets in a quintessential English setting, Churchgate would most certainly get Michael Palin's approval.

Not far from Churchgate was the Gateway of India, which had been built to commemorate the visit and King George V and Queen Mary to Mumbai, and the Taj Mahal Palace and Tower Hotel. A sombre mood surrounded the imposing hotel, which was still undergoing renovation following the terrorist attack on the 26th of November, 2008. There was a sense of defiance as local and visiting Indians gathered along the waterfront to reflect upon what had happened.

My stay in Mumbai was quickly coming to an end, but not just yet. It was 4:45pm and seeing that I had to be back at the YMCA by 7pm to catch a taxi for the airport, I had to decide whether to take a taxi from Churchgate back to the hotel or to take the train. Taxi would be the more comfortable option but there might be a chance of being stuck in the horrendous traffic. Somehow, the train seemed to make more sense at that time. I thought, if I ran to the train station quickly, I would just manage to beat the rush hours.

It turned out I was very wrong. I got onto the train without any problem as Churchgate was the right at the end of the train line, but when more and more people poured inside the carriage as the train made its way north, I came to realise that taking the train was essentially a very bad idea. I had to get off at the next stop, Mumbai Central, but I was stuck in the middle of the carriage, naively hoping that people would somehow make way for me to exit the train. The fact that I was not sure which side to leave the carriage did not help. It became obvious that I was going nowhere as the train ground to a halt. People were already literally fighting their way onto the train. Next thing I knew, the train was in motion again. While my predicament provided entertainment for some fellow passengers, others took sympathy and urged me to push my way towards the door (incidentally there were no actual doors) in preparation to do battle at the following station. I was in pole position, but the whole world collapsed on me when a tsunami of bodies came crushing in before the train even came to a stop. I did finally make it off the train, but not without getting an elbow in the face. It was simply the most traumatic experience. I had to take the train in the opposite direction to get back to Mumbai Central. Luckily, the train going south was quite a lot less packed. Standing by the door as the train zoomed past industrial estates and residential high-rises, I was still struggling to get over the previous shock.

The taxi journey to the airport was blissful by comparison. I tried to absorb the smell of this truly overwhelming city. I tried to understand the contrast between the rich and the poor that I was seeing outside the taxi window. Under a bridge, I saw a family sitting around a dimly-lit fire. 13.7 million people call Mumbai home, but not everyone in Mumbai is lucky enough to have a home.

Wednesday, March 04, 2009

Shek O 石澳 bouldering

Spring is here. Hong Kong is WET!

Weather forecast was predicting at least five days of rain, which meant any hope of bolt-clipping in Tung Long during my short stay in HK was effectively gone, so before it started getting properly wet, I decided to give Shek O a go.

Taking my rock shoes and the excellent guide provide free by the kind people at http://www.hongkongclimbing.com/ (and no, I did not have a crash mat!), I took bus no.9 from the bus terminal just outside the Shau Kei Wan (筲箕灣) MTR station. The journey took about 15 minutes, the normally scenic route was sadly not very scenic in the light drizzle and the heavy mist.

Arriving at the village, I followed the aforementioned guide and found my way to the boulder spots easy enough. The laid-back setting of the village itself provided the perfect antidote to my last few days of being constantly bombarded by car-horns in Mumbai and by people talking about the latest news on the fund-raising plan by HSBC on their mobiles in Hong Kong. Even though, I suspect if it had been a weekend and if it had not been raining, the place would have been heaving like the rest of Hong Kong!

I warmed up on a couple of V1 problems on "Block 3" in the Village Boulders area just before the footbridge. The holds were positive enough and the friction was certainly good on the granite, but I was constantly being reminded by the scattered blocks underneath my feet of the certain possibility of my ankles breaking if I was going to fall. I did not have the nerve to try the harder problems which consisted mostly of slopers for hands and flat-smearing on the slightly overhanging boulder.

I then crossed the footbridge to look for "The Corridor" but did not hang around long as, again, technical climbing was the order of the day and I was not getting over the fear of potentially breaking my ankles. Some problems certainly looked interesting and I wished I had a crash mat.

Following the path, I made my way to the "Headlands Boulders" area. "Lone Block" was definitely fun. The V0 problem was a well-deserved 3-star, albeit being a bit short. I also managed the 360 traverse but could not do it in one go. Unfortunately the area stank of piss (or maybe it was something coming from the concrete hut next to it), which was a shame as the atmosphere was perfect otherwise (despite the constant drizzling, which I had kind of forgotten about by that point). I then moved on and tried out the three aretes on the "Cliff Top" boulders. The V0 was easy enough and it was kind of fun, but the V1's did not have much in terms of hand holds on them and, again, technical climbing on hard landing was not quite what I was prepared to put myself through. I backed down both climbs.

The drizzle was getting more and more annoying, and being disheartened by my failure to climb the two v1's, I decided to call it a day.

Pretty crap climbing (not a reflection on the quality of the area, but rather a reflection on my own incompetence and the fact that a crash mat is pretty essential), but 30 mins from my doorsteps to a bit of rockclimbing above crashing waves, I could not really complain!

Tuesday, February 24, 2009

GWT Tips 2 - nocache.js getting cached in browser

I have recently discovered that, after version deployments, our GWT application does not work in some users' browsers unless they do a hard refresh on the browser.

A bit of investigation revealed that this is because their browsers are caching the all-important YOUR_MODULE_NAME.nocache.js file. This file is GWT's bootstrapping file and its content (not its name) is likely to change as you introduce changes and then gwt-compile your Java code. If a user has an older version of this js file cached in the browser, the application will not be able to properly communicate with your server code. One should make sure that this file does not get cached by the browser nor any proxy servers along the way.

You may wonder what is the significance the of the "nocache" bit in the name. What does it actually do? Well, it does not do anything! But having this identifiable string in the filename does allow you to configure your server to send HTTP header information to the client browser to stop it from caching the file.

If you are using Apache to serve your static files. You can simply edit your .htaccess file to send the correct HTTP header information to the client for this nocache.js file.

However, if you are using a Java application server such as Tomcat to server your static files, you have to create a servlet Filter to dispatch the necessary HTTP header information. Below is a filter I have written to do exactly this:



import java.io.IOException;
import java.util.Date;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
* {@link Filter} to add cache control headers for GWT generated files to ensure
* that the correct files get cached.
*
* @author See Wah Cheng
* @created 24 Feb 2009
*/
public class GWTCacheControlFilter implements Filter {

public void destroy() {
}

public void init(FilterConfig config) throws ServletException {
}

public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain) throws IOException,
ServletException {

HttpServletRequest httpRequest = (HttpServletRequest) request;
String requestURI = httpRequest.getRequestURI();

if (requestURI.contains(".nocache.")) {
Date now = new Date();
HttpServletResponse httpResponse = (HttpServletResponse) response;
httpResponse.setDateHeader("Date", now.getTime());
// one day old
httpResponse.setDateHeader("Expires", now.getTime() - 86400000L);
httpResponse.setHeader("Pragma", "no-cache");
httpResponse.setHeader("Cache-control", "no-cache, no-store, must-revalidate");
}

filterChain.doFilter(request, response);
}
}



Essentially, this filter checks whether the requested file name contains the string ".nocache.", and if it does, it sets the header to tell the browser and any proxy servers along the way to not cache the file. For an excellent explanation of the different header options, check out http://code.google.com/p/doctype/wiki/ArticleHttpCaching.

Of course, you would have to add this filter to your web.xml (host-mode as well as standalone mode):


<filter>
<filter-name>gwtCacheControlFilter</filter-name>
<filter-class>com.seewah.blog.GWTCacheControlFilter</filter-class>
</filter>

<filter-mapping>
<filter-name>gwtCacheControlFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Monday, February 23, 2009

Spring Tips 1 - reading encrypted passwords in properties file

Have you ever wondered how you can use encrypted passwords in Java properties files that get loaded into your Spring ApplicationContext via a org.springframework.beans.factory.config.PropertyPlaceholderConfigurer?

Here is the answer - http://forum.springframework.org/showpost.php?s=6d689f6afe0143a9ab3a77bfbb8e017c&p=7965&postcount=6 (thanks to none other than Juergen Hoeller)

Pretty slick!

Friday, February 20, 2009

GWT Tips 1 - Chain of Responsibility

GWT application development (and AJAX programming in general) can sometimes prove problematic for web developers who has not yet adopted the mindset of asynchronous programming. In GWT, you make a RPC as follows:


foo.bar(param, new AsycnCallback<object>) {
public void onFailure(Throwable throwable) {
// handle exception
}
public void onSuccess(Object result) {
// successful execution of asynchronous RPC
}
}


What if you have to make a series of RPC calls such that an RPC call only gets invoked after the previous call has succeeded. Of course we can simply wrap this series of RPC calls inside one single RPC on the server side so that the client does not have to worry about synchronising the calls, but you maybe you want to update the user interface between calls, or you may want to execute some client-side logic before making the second call.

You may start off writing code like this:


foo.bar(param, new AsycnCallback<object>) {
public void onFailure(Throwable throwable) {
...
}
public void onSuccess(Object result) {
// some client-side logic before invoking the second RPC
foo2.bar2(param, new AsycnCallback<object>) {
public void onFailure(Throwable throwable) {
....
}
public void onSuccess(Object object) {
....
}
}
}
}


This is hard to follow and error-prone. A way to avoid nested RPC calls is to leverage the Chain of Responsibility design pattern. The main class in the design pattern is the AbstractHandler. The following is a basic implementation of this class:


/**
* An abstract handler in the Chain or Responsibility design pattern.
*
* @author See Wah Cheng
* @param <T>
* type of request object to handle
*/
public abstract class AbstractHandler<T> {

private Handler<T> successor;

public void setSuccessor(Handler<T> successor) {
this.successor = successor;
}

public Handler<T> getSuccessor() {
return successor;
}

public void handOverToSuccessor(T request) {
if (successor != null) {
successor.handleRequest(request);
}
}

public abstract void handleRequest(T request);
}


Given this class, to execute a series of RPC calls, one simply has to create a concrete implementation of this AbstractHandler for each RPC call and chain them together as follows: (where InvocationContext is some kind of class encapsulating the details needed for the various RPC invocation)


public class FooInvoker extends AbstractHandler<Invocationcontext> {
public void handleRequest(InvocationContext request) {
Object param = request.getFooParam();
foo.bar(param, new AsycnCallback<object>) {
public void onFailure(Throwable throwable) { // handle exception }
public void onSuccess(Object result) {
// some client-side logic before invoking the next RPC
// updating request object if necessary
handOverToSuccessor(request);
}
}
}
}

public class Foo2Invoker extends AbstractHandler<InvocationContext> {
public void handleRequest(InvocationContext request) {
Object param = request.getFoo2Param();
foo2.bar2(param, new AsycnCallback<object>) {
public void onFailure(Throwable throwable) { // handle exception }
public void onSuccess(Object result) {
// some client-side logic before invoking the next RPC
// updating request object if necessary
handOverToSuccessor(request);
}
}
}
}

InvocationContext request;
...
FooInvoker fooInvoker = new FooInvoker();
Foo2Invoker foo2Invoker = new Foo2Invoker();
fooInvoker.setSuccessor(foo2Invoker);
fooInvoker.handleRequest(request);


OK you may be shouting, "Isn't this a lot of code for something pretty simple? Er, agile? Hello?" Yes, it is more verbose, and whether this approach offers any benefit depends on the situation. If you are making three or more RPC calls in series, I definitely think this will lead to better quality code. If, inside each callback's onSuccess method, you are doing more than just invoking the next RPC, this will certainly lead to better code. If the individual stages are needed elsewhere, this again will lead to more reusable code.

Finally, an added benefit of this is that the order of execution can be altered quite easy just by shuffling the handler order in the chain (assuming that the order of execution does not matter!)

Wednesday, February 18, 2009

GWT and Spring Security

Update: I have put together a sample app to demonstrate the integration technique: http://seewah.blogspot.com/2009/06/gwt-and-spring-security-sample-demo.html

As promised, I am posting this article to share my experience with using Spring Security (formerly ACEGI) with GWT. Apologies for the delay. I guess I have just been busy + lazy!

First of all, for those of you who haven't, please read my previous blogs GWT Spring Integration and GWT Spring Integration - using ContextLoaderListener to see how to use Spring-managed service beans in GWT.

Again it is not my intention to provide a complete solution for GWT / Spring Security integration as there are so many ways of doing it, and which method to choose depend hugely on how you want to actually use Spring Security inside GWT. By sharing my limited experience here, I hope to give some ideas to those of you who are trying to come up with a solution that fits your needs.

So how do I want to use Spring Security in my GWT application?

I want to:

1) leverage Spring Security's SecurityContext mechanism to store authenticated token, which I can then access anywhere in my code
2) be able to annotate my service classes, so that Spring can apply method security using AOP to my service beans

I DO NOT want to:

1) let Spring Security automatically handle form authentication for me
2) rely on Spring Security to automatically redirect user to login page

In other words, using Spring Security, I want to develop some form of RPC service which my GWT client can call to authenticate the username and password supplied by the user at the login screen. Once authentication has succeeded, a token which identifies the current user as being authenticated will be available to all subsequent RPC requests, so that I can either manually inspect the token inside the RPC request to enforce some form of authorization, or I can rely on Spring Security's off-the-shelf method security mechanism to allow or disallow user call certain methods in my Spring managed service beans.

I am going to break up the rest of the article into three different sections: An authentication service, HTTP Session Integration and Method Security

An Authentication service

The following is a simple implementation of such a service:


import org.springframework.security.Authentication;
import org.springframework.security.context.SecurityContext;
import org.springframework.security.context.SecurityContextHolder;
import org.springframework.security.context.SecurityContextImpl;
import org.springframework.security.providers.UsernamePasswordAuthenticationToken;
import org.springframework.security.userdetails.User;

public class AuthenticationServiceImpl implements AuthenticationService {

/**
* Authenticates against the supplied credentials,
* populating the Spring SecurityContext if
* credentials are OK.
*
* @param username
* @param password
* @return
* whether authentication is successful
*/
public boolean authenticate(String username, String password) {

// check credentials, e.g., by querying a database...
boolean authenticated = checkCredentials();

// look up authorities by, e.g., querying the database...
GrantedAuthority[] authorities = getGrantedAuthorities(username);

if (authenticated) {
User user = new User(username, password, true, true, true, true, authorities);
Authentication auth = new UsernamePasswordAuthenticationToken(user, password, authorities);
SecurityContext sc = new SecurityContextImpl();
sc.setAuthentication(auth);
SecurityContextHolder.setContext(sc);
}

return authenticated;
}

public void logout() {
SecurityContextHolder.clearContext();
}
}


This of course is a very simple implementation, but the point to note is within the authenticate method, an Spring Authentication object is created and inserted into the SecurityContext programmatically.

We can simply turn this into a RPC service by creating a com.google.gwt.user.server.rpc.RemoteServiceServlet implementation in which we either access AuthenticationServiceImpl directly or via a Spring ApplicationContext as described in my previous blogs GWT Spring Integration and GWT Spring Integration - using ContextLoaderListener.

Now we have a service that effectively let us "log into" the system. However, this on its own is rather useless, as the token is not available to subsequent request threads. If we have a RPC service for retrieving documents, and we would like to protect the documents, we really would want to be able to, inside the service, inspect the SecurityContext to see whether an authenticated token exists. We want be able to do the following, which we cannot do... yet:


if (SecurityContextHolder.getContext() == null
SecurityContextHolder.getContext().getAuthentication() == null
!SecurityContextHolder.getContext().getAuthentication().isAuthenticated) {

// no authenticated token found, stop user from retrieving documents
}


HTTP Session Integration

Luckily, Spring Security provides a filter, org.springframework.security.context.HttpSessionContextIntegrationFilter, which automatically synchronises the current SecurityContext with a http session object, thus, making the token available to all subsequent request threads.

We can enable this filter in a number of ways. We can create the filter directly in web.xml (host-mode or standalone mode). Or we can create a org.springframework.security.util.FilterToBeanProxy in web.xml and define a FilterChainProxy in the Spring ApplicationContext. Or we can, as most people do nowadays, create a org.springframework.web.filter.DelegatingFilterProxy called springSecurityFilterChain (note: the naming is crucial) in web.xml and use Spring Security Namespace Configuration in the ApplicationContext to create the necessary beans. We will have a look at this last approach.

In web.xml


<filter>
<filter-name>springSecurityFilterChain</filter-name>
<filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
</filter>

<filter-mapping>
<filter-name>springSecurityFilterChain</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
<listener>

<listener>
<listener-class>org.springframework.security.ui.session.HttpSessionEventPublisher</listener-class>
</listener>


In Application Context:


<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:security="http://www.springframework.org/schema/security"....>

<security:http entry-point-ref="dummyEntryPoint" create-session="always" />

<bean id="dummyEntryPoint" class="com.seewah.blog.DummyEntryPoint" />
...
<beans>


This is pretty neat, except for the fact that we have to create a "dummy" org.springframework.security.ui.AuthenticationEntryPoint, even though we are not replying on Spring Security to redirect user to login page. Unfortunately, by design, the security:http tag requires an AuthenticationEntryPoint, so I created this class:


public class DummyEntryPoint implements AuthenticationEntryPoint {

public void commence(ServletRequest request, ServletResponse response, AuthenticationException e) throws IOException, ServletException {

throw new IllegalStateException("This implementation is a dummy class, created purely so that spring security namespace tags can be used in application context, and this method should never be called");
}
}


OK, so now we have a way to "log into" the system, and we can inspect the SecurityContext in subsequent RPC requests to grant or deny users access to certain services. However, we would like to go one step further. We would like to leverage the excellent method security feature provided by Spring Security because we like our security layer to be non-invasive, and we like AOP.

Method Security

We configure method security as follows:

In Application Context:



<security:global-method-security secured-annotations="enabled" jsr250-annotations="disabled" />

<bean id="dummyAuthenticationProvider" class="com.seewah.blog.DummyAuthenticationProvider">
<security:custom-authentication-provider />
</bean>


Unfortunately, again, because of the way Spring Security Namespace Configuration works, we have to introduce another dummy class, even though it will never be used. Remember our AuthenticationServiceImpl creates authenticated token programmatically and method security will simply grant / deny access based on the presence of this token. Hence the authencationProvider here becomes redundant. We implement the dummy class as follows:


public class DummyAuthenticationProvider implements AuthenticationProvider {

public Authentication authenticate(Authentication authentication) throws AuthenticationException {

throw new IllegalStateException("This implementation is a dummy class, created purely so that spring security namespace tags can be used in application context, and this method should never be called");
}

@SuppressWarnings("unchecked")
public boolean supports(Class clazz) {

throw new IllegalStateException("This implementation is a dummy class, created purely so that spring security namespace tags can be used in application context, and this method should never be called");
}
}


Of course, as method security is implemented using AOP in Spring, only Spring-managed beans are protected, which means in order to use this mechanism to protect our service layer, we MUST access these services via the Spring ApplicationContext (GWT Spring Integration and GWT Spring Integration - using ContextLoaderListener) and NOT directly.

When a protected method is accessed without an authenticated token being present in the SecurityContext, Spring throws a SpringSecurityException. If we let this exception propagate all the way to the GWT client code, we are most certainly going to be greeted with the dreaded SerializableException error. Instead we should catch this exception and rethrow an exception that GWT can safely serialize and pass to the client side. We define such an exception, ServiceSecurityException, in our GWT client package, and now we have a mechanism that translates a Spring Security method security exception into a GWT client exception:

On the server


public class DocumentServiceImpl implements DocumentService {

@org.springframework.security.annotation.Secured("ROLE_ADMIN")
public void delete(Long id) {
...
}
}

public class DocumentServiceServlet extends DependencyInjectionRemoteServiceServlet implements ... {

@Autowired
private DocumentService documentService;

public void delete(Long id) throws ServiceSecurityException {
try {
documentService.delete(id);
} catch (SpringSecurityException e) {
throw new ServiceSecurityException(e.getMessage());
}
}
....
}



Inside the client


documentServiceAsync.delete(id, new AsyncCallback<Object>() {
public void onFailure(Throwable throwable) {
if (throwable instanceof ServiceSecurityException) {
// client side logic to redirect user to login page, for example
} else {
...
}
}
...
});


Of course, seeing that we need client-side exception handling whenever we invoke an "secured" RPC call, we may as well create an AutoErrorHandlingAsyncCallback and invoke all our RPC calls passing through an instance of this callback instead of the normal AsyncCallback object:


public abstract class AutoErrorHandlingAsyncCallback<T> implements AsyncCallback<T> {

final public void onFailure(Throwable throwable) {
if (throwable instanceof ServiceSecurityException) {
// client side logic to redirect user to login page, for example
} else {
...
}
}
}


OK this way of using Spring Security method security is not as non-invasive as we have perhaps hoped, but IMHO this is far better than writing code to check for authenticated token in SecurityContext in the services themselves.

GWT Spring Integration - using ContextLoaderListener

Update: I have put together a sample app to demonstrate the integration technique: http://seewah.blogspot.com/2009/06/gwt-and-spring-security-sample-demo.html Just take out applicationContext-security.xml if you are not interested in the Spring Security aspect!

Just a quick followup post to my last post http://seewah.blogspot.com/2008/07/gwt-and-spring.html.

In the last post I demostrated how to use the SpringApplicationContextLoader to manually load an application context from an xml file and subsequently to access Spring-managed beans via this class inside DependencyInjectionRemoteServiceServlet.

For one reason or another, I have moved away from this approach and gone back to the traditional means of using the org.springframework.web.context.ContextLoaderListener in web.xml to load application contexts. So this is what to do if you prefer this approach.

First of all, create the listener as you would with a normal Spring web application


<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>


in web.xml. If you are running in host mode, this is found under the tomcat/webapps/ROOT/WEB-INF folder inside your project. Otherwise, just edit the usual web.xml inside your webapps folder in your tomcat installation.

Now, inside DependencyInjectionRemoteServiceServlet, simply use the static org.springframework.web.context.support.WebApplicationContextUtils.getWebApplicationContext(ServletContext sc) method to access the loaded application context. This following is the revised class:


import java.lang.reflect.Field;
import java.util.HashSet;
import java.util.Set;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.NoSuchBeanDefinitionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.context.support.WebApplicationContextUtils;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;
/**
* {@link RemoteServiceServlet} that automatically injects IoC dependency.
* "org.springframework.beans.factory.annotation.Autowired" annotation is used
* for marking which fields to inject into. Uses
* {@link SpringApplicationContextLoader} to retrieve beans by name.
* <p>
* Note that the current implementation will only inject "declared" fields, and
* not inherited fields. Fields can be private, protected, package or public.
*
* @author See Wah Cheng
* @created 27 Jun 2008
*/
@SuppressWarnings("serial")
public class DependencyInjectionRemoteServiceServlet extends RemoteServiceServlet {

protected static Logger logger = Logger.getLogger(DependencyInjectionRemoteServiceServlet.class);

@Override
public void init() throws ServletException {
super.init();
doDependencyInjection();
}

/**
* Carries out dependency injection. This implementation uses Spring IoC
* container.
*
* @exception NoSuchBeanDefinitionException
* if a suitable bean cannot be found in the Spring application
* context. The current implementation looks up beans by name
*/
protected void doDependencyInjection() {

for (Field field : getFieldsToDependencyInject()) {
try {
boolean isFieldAccessible = field.isAccessible();
if (!isFieldAccessible) {
field.setAccessible(true);
}
field.set(this, WebApplicationContextUtils.getWebApplicationContext(getServletContext()).getBean(field.getName()));
if (!isFieldAccessible) {
field.setAccessible(false);
}
logger.debug("Dependency injection successful: " + this.getClass().getName() + "." + field.getName());
} catch (IllegalArgumentException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
}

/**
* Find annotated fields to inject.
*
* @return a list of all the annotated fields
*/
private Set<Field> getFieldsToDependencyInject() {
Set<Field> fieldsToInject = new HashSet<Field>();
Field[] fields = this.getClass().getDeclaredFields();
for (Field field : fields) {
if (field.getAnnotation(Autowired.class) != null) {
fieldsToInject.add(field);
}
}
return fieldsToInject;
}
}


Note that there are two changes to this class. First of all, doDependencyInjection() now accesses the application context using the org.springframework.web.context.support.WebApplicationContextUtils class as described above. Secondly, doDependencyInjection() is now invoked from inside the servlet's init() method and not its constructor. This is because, when org.springframework.web.context.support.WebApplicationContextUtils.getWebApplicationContext(ServletContext sc) is invoked inside doDependencyInjection(), a reference to the servlet's ServletContext must be obtained, and this is only available after the servlet object has been constructed.