Using Hazelcast as an OSGi blueprint service

Posted: May 4th, 2014 | Author: | No Comments »

During the last years I have been involved in a very innovative product to manage security policies in networks. The premises for the product were that the product should be scalable and to have high availability capabilities. Besides the product should be modular, enabling extensions through plugins. For the second requirement we decided to use OSGi with Blueprint (specially, Gemini and Spring) and Eclipse Virgo with Jetty as the server. For the HA requirement, finally we decided to use Hazelcast to allow sharing data between nodes. Hazelcast is a powerful technology that provides an easy API and with almost zero-configuration you can distribute data structure like user session in a cluster.

By the other hand, the benefits of using OSGi are well-know and its programming model improves the decoupling of components due to the differentiation between interface and implementation through differents bundles. This splitting allows technology and framework abstraction through service-oriented paradigm. In our case, Hazelcast is used as a custom implementation of a CacheManagerService that provides easy access to backed maps.

The CacheManagerService looks like:

public interface CacheManagerService {

         Map getCache(String name);
}

This simply interface provides a method to get a named map that will be a distributed map thanks to Hazelcast. The implementation (in another bundle) looks like:

public class HazelcastCacheManagerService implements CacheManagerService {

        private HazelcastInstance instance;

        public void setInstance(HazelcastInstance instance) {
                this.instance = instance;
        }

        public HazelcastInstance getInstance() {
                return instance;
        }

        @Override
        public  Map getCache(String name) {
                return getInstance().getMap(name);
        }
}

As the snippet shows the use of Hazelcast is very easy, delegating the way of retrieve the map in this framework. You can create the Hazelcast instance in another Spring context, configurating the name, IPs, etc, and setting this instance when you create this bean.

So far, the idea is quite simple, to use this service the first step is to create the bean, expose as an OSGi blueprint service and consuming in another bundle by importing just the CacheManagerService API bundle. With this approach the Hazelcast implementation maintain hidden because Blueprint and Spring use Java proxies to inject the service implementation in the consumer beans.

What’s wrong with this approach?

Ok, if you have some experience with OSGi, the classloading problem is known for you. Sometimes, it’s hard to deal with import-package directive in MANIFEST.MF, especially when the classes you have to import aren’t used by you directly. With Hazelcast (or another library that needs reflection) the problem is maybe a bit more complicated because when Hazelcast needs to serialize objects, it needs the definition class in its classloader, so the bundle with Hazelcast implementation should have as many import-package as it needs to serialize. This strategy doesn’t look too good, because we are losing extensibility and modularity due to the Hazelcast bundle needs to import each class we want to put in a distributed map. The MANIFEST.MF file of this bundle will be changed in accordance we add new bundles that consume the CacheManagerService. This sounds really bad :(

Solution

Googling a little bit, it seems it’s a common problem when dealing with this kind of libraries. A solution could be use the directive DynamicImport-Package, but with this solution you lose the control of the package version you want to import. So, taking advantage the Hazelcast allows us to control its classloader through its instance, we can extend this classloader at will. To do that, we inject a custom classloader to Hazelcast instance, after that we inject this instance to HazelcastCacheManagerService. The custom classloader is a CompositeClassLoader like this, that aggregates different classloaders from each bundle that consumes the cache service.

Finally, we need to control how the different classloaders are added to this CompositeClassLoader. A trick that Gemini Blueprint offers is to access to the bundle context that invokes the method and get its classloader.

LocalBundleContext.getInvokerBundleContext().getBundle().adapt(BundleWiring.class).getClassLoader();

Once this has been done, Hazelcast can serialize every class the consumer bundle can access. The code of HazelcastCacheManagerService looks like:

public class HazelcastCacheManagerService implements CacheManagerService {

        // ... previous code

        public void init() {
                addInvokerClassLoader(getClass().getClassLoader());
        }
        @Override
        public  Map getCache(String name) {
                addInvokerClassLoader(getInvokerClassLoader());
                return getInstance().getMap(name);
        }
        protected ClassLoader getInvokerClassLoader() {
                return LocalBundleContext.getInvokerBundleContext().getBundle().adapt(BundleWiring.class).getClassLoader();
        }

        protected void addInvokerClassLoader(ClassLoader cl) {
                ((CompositeClassLoader) getInstance().getConfig().getClassLoader()).add(cl);
        }
        // ...
}

The method init add the own bundle classpath to the CompositeClassLoader and should be declared as init-method in Spring configuration (or annotated with @PostConstruct). Then, for each getCache invocation the classloader of the client will be added to the CompositeClassLoader. To prevent duplicated classloader entries in the CompositeClassLoader we change the List for Set as a backed collection.


Tags: , , , , ,

Leave a Reply