1.0 Introduction
One of the fundamental goals of Cloud computing is optimal usage of limited resources. This involves maximizing ROI for all parties concerned, and making sure that all parties win. Multi-tenancy is a core attribute of a Cloud deployment. We have to ensure that all available resources can be optimally utilized by many tenants as possible. Lazy loading plays a major role in resource usage optimization.
In this article, we will take a look at how deployment artifacts (webapps, service artifacts etc) can be optimally loaded in a Platform-as-a-Service (PaaS) implementation. We will describe how we have implemented lazy loading in the WSO2
Stratos Cloud Middleware Platform & the WSO2
StratosLive PaaS.
2.0 Lazy Loading
Lazy loading is a widely used & well-known concept in the field of computing. There are
four different variants of the lazy loading pattern, as documented by Martin Fowler. The differences between these variants are subtle.
1. Lazy Initialization
A null field indicates that there is no data. When the value is requested, a null check is performed to see if the actual data needs to be loaded.
2. Virtual Proxy
The virtual proxy implements the same interface as the real object, and when called for the very first time, it loads the real object & delegates to that object.
3. Value Holder
A value holder is an object with a getValue method, which the clients will invoke in order to obtain a reference to the real object. Note that the method may not necessarily be named getValue.
4. Ghost
The real object without any data. The data is loaded as and when required.
In the next sections of the article, we will explain how we used the
Value Holder &
Ghost variants of the lazy loading pattern in the
Stratos Cloud Middleware Platform & the
StratosLive PaaS.
3.0 Lazy Loading Tenant AxisConfiguration
The Stratos deployment model revolves around the Axis2 deployment model. We have found the
Axis2 deployment model to be a very flexible & powerful one, hence we decided to go ahead with the Axis2 deployment model for deploying artifacts in the WSO2
Carbon middleware platform & WSO2
Stratos Cloud middleware platform. The Axis2 deployment model totally isolates artifact deployment & classloader isolation, hence is a very good model to adopt for a multi-tenanted Cloud deployment.
When the WSO2 Stratos processes start up, we do not load all the tenants into memory. Obviously, it will not scale if we were to do so. Tenants are loaded only when they are needed, and if a tenant is not being used for a certain time period, we unload such tenants. Tenants will be loaded in the following scenarios;
1. A request is received for an artifact deployed by the tenant.
These can be requests to webapps, Web services, proxy services, tenant UI components and so on.
2. A tenant's management console view is loaded.
Tenant's can manage services in their view of the PaaS using the management console. When tenants want to perform such management activities, we load the tenant.
3. A periodic task scheduled for the tenant gets activated
In a PaaS, tenants should be able to schedule tasks that run periodically. In such cases, when the task starts to run, if the tenant has not been loaded, we will load the tenant.
In WSO2 Stratos, loading a tenant mainly translates to creation of an Axis2 Configuration & ConfigurationContext in programming terms. When an AxisConfiguration is created for a tenant, a series of OSGi service calls results in the relevant permission scheme & other metadata being loaded. These are lightweight service calls that do not impose high overhead.
We have used the ValueHolder variant of lazy loading in this instance. The relevant method call is;
TenantAxisUtils.getTenantConfigurationContext(String tenantDomain)
The getTenantConfigurationContext method is the equivalent of the getValue method in the ValueHolder. This method will create & return a new Axis2 ConfigurationContext for that tenant domain, if it does not already exist, and will return the already created ConfigurationContext if the tenant has already been loaded.
One shortcoming of loading the entire AxisConfiguration is that all artifacts belonging to that particular tenant will also be loaded. Hence, the time taken for the execution of this method will increase with the number of artifacts deployed by the tenant. This can lead to unfavorable consequences such as the very first request received for a tenant always timing out. Such requests are targeted at a single artifact, but loading the AxisConfiguration means the entire Axis2 repo gets loaded, hence we will end up loading all artifacts, leading to unnecessary memory usage & processing time. In section 4.0, we will describe how we can overcome this issue by using the Ghost variant of the lazy loading pattern.
4.0 Lazy Loading Tenant Deployment Artifacts
As described in section 3.0 above, loading an entire tenant AxisConfiguration can lead to increasing first-request response times since all artifacts have to be loaded. We can overcome this by loading the required artifact into a tenant's AxisConfiguration on demand.
We use the Ghost variant of the lazy loading pattern to accomplish this. Let's take a look at how normal artifact deployment works.
 |
| Figure 1: Deployment without Ghost Deployer & Ghost Dispatcher |
As shown in the above diagram, a set of Axis2 deployers are registered for each artifact type. The DeploymentEngine maintains a collection of such deployers, and then when an AxisConfiguration is created, it will call all the deployers to load all the artifacts found in the Axis2 repository. The deployers will load all the artifacts into memory. Needless to say, this will take a long time if there are many artifacts to be loaded. Next let us look into the concept of Ghost artifact deployment, to see how we reduce this initial loading time.
 |
| Figure 2: Ghost Deployment with Ghost Deployer & Ghost Dispatcher |
As shown in the above diagram, we change the deployment slightly & register GhostDeployers for all artifact types. The GhostDeployer instance will maintain references to the actual Axis2 deployers. Let us look at the sequence of events.
1. When a new artifact is deployed, either by uploading using the Management console UI, or copying the artifact to the Axis2 repo, the GhostDeployer will call the real deployer, which will create the real Axis2 Service
2. The GhostDeployer will then pick up that Axis2 service, and create a simple metadata file which has some basic information needed to create a Ghost Axis2Service. This will mainly contain the service name & operations; information needed for dispatching requests.
3. On subsequent deployments of the service which was first created in step 2, the GhostDeployer will pick it up from the metadata file created in step 2 above.
4. Next the GhostDeployer will create a Ghost Axis2 Service which will contain a special parameter (Ghost Parameter) which will identify the service as a Ghost service, and register that service into the AxisConfig
5. Next, say a request comes in for the said service, which at the moment is deployed in Ghost form.
6. There is a special Axis2 dispatching handler called the GhostDispatcher, which is registered after all standard Axis2 dispatchers. It will check the Ghost Parameter added to the service in step 4 above to check whether the Actual service deployment needs to take place, if the service in concern is a Ghost service.
7. If the service is determined to be a Ghost service in step 6 above, the GhostDispatcher will call the GhostDeployer to carry out the actual service deployment. At that point, the GhostDeployer will call the actual deployer, which will load the rest of the metadata & information required by the actual service. This will load all metadata from the registry, load service policies & so on.
In addition, if certain services are not used for some time, they will be undeployed & reloaded in Ghost form. This will ensure that rarely used services, or services that don't get called too often, do not unnecessarily use up resources.
5.0 Performance Comparison
We carried out performance comparisons & plotted the graph of "Initial Response Time" vs. "Number of Artifacts". The initial response time is the response time seen by a client when a tenant has not been loaded. The number of artifacts is the number of deployment artifacts deployed under that tenant.
We sent requests targeted at a particular operation in a particular service. We recorded the initial response times while linearly increasing the number of deployment artifacts up to 300 in that tenant. The resulting graph is shown below.
 |
| Figure 3: Performance comparison - with & without Ghost Deployment |
As can be seen, without Ghost Deployment, the initial response time linearly increases. Eventually, clients sending the very first requests will start timing out. With Ghost Deployment, we ensure that the first response time is linear, hence this will improve the client experience on first request, and will also ensure that the first requests don't timeout.
Ghost Deployment is one of the major enhancements we introduced in the recent StratosLive upgrades to enable our tenants to experience better performance.
6.0 Conclusion
Lazy loading is a core requirement in Cloud deployments. In this article we looked at some of the core concepts behind lazy loading. We saw how Ghost Deployment yields drastic performance improvements in a multi-tenanted PaaS environment such as StratosLive. Future enhancements in this area in Stratos will include lazy loading of webapps using a GhostWebappDeployer Tomcat valve.
