Saturday, September 24, 2011

How to setup WSO2 Elastic Load Balancer

In "WSO2 Load Balancer - how it works" we discussed the internal workings of the WSO2 Elastic Load Balancer. In this article, we will discuss in detail how to configure this load balancer. Even though not necessary, understanding how the WSO2 load balancer works will enable the reader to understand the rationale behind the way certain things are configured.

1.0 Example Setup
Let us take the example of configuring the following setup shown in the diagram below. We will look at how to configure the elastic load balancer as well as the worker node clusters in this setup. By understanding this setup, you should be able to setup WSO2 ELB fronted clusters with any of the WSO2 Carbon based products or WSO2 Stratos Services.

In the above setup, we will have a LB cluster setup in primary-secondary configuration. This LB cluster will be fronting 2 clusters; App Server cluster & Data Services Server cluster.

In this exercise, we will assume the following ports.

Load Balancer
HTTP port: 80
HTTPS port: 443

App Server
HTTP port: 9762 --> proxied via port 80
HTTPS port: 9443 --> proxied via port 443

Data Services Server
HTTP port: 9762 --> proxied via port 80
HTTPS port: 9443 --> proxied via port 443

We will also assume well-known address based membership discovery (wka) to keep the article simple. The same configuration will work with multicast based membership discovery too, with the membershipScheme set to multicast in the axis2.xml file.

In the following sections, we will look at how to configure the worker node clusters & the ELB.

2.0 Configuring the App Server Cluster
You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the App Server cluster. These files are available in the CARBON_HOME/repository/conf directory.

2.1 mgt-transports.xml
Uncomment the HTTP proxy port in this file. The HTTP proxy port has to be set to port 80. In this case, the LB is the HTTP proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTP port is 80.
 
<transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">80</parameter>
</transport >
Uncomment the HTTPS proxy port in this file. The HTTPS proxy port has to be set to port 443. In this case, the LB is the HTTPS proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTPS port is 443.
 
 <transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">443</parameter>
 </transport >


2.2 carbon.xml
In this file, we have to specify the Host of the App Server. In this example, it is appserver.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.
<HostName>appserver.stratoslive.wso2.com</HostName>


2.3 axis2.xml
In the axis2.xml file, you have to enable the clustering. This is used for mainly membership discovery. As explained in "WSO2 Load Balancer - how it works", the LB discovers worker nodes using the underlying Axis2 clustering mechanism.
      
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">appserver.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.as.domain</parameter>
    <members>
        <member>
            <hostName>appserver.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>
Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer. The localMemberHost can be an IP address which has private visibility. You can keep the localMemberHost entry empty, and if it is empty, one of the available interfaces of the machine the App Server node is running on will be picked. The important thing to note here is that the clustering domain has been set to wso2.as.domain.


3.0 Configuring the Data Services Server Cluster
You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the Data Services Server cluster, and the configuration is almost the same as that of the App Server cluster.

3.1 mgt-transports.xml
The configuration is identical to what we saw in section 2.1 in the App Server cluster configuration section.

3.2 carbon.xml
In this file, we have to specify the Host of the Data Services Server. In this example, it is data.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.
<HostName>data.stratoslive.wso2.com</HostName>


3.3 axis2.xml
Most of the configuration is identical to what we've seen in section 2.3. The main difference will be that the clustering domain has to be set to wso2.ds.domain. The relevant configuration is shown below.
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">data.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.ds.domain</parameter>
    <members>
        <member>
            <hostName>data.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>
Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer.

4.0 Configuring the Elastic Load Balancer Cluster


axis2.xml
In this file, locate the transportReceiver entries for the HTTP & HTTPS ports and change them to port 80 & 443 respectively. The relevant XML segments are shown below.
<transportReceiver name="http" class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener">
    <parameter name="port" locked="false">80</parameter>
</transportReceiver>

<transportReceiver name="https" class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener">
    <parameter name="port" locked="false">443</parameter>
</transportReceiver>



loadbalancer.xml
Generally, you will only need to change the CARBON_HOME/conf/loadbalancer.xml file. All other relevant files have the proper defaults, and will have to be changed only in the case of advanced configurations.

The following segment in the loadbalancer.xml file shows how we map the HTTP host to the clustering domain.
<loadBalancerConfig xmlns="http://ws.apache.org/ns/synapse">
    <services>
        <service>
            <hosts>
                <host>appserver.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.as.domain</domain>
        </service>
        <service>
            <hosts>
                <host>data.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.ds.domain</domain>
        </service>
    <services>
</loadBalancerConfig>


4.1 Disabling Autoscaling
At the time of writing, the WSO2 Elastic Load Balancer only supports autoscaling on Amazon EC2 & compatible IaaSs since it is based on the EC2 API. So, you will see error messages if you try to run this setup on a local network, for example. So, in such cases, you will have to disable autoscaling. You have to do the following to disable autoscaling;

  1. Delete $CARBON_HOME/repository/deployment/server/synapse-configs/tasks/autoscaler.xml
  2. Remove the autoscaleIn & autoscaleOut mediators from $CARBON_HOME/repository/deployment/server/synapse-configs/sequences/main.xml
5.0 Testing the Setup
Now we have completed setting up the system, so it is just a matter of starting up the nodes. If you have autoscaling setup, you will simply have to start only the Elastic Load Balancer. This LB will take care of starting up all other nodes in the system. If everything is properly setup, you should see messages on the LB log file saying members from wso2.as.domain & wso2.ds.domain joined the cluster.

To make sure that everything is working properly, in your client machine, map the appserver.stratoslive.wso2.com & data.stratoslive.wso2.com names to the relevant IP address of the primary Elastic load balancer & go to the following URLs using your Web browser.


If you have properly setup your cluster, you should see the login screens of the relevant WSO2 product.

Post a Comment