Sunday, February 17, 2013

How to Run Jaggery.js App in Heroku Cloud

Jaggery.js is a server  side javascript framework for writing front-end ,communication ,server side logic and persistence in pure javascript.I felt that there
is no public cloud hosting service available for hosting jaggery apps for free(Other than stratoslive.wso2.com).So i have attempted to run a jaggery server on famous platform as a service Heroku.In this post i am going to describe how to run a jaggery app on Heroku.

Pre-request for running apps in Heroku cloud.

1.The app should be written in supported languages(Ruby, Node.js, Clojure, Java, Python, and Scala).
2.All the java apps should be pushed to git as Maven project  
3.Heroku only allow apps to open port that is given by a system variable PORT.
4.App should bind to the port within 60s.
5.App should not store anything to file for long term persistence.
6.Data base details will be given by a system variable DATABASE_URL.
etc...


So the challenge is to make the Jaggery server to obey above rules.

Jaggery server is actually a WSO2 carbon server.Which is actually a java app.so it should run in Heroku.To run a WSO2 Carbon app we have to set the JAVA_HOME.
But in Heroku during application launch java will be added to system path but JAVA_HOME will not be set .I have added following to WSO2 Carbon startup script to set the JAVA_HOME .

export JAVA_HOME=$(which java |xargs  readlink -f  | sed "s:bin/java::");
echo "Setting JAVA_HOME to $JAVA_HOME "

 Next thing is, i have to create a Maven project that should build a Jaggery server and copy Jaggery app into deployment folder.There are no Jaggery server dependency that i can get it as a zip file.But fortunately all the required dependencies for building a Jaggery server from a Carbon server are available.
I was able to create a Maven project that can build a jaggery server form carbon server.
The usual way of running java apps in Heroku, is specifying the launch command in Procfile.Procfile for the above Maven project looks like following

web: sh server/heroku-run/jaggery-1.0-SNAPSHOT/bin/server.sh

This means the Maven project has to unzip the built Jaggery server into heroku-run folder of server Maven module.

Now the a Maven project is ready that can build a Jaggery server and has a Procfile to launch the Jaggery server.But in order to start the server  I have to make sure that the Jaggery server opens only one port.
Jaggery server will open altogether 6 ports by default.Following are the ports and how i disable those ports.

1.http/https:These ports are opened by tomcat connectors.I need to make sure only one port to be opened.so i disabled the https connector in JAGGERY_HOME/carbon/repository/conf/tomcat/catalina-server.xml

2.jmx(2 ports):There is a configuration to not to start the jmx server.The file name is CARBON_HOME/repository/conf/etc/jmx.xml

3.Zookeeper server(opens one port):To disable Zookeeper  there should be a property(start_zk_server=false) in /carbon/repository/conf/etc/zoo.cfg

4.jmx remote (opened by tomcat):To disable this -Dcom.sun.management.jmxremote
should not be set.

Finally the port number in tomcat connector has to be set to the port number in PORT system variable provided by Heroku.All the configuration files in jaggery server can accept java system variable in the form of ${SYSTEM_VARIABLE}.To set that system variable we have to pass -DSYSTEM_VARIABLLE=value in start up script.In this way the value in PORT system variable can be passed to tomcat connector configuration.

The same way the information in DATABASE_URL(database url,username,password and database name) system variable should be passed to JAGGERY_HOME/carbon/repository/conf/datasources/master-datasources.xml.

The Maven project can be found in Github here
https://github.com/ajanthan/heroku-jaggeryjs-app-runner

Follows are steps to run your Jaggary app in Heroku.

0.Register in https://heroku.com and install Heroku tool
1.git clone http://github.com/ajanthan/heroku-jaggeryjs-app-runner.git
2.heroku create
3.heroku addons:add heroku-postgresql:dev
4.heroku config
5.heroku pg:promote HEROKU_POSTGRESQL_RED_URL
6.Insall postgresql (https://devcenter.heroku.com/articles/heroku-postgresql#local-setup)
apt-get install postgresql
7.heroku pg:psql
\i /home/heroku-jaggeryjs-app-runner/dbscript/postgresql.sql
8.add your Jaggery app to apps/ROOT directory
9.git add .
10.git commit -m "adding initial project"
11.git push  heroku master
12.heroku logs -t
Access my app here http://ajanthan.herokuapp.com/


Friday, February 15, 2013

WSO2 App Factory Preview

Yesterday WSO2 announced the public preview of WSO2 App Factory  beta on their annual conference WSO2Con 2013.You can access App Factory  here


In order to login to the App Factory you have to be a registered user in WSO2 Oxygen Tank.If you are still not a registered user,click get started button it will direct you to


After success full login you can create new application.

Each Application creation will create a git repository for you with skeleton Maven project.Edit the Maven project to include your changes and push to git repository.App Factory will build your project and deploy to development cloud using Jenkins.
Press goto buid server button in build tab to go to App FactoryJenkins CI.


Use same username and password to login into Jenkins CI.


To manually tricker a build use build button in builds page.


Observe notification panel to know the build status if you do not want to go to Jenkins.

Go to repository tab to create new versions of application.



Manage your application versions through various stage in governance tab.


Download logs of your applications in various stages(dev,test,staging and prod) in same tab.


Invite new users to other registered users through user administration tab


Manage your issues in Redmine.


Create new databases and associate those with data sources in configure resource tab.


Subscribe to APIs or create new APIs using WSO2 API Manager and use those in your application.


Create new properties as a resource in WSO2 Governance registry and use that in your application.

Note that in all case you will get complete isolation between resources across different stages.
Play with preview and report any issues or new features in https://wso2.org/jira/browse/APPFAC

Sunday, December 23, 2012

WSO2 App Factory v1.0.0 M8

Recently WSO2 App Factory team released their 8th mile stone[Release mail]. This may be the pre alpha release.You can down load preconfigured VM from here.
Lets look at the new features of this release briefly.

1. Now users can create applications either with svn or git repository:In the VM a Apache server based svn deployment and Gitblit based git deployments are available .User can select preferred repository type while creating application.The Jenkins job also will be configured automatically according to users' selected repository type.

2.Integration of Human Task:While user submit registration from we can configure App Factory to fire a human task to get approval from an admin.This feature is valuable in the environment where strict approval process is needed. Refer this blog post for further information. 

3.New Gadgets:There are two new gadgets to view application event stream and search applications.

4.Log download:There is a way to download logs from build ui. App Factory uses WSO2 Stratos logging frame work to collect and display logs.With this release log download links is provided.

5.Improved Build and Deploy model:Ability to tag a build is provided and the default artefact storage is moved into Jenkins side.There is a REST api is exposed from the Jenkis to deploy a built artifact to a server.All the Jenkins plugins are merged into one "appfactory-plugin" and fixed some problem while deploying Jenkins in master slave mode.

Ok ,now the time to look at some new features that may come with next(probably Alpha) release.

 *Integration with GitHub :There will be Github based repository provider.
 *Artifact deployment with WSO2 Stratos 2 : Final Artifact deployment model will be with WSO2 Appserver Cartridge cluster.WSO2 recently released Stratos Alpha,this is a completely different cloud Linux lxc container based architecture.
*And more..
That's all  for now ,Wait for next release...


Saturday, September 8, 2012

Creating SVN Repository Remotely

Linux distribution of Apache subversion provides svnadmin to create a svn repository.But this is not capable of creating repository remotely.By issuing svnadmin command we can only create a repository within local file system.
Here i am going to describe couple of ways by which we can create svn repositories remotely.Here all my ways achieve above mentioned objective through  java.Here is the list of method that i found.

1.Using ssh to execute svnadmin command remotely.
    a)Using apache sshd library
    b)Using JSch library
2.Using SCM -Manager
3.Using PHP+java Client

Using ssh to execute svnadmin command remotely

In this approach the repository is created by executing svnadmin or relevant command in remote host using ssh.Here there are two ssh opensource libraries available for java.

a) Using apache sshd library

apache sshd is a pure java implementation of ssh available under apache 2.0 license.This is a sub-project of apache Mina.Here is a example code for executing svnadmin command on remote host as a superuser.

To use apache sshd library you have to add following maven dependency. 


b)Using JSch library

This is another popular java implementation of ssh.There are  good examples also available for this libarary.Here is how to execute svnadmin command using JSch library.
Following is the maven dependency which can be used to create maven project.

Using SCM -Manager

Next approach is using a source code management system.SCM Manager is an open source source code management system which is implemented in java.This is
a server which provide user interface to create repository and manage them with users.Currently it support svn ,git and mercurial.In addition to user interface, it expose REST API to manage repositories.Here i am going to use the REST API to create a repository.Before staring writing the REST Client Download the SCM Manager from here and start the server by executing the script which can be found in bin folder of the distribution.
To write the REST client there so many ways.
  1.Using HTTP client
  2.Apache Wink client
  3.Jersey Client
I am using http client wich is very simple to use.All the REST calls are protected by BASIC Auth in SCM  Manager.We have to set the credential in http client.
Here i used Axiom for generating XML payload.

Using PHP+java Client

In this approach a php page that is hosted in apache 2 server in remote host will create a svn repository using svnadmin command and there will be a java client that is used to call the php.The php page is secured using Basic Auth.Here is the very simple php code to create svn repository.

Here is the java client that is used to call the php page.
 
This is how we can create svn repositories remotely.This will be helpful when ever there is a requirement of automatic repository provisioning.
  

Thursday, June 28, 2012

Getting Started with WSO2 Appfactory VMware Image

In this post i am going to describe how to getting started with WSO2 Appfactory VMware image first mile stone  release.

WSO2 Appfactory

WSO2 Appfactory is a new solution that provides a convenient environment to manage Applications between different stages of development.For example each application developed in enterprise goes through so many stages (such as dev and QA)before going to live.While it is moving between stage it goes through so many reviews(such as security review and policy review) by different users and authorities.WSO2 Appfactory provides a convenient and easy to use environment to manage application between different stages.For more information on WSO2 Appfactory refer following blog.

WSO2 Appfactory VMware Image(M1 release)

WSO2 Appfactory team recently released their first appfactory milestone release as a VMware image.The image has following servers
1.WSO2 Appfactory
2.WSO2 Stratos Manager
3.WSO2 Application server (for each stage(dev and live))
4.SCM manager
All the servers are connected to a single LDAP and a remote  governance registry
Following diagram shows the deployment pattern that the image uses.

Now lets play with this image.

Prerequisites

1)VMware player should be installed in 64 bit machine with visualization enabled.
2)The host machine should have at least 4 GB memory and connected to Internet.

Getting Started with vm

1)Download the image from http://builder1.us1.wso2.org/~appfactory/releases/wso2appfactory-image-m1.zip
2)Unzip the downloaded file.
3)Start the VMware player.
4)Open the image as following





5)Edit the image setting as follow

Increase the memory to at least 2 GB(Recommended ~8GB)  and enable bridge network setting.
6)Start the image



use following credentials to login to the image.
User-name: appfactory
Password: appfactory

Note:
1.Change the /etc/hosts entry of vm.svn.appfactory.wso2.com to
 0.0.0.0 vm.svn.appfactory.wso2.com
2.Change keystore location of scm manager.
Open the carbon-auth.xml that can be found in /home/appfactory/final-zip/scm-server/repository/config.
Change the keystore location to /home/appfactory/final-zip/appfactory/wso2appfactory-1.0.0-SNAPSHOT/repository/resources/security/wso2carbon.jks

7)Start all the servers as back-end process.
Start Appfactory
cd /home/appfactory/final-zip/appfactory/wso2appfactory-1.0.0-SNAPSHOT/bin
nohup ./wso2server.sh &
Start Stratos Manager
cd  /home/appfactory/final-zip/cloud-manager/wso2stratos-manager-2.0.0-SNAPSHOT/bin
nohup ./wso2server.sh &
Start Dev cloud Application server
cd /home/appfactory/final-zip/dev-cloud/wso2as-5.0.0-SNAPSHOT/bin
nohup ./wso2server.sh &
Start Live cloud Application server
cd /home/appfactory/final-zip/live-cloud/wso2as-5.0.0-SNAPSHOT/bin
nohup ./wso2server.sh &
Start SCM Manager
cd /home/appfactory/final-zip/scm-server/bin
./scm-server start
8) Get IP address of the VM and enter following to /etc/hosts of the host.
In VM issue sudo ifconfig and get the IP.

172.16.9.130 vm.appfactory.wso2.com
172.16.9.130 vm.cloudmanager.appfactory.wso2.com
172.16.9.130 vm.dev-cloud.appserver.wso2.com
172.16.9.130 vm.live-cloud.appserver.wso2.com
172.16.9.130 vm.svn.appfactory.wso2.com

Add above entries to /etc/hosts of host machine.Here change the IP address as you got in previous step.
Wait for some time to start all the servers success fully then you can access the servers using following link in host machine browser.
Access Appfactory https://vm.appfactory.wso2.com:9443/appmgt/
Access Dev cloud Application server https://vm.dev-cloud.appserver.wso2.com:9445/
Access Live cloud Application server https://vm.live-cloud.appserver.wso2.com:9446/

Getting started with Appfactory

Now it is the time to create a application and deploy it to cloud.
1.Register a new user
 1.Access https://vm.appfactory.wso2.com:9443/appmgt/
 2.Register a user by clicking sign up button at left top of the sign in page as follow.
Fill the sign up form


Check your mail  and click the link sent by mail to validate the email address.


2.Create an App
 Login as created user
By clicking add apps create new app
After successful creation of application it will display the created SVN url.
 When ever user login he always see the list of applications with short details
When the user click on a app depending on the role of the user he will see different dashboards.
All the member can view the dev-view as follow.
The users in admin role can view admin view.The admin user can invite already registered users to application.

The user in devOps role can view devOps view.Here he can deploy the application to stages(dev or live).


3.Invite a user as a developer
Register a new user.
Invite him as a developer to created project.

4.Invite a user as devOps
Same way invite another user to devOps role.
5.Commit a CAR project into svn as developer
Check out the svn repository  displayed on dev-view as developer.
Add a car project  to the checkout directory(use following car project)
Commit the project


6.Login as devOps and you can
              -deploy to dev
              -deploy to live in dev-ops view.

7.Then finally you can check the relevant application server in cloud for the deployed application.
Login to https://vm.appfactory.wso2.com:9443/appmgt/
as the user who created application using user name as {user-name}@{application-id} and check the deployed car applications.


Thats all for milestone one release.I hope wso2 appfactory will come with more functionality with less distribution size in future releases.

Sunday, May 8, 2011

GSOC 2011:Weekly Report(May-01 to May-08)

Last week as i planned before i spent on time designing UI mock-ups and database tables.The good thing is that the designing process was a collaborative work of my both mentors and myself.They have given constructive feedback on the design and finally helped me to come up with the final design.I have updated my design details on wiki page.While designing i am able to realize some insight into theories of database design as well as UI design.

UI Design
I identified that there are four sets of UI should be designed.

Main UI:
This is the place where the list of available apps are displayed on the side bar and the user is allowed to launch the app within a HTML iframe  by clicking on the icon of the app.This is an usual UI every SMART container may contain.Here my job was to identify the best place to put this UI in the OpenMRS.The place should be appropriate to display patient level health data because every SMART UI app is a patient level application.With the help of my mentor i decided to put this UI at patient dashboard where the access to the dashboard is allowed only after selecting a patient.There the patient object can be received which is necessary to create a record in SMART container helper methods.
Main UI



Manage Apps at user level UI:
In medical world there are range of peoples who need different types of applications.For example the doctor uses application that related to diagnosis process but this type of applications are not relevant to a patient.So it is better to provide a UI for user to customize the available application at main UI.It is usual to put all the user level setting under My Profile in OpenMRS.According to the normal convention i decided to put the UI for managing SMART apps at user level in a vertical tab under the My Profile page.
Manage Apps at user level

manage Apps at seystem level(system wide settings) UI:
Normally it is a convention to allow only administrator to install new modules in OpenMRS.This rule is appropriate for SMART apps also.As SMART mentor Josh said i decided to put the add or remove SMART app UI at admin page.
Manage app (add/remove)

UI for installing the Apps:
When the user clicks on the add button on the manage app at admin page, a pop up widow will be displayed.This window will allow user to upload a SMART app manifest or to provide a link to the manifest file.
Add App UI
Database Design
Here the primary data are information about users and the SMART apps.OpenMRS alredy has a user table.The new table should be created for modeling SMART apps.SMART app uses a manifest file to save the information about itself.It is straight forward that there are one to one relationship between app database table and the manifest file.In other words database table is used to save informations in the manifest file.I directly modeled database table using manifest file as a guide and created obvious relationships.
Final Database Table(the user table should contains more field,it can be found here)