Monday, August 13, 2012

MongoDB and Play2 at ease: using Play-Salat plugin and Embed Mongo plugin

A while ago I've blogged about MongoDB with Play2 using Salat, here.

This post was describing how to integrate Salat easily with Play2 and gave some advice on actions to care of.

Preface

Play 2 gained in popularity and an amazing plugin as emerge for this purpose: play2-salat.

This plugin offers a lot of configuration to hit running instances including replicasets! It integrates very well by applying the advice I talked about in my post, but not only. It defines binders to enable us using Casbah stuffs in our routing and action definition (ObjectId, and so on).

This post is not dedicated to explain how to use it, I'd recommend you to browse the project page (play2-salat), plus the wiki that points to relevant URLs.

Goal

This post is dedicated to developers teams that follows (or not...) the convention of Continuous Delivery, especially the Single Command Environment pattern. That is, the environment must be set up in one single command... in Play2 => play run OR sbt run

Context

Create an application that uses MongoDB as (one of its) persistence backend service, use play2-sala to have access to the `ORM` for our object and easy collections connections.
When runnning in production, of course, a MongoDB instance runs somewhere that can be configured (or a replicatset).

But in Dev?

Embed Instance

When another developer is cloning the related repo, knowing that it's a play application, he's best will would be to enter the directory and launch the application. > BANG <
No running instance...

So I created a Play2 plugin that uses this amazing work which retrieves a mongodb installer, installs it and enable us to launch/stop it... Keep in mind that MongoDB is not JVM based!

Adding this plugin to the application, setup the dev configuration to starts an embed MongoDB and Play2-Salat to target it, will gives the satisfaction to our developer... Moreover if he is a Designer (the only kind of guy that add values to any app ^^) who don't care about MongoDB, at all!

How To

Add the plugin dependencies (used in PlayProject):

     //MY OWN REPO where is deployed the following plugin
     val skalaCloudbeesSnapshots = "Ska La SNAPSHOTS" at "https://repository-andy-petrella.forge.cloudbees.com/snapshot/"

    //THE NEW PLUGIN => EMBED
    lazy val embedMongoPlayPlugin    = "be.nextlab" %% "play-plugins-embed-mongodb" % "0.0.1-SNAPSHOT" changing()

    //THE WORTH ONE
    lazy val salatPlayPlugin         = "se.radley" %% "play-plugins-salat" % "1.0.8"

    //DECLARE the deps
    val appDependencies = Seq(
        embedMongoPlayPlugin,
        salatPlayPlugin
    )

A bit of configuration (application-dev.conf)
embed.mongodb.start=true
embed.mongodb.version=V2_1_1
embed.mongodb.port=27017

mongodb.default.db=meinGot
mongodb.default.host=localhost
mongodb.default.port=27017

And the most only thing that requires a bit of explanation (in conf/play.plugins)
600:se.radley.plugin.salat.SalatPlugin
550:be.nextlab.play.mongodb.EmbedMongoDBPlugin
See? Yes, the Play2-Salat plugin MUST be started AFTER the embed plugin... of course (what an explanation huh).

Code

The one-single-file-of-33-lines plugin can be forked here.



That's All, Folks!

Friday, July 20, 2012

Gatling-Tool in SBT or Play : Sample Projects

Content

This post is a direct follow up of this one where I introduced a bit what I did in order to integrate Gatling in SBT and in Play2.
Where this post was more about bits and bytes necessary to accomplish the task, this one will talk about how to use this mess.

Gatling for SBT

Now, that we've a dedicated SBT plugin in hand we can create a sample project that uses it (I already created one here).
In this new project, we'll need to create a file plugins.sbt which will contains the reference to the gatling-sbt-plugin.
Actually, it's the classical way to add a plugin to a SBT project (and the easier and the "semanticest").
We're now prepared to configure our build by using the pieces provided by the plugin.
At the end, we'll be able to write a first test and launch it.

Project Build

First of all, we must create a directory for your project with this basic structure:

Looking at the structure, we can see a Build.scala that will define our project build, a build.properties that defines the SBT version with a single line : sbt.version=0.11.3
And the latest file in the project folder is plugins.sbt which... declares the plugin.
Let's put the Build.scala aside for now, and have a look at the content of the latter. 
Self descriptive isn't it? Yes, we've just told that SBT has to use our gatling plugin... that's all? Not yet.
Actually this line will provide us everything that have been declared in the plugin, such as the Gatling configuration keys, the command, and the basic SBT settings. 
Now, we're gonna us them within our Build.scala. 

Before going in specific details, note that we had to add my own repo (hosted at Cloudbees) in order to fetch the project... but you could also use the URI fetch provided by SBT... 
What are really important to point out is the allSettings declaration and the import of the GatlingPlugin. 
At first, we reuse the default one by using Project.defaultSettings which is being appended the gatling settings, using Gatling.gatlingSettings... that object comes from our plugin! 
This will add all relevant keys for the "gatling-test" Configuration with default values. 
At this stage we've almost finish our build definition... All that remain (and shouldn't... have to figure out any help is welcomed?) is to add two things:
  • Declare the test framework to have access to the gatling classes and the custom Simulation :
    "be.nextlab" %% "gatling-sbt-test-framework" % "0.0.1-SNAPSHOT" % "gatling-test"
  • Declare the command :
    commands ++= Seq(gatlingTakatak)
After having used everything in your project (don't forget to add the GatlingTest configuration...) you're so close to write your first test.

Gatling Conf

This section will cover the other folder src wherein reside the classical main and test folders.
But there is an other one, gatling-test that is unknown at this stage.
In fact, this folder will be the root for our Gatling tests, holding the configuration file (I provided a basic one in the sample app) and the future simulations that you'll write.

The first is dropped in conf/galing.conf, where the latter is simply simulations.

Write your first test

At this stage, I'll make an assumption that you either know the Gatling api... Nah, the Gatling DSL is sufficiently self descriptive (otherwise check the wiki).

As you can see, it's very very easy (that's why I like Gatling-Tool).
All we had to do, since we're benchmarking google, is to implemnent the apply method of a simple scenario within a class (with must have an empty constructor... a limitation for now) that must extend our custom Simulation (yeah! I know ... I'll choose another name for it that's why GSimulation is needed).
Note the dummy implementations of the interceptors whose aren't needed here.
This Simulation must be located somewhere under the simulations folder.

Now, we can launch it and see where are stored the results.

That's enough boy...
Let it shot the server and generate the result.
 ...
...
Done? ok.

Now, that all bullets have been shot, you can go in this folder gatling-project/target/gatling-test/result.
See? True! Every run will create a specific folder under this one, named run-${scenario-name}-time.
Make some jump in it and locate the index.html file, open it in your browser and see the magic happened. 
NB: check this page for information about the reports. 
So far, so good. But we're writing web apps and we want to stress them... we all know that the google server won't crash (that easy). 
Let's move to Play2 (or you can write your own app w/o Play2 using Spray or another Java framework maybe, then mimic the following).

Gatling for Play2

For this part, I also wrote an application that makes use of the "plugin" introduced in the previous post.
It's quite complicated, not yet polished and might be a good topic for a future post, because I tried to make an application based on Event Sourcing with some workflows and usages of pure functional structure like Lens, Writer, State, Monad and so forth.

But let's stick with the actual topic, if you fork the project and browse a bit its build configuration you'll see that it is near from the one we just talked about. But still that some particularities are presents...

Definition

First of all, since Play2 has its own convention for sources folders (app, test under the root) we'll redefine those folders for gatling as well.
In order to do that, we must adapt the following configuration keys:

Actually, yeah, we've just skipped the "src" folder. Anyway. 
So far, so good... 
I heard you... 
But, do you known that we're already done? 
Allez, let's write a test.

Write

If you started from scratch (without forking the sample app) you can now stress test your index page that will render the default welcome page defined in Play2.
See the below Gist for that, or its follower for a more complicated one using the app.


Look what I've done Dad! 
In the Simulations, all we had to do is to simply declare an available port on which the framework can start the server, and its needed FakeApplication to have access to the routing. 
Then when we had to build a request path, we've just had to do the same as in our templates, that is, use the routes object under the controller package. Amazing
Furthermore, we can also reuse our type checked Form to build our feeder. Waouw
But, at this stage, there still some boilerplate (parameters). Looser.

Launch

And now, it's time to stress test our server and see how it'd behave on production. (Ok, that's not 100% true, since Gatling is running on the same machine...)

For that, just do the following:

Wait. Again.

Now, go to you result directory (the path hasn't changed). Open in your browser and let the shine comes in.

Wrap Up

At this stage, with small and few injuries we can use Gatling-Tool without having to install or trick stuffs in order to have information about our web app.
This by using either a full SBT application or a Play2 based one.
I know that there still a lot to do, but at least the basic features are there, and I'll enjoy anyone forking it, scrapping it if necessary and let me know.

Last words

I hope you liked these posts, otherwise I'd like to apologize because I did all projects and the blogs on the train, half asleep every morning and evening, back and forth from my actual mission's workplace.

Now I'm gonna sleep in this train...
Oh no, I can't...
I've to write my book now.

Gatling made easy for SBT or/and Play2

Forewords

These last few weeks, I took some time to understand a level deeper how SBT works and what it can provide. Since this post is not related to this learning trip (which was along existing blogs and wikis), I'll jump directly to the idea this new understanding gave me.
A while ago I started (and paused) a work on Gatling-Tool to have it integrated with Play 2.0 (see these posts here and here), this work has been refactored to better integrate with SBT.

What we'll find in this post

In this post, I'm gonna give you all tools in order to stress test you Web based application, either built upon Play 2.0, either only using SBT. By directly starting writing your tests rather than having to configure stuffs or to start others... So this post is composed of two parts:
  • How I built these tools (can be skipped -- definitively)
  • How to use these tools (it's probably the useful part)

Gatling shot 3 aside projects... for goods

This part is related to what was necessary to have gatling-tool easily integrated with SBT, and after with Play2.

Gatling SBT Test Framework

This project (on github) has a sole and simple goal, to implement the Test Interface (see) used by SBT to integrate new Test Framework (like was done for ScalaTest, Specs, JUnit, and...). This project contains only 2 classes and one convenient trait.

GatlingBootstrap

This class is one of the helper that avoid boilerplates in the future, because it starts the GatlingConfiguration stuff needed by Gatling-Tool to execute. This class requires two things "the path to the gatling configuration file" and "the path to the folder where will be stored the stress results".

Simulation

This trait is another tool in hand that extends the com.excilys.ebi.gatling.core.Predef.Simulation provided by Gatling in order to add it two interceptors whose are pre and post. Those interceptors will be really useful in the future, in order to start/stop a server for instance (e.g. FakeServer in Play2).

GatlingTestFramework

This class is the real processor of the project. Actually, it's also the implementation of the interfaces declared by the SBT test framework which are Framework, Runner2 and some Fingerprints. Basically, what is done there are those tasks:
  • Create the gatling bootstrap based on sys properties
  • Declare fingerprint to discover Stress Test based on the parent class being be.nextlab.gatling.sbt.plugin.Simulation
  • Create the stress tests instances (only classes are handled for now) by reflection
  • Call the pre interceptor
  • Execute the stress test using the gatling api
  • Call the post interceptor
  • Generate the reports using the gatlin api

Gatling SBT Plugin

This project (here on github) aims to provide the basics to use the test framework easily within an SBT project
Its intent is to have testers able to write stress tests directly, just after having imported the module in SBT. 


It basically provides an SBT Configuration (gatling-test) that extends the SBT's Test one and one Command (takatak). The other goodies are settings which are common for all such stress tested projects using gatling. 
Those settings are essentially the conf file path, the result directories, the libraries (gatling ones principally) and so on. The last important thing that it does is to add the gatling test framework to the already existing list of test frameworks supported out of the box by SBT.

Gatling Play2 Plugin

This misnamed project (no more a Play2 plugin, neither a SBT one... but it's ok for now) brings only one little thing. It comes with a base implementation the custom Simulation declared in the Gatling Test Framework.

This base implementation is Play2 specific while creating a fake server and starting it in the pre interceptor, stoping it in the post one.

This will ease further tests in the Play2 environment.

Save Point

So far so good, if you read this you can now navigate the small projects that were necessary to do what we'll discuss in the next blog...

This one is already too long...