Dancer is a new Perl web framework that I’ve been playing with since April. I finally got some time to build a small application and take it live. I chose Amazon’s EC2 for deployment because in addition to Dancer, that’s another area which I had been wanting to explore and with their (then) recently introduced free usage tier, there wasn’t much to lose. Here are some details on how the app was built and deployed:
Dancer:
stupidtwitterstats.com pulls out some interesting stats about people you follow on twitter. I use Net::Twitter::Lite to talk to Twitter. I wrote a small class to analyze the data I get from Twitter and that keeps my route clean. I use Template Toolkit for, well, templating. There is a ‘lite’ version of Template Toolkit which comes as default with Dancer, but since I’ve been a TT user (dare I add a power user) for a while now, I went with the real thing.
I initially ran my app using Perl as:
perl bin/app.pl
I tried enabling “auto reloading” of my module so that any changes to it are immediately availble without restarting the app but for some reasons I couldn’t get it to work consistently on MacOS. A quick note to the Dancer mailing list revealed an alternate solution – using Plack with the ‘shotgun’ loader. The latter reloads your entire app for each request – a bit like CGI. If you are using modules that tend to have a long start-up time (like Moose), you can also tell plack to not load them every time:
plackup -L Shotgun -p 3000 bin/app.pl
To prevent certain modules from being reloaded:
plackup -MMoose -MDBIx::Class -L Shotgun bin/app.pl
This post from the 2009 plack advent calendars has more details.
Deploying on EC2:
The biggest problem I had starting with EC2 was documentation. Amazon overwhelms you with a lot of TLAs and their circuitous documentation makes going in circles seem like walking in a straight line (as the diameter of the circle tends to infinity this is indeed how it will feel, but I digress). That said, I came across their getting started guide which, alongwith the new web-based management console, made things a breeze.
The next big hurdle was picking the right distro of Linux to deploy on. I have more experience running Debian/Ubuntu in production than any other distro. Canonical’s 10.04 LTS Server was my first choice. While setting it up was a breeze, logging into it for the first time informed me that I had some pending updates. Installing those updates led me to a point in Grub configuration where the machine just froze. I didn’t take things any further.
To keep things simple I decided to go with the default Amazon 64-bit Linux instance. Now I would’ve loved to get hold of their custom Linux build just to replicate the production environment on my machine but Amazon doesn’t give it out. It looks like it is a Fedora derivative (there are fingerprints all over the place – e.g. in the welcome page you get when you install niginx) so one could run Fedora and get quite close to the Amazon provided Linux instance.
The Amazon Linux image comes with Perl 5.10.1. My first step was to install the ‘Development Tools’ bundle so that I could build things from source if needed.
sudo yum groupinstall 'Development Tools'
I then installed CPAN Minus (App::cpanminus), which is my preferred tool for installing things off CPAN.
sudo cpan App::cpanminus
I then used it to get Dancer:
sudo cpanm dancer
followed by other CPAN dependencies my app had.
At this stage I opened port 80 through the AWS EC2 console and tested my app to make sure it was running fine and was accessible over the internet using the temporary Amazon supplied domain name. I then got an Elastic IP and tied it to my running machine instance. I also went to my domain registrar (Dreamhost) and pointed my domain’s A record to this IP.
My next step was to install the Starman web-server under which my application is deployed.
sudo cpanm starman
I ran my app again – this time under Starman and checked it over the internet to make sure that everything was fine so far:
sudo /usr/local/bin/plackup -s Starman -p 80 -E deployment --workers=10 \
-a /home/apps/TwitterToys/bin/app.pl
Satisfied, I moved on to the next big step – installing and configuring nginx. While ideally I would’ve loved to install the latest 0.8.x branch of nginx, it wasn’t available out of the box on Amazon’s Linux image. Indeed, even the most recent Linux distros (Ubuntu 10.04 Server or Debian Lenny/Squeeze) seem to give it a miss. While nginx compiles from source on most distros without problems, keeping it updated, patched and running can be daunting. So I settled for the default 0.7.67 install via yum:
sudo yum install nginx
I use nginx to do all the HTTP related stuff like gzipping content, serving static files, adding the right expires header and so on. It also acts as a caching proxy in my setup. Dancer running under Starman/Plack does everything else.
The following lines setup gzip response compression and a caching zone:
http {
..
..
..
gzip on;
gzip_min_length 1024;
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=twitter:8m
max_size=64m inactive=60m;
proxy_temp_path /tmp;
proxy_cache_key "$scheme://$host$request_uri";
proxy_cache_valid 200 60m;
..
..
..
I then setup the server to proxy the requests to my running Dancer instance:
server {
server_name stupidtwitterstats.com;
listen 80;
location / {
proxy_cache twitter;
#bypass cache for this path so that I can check my API usage
set $do_not_cache 0;
if ( $request_uri ~ "^/xyz$" ) {
set $do_not_cache 1;
}
proxy_no_cache $do_not_cache;
proxy_pass http://127.0.0.1:5001;
proxy_redirect http://127.0.0.1:5001/ http://$host/;
expires 1h;
}
I also setup another location block within the same server block to let nginx serve all the images directly:
location /images/ {
alias /home/apps/TwitterToys/public/images/;
expires 30d;
access_log off;
}
I fired up Starman again – this time on port 5001 and bound to 127.0.0.1 and tested nginx from the internet to make sure that everything was working fine. I did run into a problem with serving static content. A look at the error log (/var/log/nginx/error.log) showed that nginx worker process was running into a permission issue reading the files:
2011/01/02 07:01:39 [error] 3781#0: *1 open()
"/home/apps/TwitterToys/public/images/logo.png" failed
(13: Permission denied), client: 122.167.81.253, server: _,
request: "GET /images/logo.png?x=1 HTTP/1.1", host:
I gave ‘others’ read and execute permissions on the /home/apps/ folder to make sure that nginx worker process could get in and read the files and the ’13: Permission denied’ errors went away.
sudo chmod -R o+rx /home/apps
This brought me to the last big task – configuring my Dancer application to run as a daemon so that it runs in the background and comes up when the OS boots the next time. I chose Daemontools for this. Unfortunately Daemontools were not available on Amazon’s Linux image via yum (they are available on Ubuntu 10.04 via the default repositories using apt-get), so I decided to roll pull the source and build. Here, I ran into another wall – the compilation would stop with some vague reference to errno.h. After some tense moments of frantically searching the internet, I found that I had to modify error.h in the src/ directory of the daemontools source distribution to something like this:
/* extern int errno; */
#include <errno.h>
The compilation and subsequent installation went fine. I restarted my EC2 instance to make sure that ‘svscan’ came up after the reboot. Things were much simpler this point on. All I had to do was create a folder for my daemon (I called it TwitterToys) under /service and place a shell script called ‘run’ with execute bit set:
#!/bin/sh
export PERL5LIB='/home/apps/TwitterToys/lib'
exec 2>&1 \
/usr/local/bin/plackup -s Starman -l 127.0.0.1:5001 -E deployment \
--workers=10 -a /home/apps/TwitterToys/bin/app.pl
Within moments my new and shiny daemon came up. I did another reboot to make sure that it indeed does come up as expected. By this time my domain changes had replicated to my ISP and pointing my browser http://stupidtwitterstats.com brought up my site.
My last steps before announcing it to the world were:
1. to add a CNAME record for ‘www’ pointing to stupidtwitterstats.com so that people who prefix URLs with www do make it to my site.
2. to add the following server block to my nginx configuration so that www.stupidtwitterstats.com redirects to stupidtwtterstats.com (via)
server {
server_name www.stupidtwitterstats.com;
rewrite ^(.*) http://stupidtwitterstats.com$1 permanent;
}
There you have it! A site running on Perl and Dancer from start to finish.