platform agnostic

I recently came across a tip on Hacker News which along with Firefox, MozRepl plugin and this snippet:

autocmd BufWriteCmd *.html,*.css,*.gtpl :call Refresh_firefox()
function! Refresh_firefox()
  if &modified
    write
    silent !echo  'vimYo = content.window.pageYOffset;
                 \ vimXo = content.window.pageXOffset;
                 \ BrowserReload();
                 \ content.window.scrollTo(vimXo,vimYo);
                 \ repl.quit();'  |
                 \ nc localhost 4242 2>&1 > /dev/null
  endif
endfunction

allows you to refresh a tab in Firefox the moment you save your edits in Vim. No longer do you need to switch to Firefox, hit refresh to see the edits and then come back to your work. This is especially useful when you are on a multi-monitor setup.

My primary browser these days is Google Chrome. I was wondering if such a thing would be possible with Chrome too. It so happens it is. If you start Chrome with –remote-shell-port=9222 switch (on my Mac I do it like this: ~/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome –remote-shell-port=9222), you can connect to it using a TCP socket over port 9222 and then issue it commands using Chrome Dev Tools Protocol.

I wrote a small Perl wrapper around the protocol and then wrote another simple script that simply refreshes the last open tab in your browser:

#refresh.pl

use strict;
use ChromeTool;

my $chrome = ChromeTool->new;
if($chrome) {
    my $tabs = $chrome->tabs;
    if(scalar(@$tabs) > 0) {
        $chrome->eval($tabs->[-1]->[0],
                                 "javascript:window.location.reload()");
    }
}

I modified the original Vim snippet mentioned above so that each time I save my code, both the browsers get auto-refreshed:

autocmd BufWriteCmd *.html,*.htm,*.css,*.gtpl,*.tt2 :call Refresh_firefox()
function! Refresh_firefox()
  if &modified
    write
    silent !echo  'vimYo = content.window.pageYOffset;
                 \ vimXo = content.window.pageXOffset;
                 \ BrowserReload();
                 \ content.window.scrollTo(vimXo,vimYo);
                 \ repl.quit();'  |
                 \ nc localhost 4242 2>&1 > /dev/null

    silent !perl -I/Users/deepakg/Projects /Users/deepakg/Projects/refresh.pl
  endif
endfunction

Of course, you’ll need to change /Users/deepakg/Projects to where you save the ChromeTool.pm file and refresh.pl. Here is a quick screencast showing this in action:

Credits: I learned about existence of Chrome Developer Tools Protocol from this post. You can also find link to a Ruby REPL client for talking to Chrome on the same page.

Nginx ships with support for embedded Perl. At the moment execution of Perl code blocks the nginx worker process and therefore anything that might take an indeterminate amount of time to finish (say a DB query) is discouraged.

That said, there could be scenarios where it could come in very handy – like redirecting users to browser-specific static content, generating CAPTCHA – and given Perl’s versatility I am sure several other.

Unlike Apache – where you can load mod_perl as a module, the embedded Perl support in nginx has to be “baked in” at the time of compilation.

Assuming you downloaded nginx-0.7.65, here is how you’ll build it with Perl support:

cd nginx-0.7.65
/configure --with-http_perl_module
make

Things should go smoothly from here, but you might get the following error:

	objs/ngx_modules.o \
	-lcrypt -lpcre -lcrypto -lz \
	-Wl,-E -L/usr/local/lib -L/usr/lib/perl/5.10/CORE -lperl -ldl -lm \
        -lpthread -lc -lcrypt
/usr/bin/ld: cannot find -lperl
collect2: ld returned 1 exit status
make[1]: *** [objs/nginx] Error 1
make[1]: Leaving directory `/home/deepakg/nginx-0.7.65'
make: *** [build] Error 2

To fix it create a symbolic link – libperl.so, that points to the version of libperl installed on your system:

cd /usr/lib
sudo ln -s libperl.so.5.10.0 libperl.so

You might need to replace libperl.so.5.10.0 with the version of Perl installed on your system. The compilation should now go smoothly. From here you can follow the usage examples off the nginx Wiki

Ruby 1.9, among other things, brings much needed improvements to the way unicode strings are handled. The string class now includes a property called encoding which tells us the – well – encoding of a given string. By default a string’s encoding is same as the encoding of the source file, which in turn can be set by using the coding comment. For example, to use utf8 as the source’s encoding (and to be able to use utf8 characters as part of string literals) you’d use: # -*- coding: utf-8 -*-

Let’s look at some sample code and its output under Ruby 1.9

Code:

# -*- coding: utf-8 -*-
str = "café"
puts "Encoding    : #{str.encoding}"
puts "Length      : #{str.length}"
puts "Byte Size   : #{str.bytesize}"
puts "#{str} in upper case is: #{str.upcase}"

Output:

Encoding    : UTF-8
Length      : 4
Byte Size   : 5
café in upper case is: CAFé

As is evident from the output above, Ruby 1.9 still doesn’t handle casing beyond the ASCII range. Upper casing café, gave us CAFé as opposed to CAFÉ (which is the correct response).

Also the byte size of the string is 5 because under the utf-8 encoding, é takes up two bytes – 0xC3, 0xE9.

MacRuby – to quote the project site – is a version of Ruby 1.9, ported to run directly on top of Mac OS X core technologies such as the Objective-C common runtime and garbage collector, and the CoreFoundation framework.

This means that the Ruby datatypes have been implemented on top of Mac “native” (Cocoa) datatypes – e.g. Ruby strings are implemented on top NSString.

This introduces some differences in the way strings are handled by MacRuby. To start with, non-uncode strings use the ‘MACINTOSH’ encoding (Ruby 1.9 default is US-ASCII) while the unicode strings use utf-16 (even if you’ve set the coding comment to use utf-8). MacRuby also handles casing correctly.

So the same code snippet as above gives different output:

Output:

Encoding    : UTF-16
Length      : 4
Byte Size   : 4
café in upper case is: CAFÉ

Note that casing is handled correctly by MacRuby.

The byte size for the string is 4 because É is 0xE9 under the utf-16 encoding. Technically most characters, when using the utf-16 encoding, should take up 2 bytes (c should be 0×0043, é should be 0x00E9 and so on) but I guess the most significant byte is not used if it is 0×00.

p.s. the versions of the products used in the examples above are:

1. Ruby – 1.9.1p376 (2009-12-07 revision 26041) [i386-darwin10] (installed via macports 1.8.2)

2. MacRuby version 0.5 (ruby 1.9.0) [universal-darwin10.0, x86_64] (binary distribution from the official MacRuby project site)

I was recently looking for a Perl Captcha implementation that’d generate number-only Captchas. I looked inside Authen::Captcha and figured I could subclass it do just numeric captchas. It turned out to be a lot simpler than I thought it would be. Here is the code:

package Authen::NumCaptcha;

use strict;
use base qw(Authen::Captcha);

sub new {
    my $class = shift;
    my $captcha = $class->SUPER::new( @_ );
    return bless $captcha, $class;
}

sub generate_random_string {
    my $self = shift;
    my $length = shift;

    my $code = "";
    my $char;

    for (my $i=0; $i < $length; $i++) {
        $char = int(rand 7)+50;
        $char = chr($char);
        $code .= $char;
    }

    return $code;
}

1;

We basically override Authen::Captcha's generate_random_string and only use ASCII characters 50 to 57 (digits 2 to 9). Authen::Captcha doesn't include the digits 0 and 1 because they could be confused with oh (O) and lower case elle (l) or upper case ai (I).

To use Authen::NumCaptcha, place it under /usr/share/perl5/Authen/ (or wherever you have Authen::Captcha installed) alongside Captcha.pm. Then use it as you'd use Auhten::Captcha. For example:

use Authen::NumCaptcha;

my $num_chars = 6;
my $num_captcha = Authen::NumCaptcha->new (
                                          data_folder => '/tmp',
                                          output_folder => '/tmp'
                                          );

my $num_md5sum = $num_captcha->generate_code($num_chars);

If you are building an X-less command-line-only Debian VM, here is what you’ll need to do in order to install VMWare tools so that you can use features like shared folders:

1. Login as root or su.
2. Make sure you have installed the kernel sources and build tools.

apt-get install build-essential

Then run:

apt-get install linux-headers

This will probably give you a message saying: “Package linux-headers is a virtual package provided by:”, followed by a list of available kernel versions. Choose your version by looking it up:

cat /proc/version

or

uname -a

e.g. in my case I chose:

apt-get install linux-headers-2.6-2-amd64

3. Next choose Virtual Machine -> Install VMWare Tools option from the menu (this assumes VMWare Fusion running on Mac but there should be a similar option for VMWare Workstation on Linux and Windows as well).

Now mount the VMWare tools virtual CD:


mount /cdrom
cd /cdrom


Copy VMWare tool source code to /tmp and extract the files (the actual filename will depend on your VMWare version [which the VMWare installer would have reported in case of a mismatch]):

cp VMWareTools-7.9.6-173382.tar.gz /tmp
cd /tmp
tar -xzvf VMWareTools-7.9.6-173382.tar.gz


At this stage you should have a folder called vmware-tools-distrib inside your /tmp folder. Visit this folder and run vmware-install.pl, then follow the on-screen instructions.

cd /tmp/vmware-tools-distrib
./vmware-install.pl


Now at one stage the VMWare tool installer complained about my installed gcc (4.3) version being different from the gcc version that was used to compile my kernel (4.1). Press Ctrl+C to abort the installer at this stage.

Turns out that gcc is merely a symbolic link in /usr/bin folder. Chances are that you’ll have 2 versions of gcc on your system (e.g. Lenny comes with /usr/bin/gcc-4.3 and /usr/bin/gcc-4.1). Just make the symlink point to the version of gcc that was used to compile your kernel:

cd /usr/bin
rm gcc
ln -s /usr/bin/gcc-4.1 gcc


Run vmware-install.pl again and this time things should go through. Restart your VM to finish the installation. You should now be able to see folders shared from the host machine under /mnt/hgfs.

Most of these steps should also apply to Ubuntu.

I was at Microsoft TechEd India 2009 yesterday and presented a session titled – jQuery – the ‘write less do more’ javascript library. You can download the slides and demos here (Zip file, 1.07 MB).

The demos on Ajax will require you to place the files in the Demos/3-Ajax folder on a web server (http://localhost will do). You’ll also need to get the php files up and running. Finally, you’ll need to edit default.htm in the Demos/3-Ajax folder to change the paths to point to where you have hosted the php files.

Among other things, mod_perl allows you to write authentication handlers that fit nicely into Apache’s authentication scheme. A Perl authentication handler is a Perl module that decides whether a given user gets access to a resource or not. How you determine whether the user is allowed or denied access is up to you.

The mod_perl documentation comes with a sample that checks the length of the username+space+password supplied by a user and allows the user access only if the length is 14 characters long. Here is what the code looks like:

  package MyApache2::SecretLengthAuth;

  use strict;
  use warnings;

  use Apache2::Access ();
  use Apache2::RequestUtil ();

  use Apache2::Const -compile => qw(OK DECLINED HTTP_UNAUTHORIZED);

  use constant SECRET_LENGTH => 14;

  sub handler {
      my $r = shift;

      my ($status, $password) = $r->get_basic_auth_pw;
      return $status unless $status == Apache2::Const::OK;

      return Apache2::Const::OK
          if SECRET_LENGTH == length join " ", $r->user, $password;

      $r->note_basic_auth_failure;
      return Apache2::Const::HTTP_UNAUTHORIZED;
  }

  1;

And you hook this handler up in your apache2.conf as:

  <Location / >
      PerlAuthenHandler MyApache2::SecretLengthAuth
      AuthType Basic
      AuthName "The Gate"
      Require valid-user
  </Location>

As the documentation points out, the authentication handler can be configured for any sub section of the site, it doesn’t matter if it is served by a mod_perl response handler or not.

Now obviously in real life, you’d want an authentication handler that does a little more – like verify the password supplied by the user against a database or LDAP . Here is a version that I wrote that uses Perl’s Net::POP3 package to verify user’s credential against a POP 3 server.

package Apache::PopAuth;

use strict;
use warnings;
use Net::POP3;

use Apache2::Access ();
use Apache2::RequestUtil ();
use Apache2::RequestRec ();
use Apache2::Const -compile => qw(OK DECLINED HTTP_UNAUTHORIZED);

sub handler {
    my $r = shift;
    my ($status, $password) = $r->get_basic_auth_pw;
    my $user = $r->user;

    return $status unless $status == Apache2::Const::OK;

    return Apache2::Const::OK
	   if valid_user($user, $password);

    $r->note_basic_auth_failure;
    return Apache2::Const::HTTP_UNAUTHORIZED;
}

sub valid_user {
    my($user, $password) = @_;
    my $pop = Net::POP3->new('pop.yourdomain.com');
    my $status = $pop->login($user, $password);
    $pop->quit();
    return defined($status);
}
1;

This saves you from having to maintain a separate set of users on your web server and passes on the baton of user management to the POP server. One word of caution – this just proof of concept code and I am not sure on well it will scale outside a small intranet.

Ruby has a nifty feature that allows you to “subtract” two arrays. e.g.

Fruit = ["Apple", "Kinnow", "Mango", "Orange"]
Citrus = ["Lemon", "Kinnow", "Orange", "Tangerine"]


Then Fruit – Citrus gives:
["Apple", "Mango"]


Notice that elements in Citrus not in Fruit (Lemon, Tangerine) are not in the difference.

Now I needed something similar in Javascript. So I started by pushing my luck:

var Fruit = ["Apple", "Kinnow", "Mango", "Orange"];
var Citrus = ["Lemon", "Kinnow", "Orange", "Tangerine"];
var Diff = Fruit - Citrus;

Depending on where you are running this code, Diff will be 0 or NaN. This meant that I would have to come up with something of my own. I figured I’ll put javascript’s regular expressions to some use and came up with this:

function diffArrays (A, B) {

  var strA = ":" + A.join("::") + ":";
  var strB = ":" +  B.join(":|:") + ":";

  var reg = new RegExp("(" + strB + ")","gi");

  var strDiff = strA.replace(reg,"").replace(/^:/,"").replace(/:$/,"");

  var arrDiff = strDiff.split("::");

  return arrDiff;
}

diffArrays(Fruit, Citrus) gives:
["Apple", "Mango"]


The thing with dynamically typed languages is that you can take even numeric arrays through this regular expression rigmarole and get a correct result back without anyone complaining:

var Natural = [1,2,3,4,5,6];
var Prime = [1,2,3,5,7];


diffArrays(Natural, Prime) gives:
[4, 6]

A brief explanation of how this works:

I start by converting the first array to a string delimited with two colons, :: – the choice was arbitrary. The fact that regular expressions don’t treat a colon any specially helps. I then prefix and suffix the resulting string with another colon. The idea here is to have each element “surrounded” by its own pair of colons. For instance, [1,2,3,4,5,6] becomes :1::2::3::4::5::6:

The array to be subtracted from it – say [1,2,3,5] is converted to the form – :1:|:2:|:3:|:5: which I then use to create a regular expression. The pipe | has a special meaning in the regular expression world and will cause :1: or :2: or :3: or :5: to be matched.

Calling strA.replace with our just-crafted regular expression replaces :1:, :2:, :3: and :5: with “” giving us :4::6:, which I strip clean of the leading and trailing colons through another couple of replace calls. Finally, spitting the string on :: gives the delta array!

Now this implementation is certainly not going to win me any prizes in speed/elegance pageants, but I think this somewhat awkward application of regular expression was something worth sharing!

On a somewhat related note, a special thanks goes out to Steve Yegge for his wonderful js2-mode and ejacs. The latter especially comes in handy when you are just “doodling” around on JavaScript problems like these.

A mod_perl handler can parse the incoming client request (querystring, form post data etc) using Apache2::Request. It is *not* installed when you install mod_perl. Getting it working is a 3 step process.

First issue the following command:

sudo apt-get install libapreq2

This installs 2 things – the libapreq2 shared library, and an Apache module – mod_apreq2.

Next we install the Perl bindings – Apache2::Request – which we use in our handler code.

sudo apt-get install libapache2-request-perl

At this stage if you restart Apache, it will load your Perl handler without any complaints. However if you visit a handler that uses Apache2::Request, it’ll error out with the following entry in error.log:


/usr/sbin/apache2: symbol lookup error: /usr/lib/perl5/auto/APR/Request/Apache2/Apache2.so: undefined symbol: apreq_handle_apache2


This is because unlike our mod_perl installation, apt-get doesn’t enable mod_apreq2 after installing it. We enable it manually by creating a symbolic link to /etc/apache2/mods-available/apreq.load under /etc/apache2/mods-enabled/:


sudo bash
cd /etc/apache2/mods-enabled
ln -s ../mods-available/apreq.load
apache2ctl restart


Updated: You can also run a2enmod to enable an Apache module and use a2dismod to disable it.

Visit your handler url again and this time it should work withour errors. Here is the modified handler from the mod_perl installation post that now uses Apache2::Request to enable you to test if your installation is working correctly:

package Hello;
use strict;

use Apache2::RequestRec ();
use Apache2::RequestIO ();

use Apache2::Const -compile => qw(OK);

use Apache2::Request;

sub handler {
    my $r = shift;

    $r->content_type('text/plain');

    my $req = Apache2::Request->new($r);
    my $name = $req->param("name");
    $name = $name ? $name : "World";

    print "Hello $name, the time here is " . localtime() . "\n";

    return Apache2::Const::OK;
}

1;

Ubuntu allows you to add and remove components from your system using apt-get. Perl allows similar functionality for maintaing Perl modules through the CPAN module’s shell (invoked by perl -MCPAN -eshell). Now if you are just going to add a Perl module to your system, should you be using apt-get or CPAN?

I prefer to use apt-get because this allows me to keep track of everything I’ve added to my system in one place. It also keeps things neat in case a given Perl module has binary dependencies. For 99% of the cases, I’ve found an apt package equivalent of a given CPAN module. The naming convention of the modules varies between CPAN and apt. For example, the package Algorithm::Diff at CPAN, is known as libalgortihm-diff-perl in the apt world.

So sometimes the trick is knowing the correct apt name for a CPAN module. Now in most cases, if you know the Perl name of a module – say XML::Simple then you can arrive at the apt name by converting the package name to lowercase, replacing the “::” with a “-”, prefixing “lib” and suffixing -perl to it. i.e.


echo "XML::Simple" | perl -e '$x=<>; chomp($x); $x=~s/::/-/; $x=lc($x); print "lib$x-perl"'


And if you want to automate things further:

sudo bash
echo "XML::Simple" | perl -e '$x=<>; chomp($x); $x=~s/::/-/; $x=lc($x); print "lib$x-perl"' | xargs apt-get install


Just bear in mind that there are exceptions to this naming convention. The one I ran into recently involved Sablotron; which is known as XML::Sablotron in the CPAN world, but as libxml-sablot-perl in the apt world.

Also, apt might not always have the most recent version of a given module. So you have to depend on CPAN if you need the latest, greatest version of a module.

Lastly, some modules might not have an apt version – for example at the moment Devel::PerlySense exists on CPAN alone.