I’ll have a better fix up in the near future, but for now, the ruby gem “Nokogiri” crashed out on me on Joyent’s Solaris platform. Its error was a little cryptic unless you’re familiar with libc. I can’t reproduce it here because unfortunately it’s lost in the sands of scrollback buffer.

The crux of the problem was the reference to a missing vasprintf function. Nokogiri has an extension implemented in C that speeds it up with native functions. The extension is compiled when you do ‘gem install nokogiri’ and will compile even though the vasprintf function is missing in your environment.

vasprintf is available in Linux, but not in Windows or Solaris. Nokogiri already have a workaround in place for Windows, but they don’t yet detect the issue in Solaris so the error still occurs. I plan to submit a patch to detect this automatically. If for any reason you need to get nokogiri working immediately on your Solaris 11-based Joyent accelerator, I did the following after installing the nokogiri gem:

cd /opt/local/lib/ruby/gems/1.8/gems/nokogiri-1.2.3/ext/nokogiri/

vi native.c

[remove the #if XP_WIN line and its corresponding #endif]

vi native.h

[remove the #if XP_WIN line and its corresponding #endif]

make

make install

FWIW, on Linux and Mac OS X nokogiri just works.

BCP is a method for quickly copying data into SQL Server or Sybase databases. It’s much faster than running INSERT statements, so if you have a large amount of data, it’s a tempting option.

However, the trade-off for the speed is that when importing data, it bypasses a large amount of error checking and handling. As a result, any error messages you do get from the database server can often be cryptic or can even appear to be unrelated to what you’re trying to accomplish!

As well as the command-line BCP tool, JBCP is a pure Java library that reimplements the BCP protocol for Java developers. Unfortunately it’s still relatively immature, and you may find that Sybase gives the following error message in response to almost all JBCP issues:

Bad row data received from the client while bulk copying into object 1874356894 in database 4

This can be caused by several things. The most common is simply that the table definition has not been setup correctly to allow rows to be BCP’d into it. One way to resolve the issue is to ensure that the table you are copying into has ‘lock allpages’ as part of its definition, for example:

CREATE TABLE bcptest (some_key int not null, mydata varchar(50) null) lock allpages
go


Other issues can be caused by using univarchar columns that exceed 255 bytes. This means that all univarchar columns you attempt to copy into must be no larger than univarchar(127). A valid workaround for this issue is to split large column definitions into several smaller columns, and then use an UPDATE statement to reassemble the data from the split columns after it has been uploaded.

To help googlers, here is the complete stack trace I was typically seeing from JBCP before I created my table with the additional lock allpages clauses.

java.sql.SQLException: Bad row data received from the client while bulk
copying into object 1874356894 in database 4. Received a row of length
72 whilst maximum or expected row length is 15.
[19/12 10:46:37.160] [TID:lee2BCP1]:Upload of 1 result: FAILED
       at
net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
       at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:3149)
       at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2587)
       at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:734)
       at net.sourceforge.jtds.jdbc.TdsCore.bcpBatch(TdsCore.java:2394)
       at net.sourceforge.jtds.jdbc.BCP.bcpBatch(BCP.java:1134)
..snip..


One final warning for using JBCP - you don’t always even see the problem by default - some exceptions are supressed causing silent failure! There be dragons!

rubygems makes package management a nightmare. This is demonstrated every time you try and setup a new server using your operating system’s ruby/rails packages. It gets fully demonstrated when you try and upgrade to Rails 2.2 on a Mac. You can go read the full details of the nightmare if you’re so inclined.

The summary is that rails 2.2 removes the mysql driver and so you have to install a native driver using rubygems. Normally this would be a single command, but MySQL is a mess on Mac OS X. If you have a working MySQL installation on a Mac, chances are you’ve used MAMP and/or installed it yourself.

After about two hours of pain, I finally came across this forum post which has the horribly hacky but wonderfully simple idea of copying the mysql driver that worked in older versions of rails into your rails 2.2 app! This finally worked! On my Mac, this meant running the following command:

 cp /Library/Ruby/Gems/1.8/gems/activerecord-2.1.2/lib/active_record/vendor/mysql.rb ~/Software/myapp/lib/

When I restarted my rails app, it actually worked! Joy!

It’s worth noting that it’s probably worth pushing through the pain to install the native mysql gem on your production servers, but this technique is a winner to keep development on the go.

I’m pleased to announce the first public beta release of an exciting new service from Crossbone Systems.

Head Hunting is a revolutionary new way to recruit employees at a fraction of the cost of other methods such as recruitment agencies and big job sites. As a Facebook application, Head Hunting allows you to hire new employees with a little help from your friends.

This week the service has launched into “public beta”, which means anyone can post and search job listings for free.

Read the rest of this entry »

I’ve been trying to get a slider bar working recently as part of a facebook application. To get the example on the Facebook wiki working out-of-the-box was no problem.

The problems came when I wanted to create the slider bar as the result of a user clicking on an input. For some reason, Facebook only loads 3rd party Javascript (ie. mine) in the body of the page. If a user clicks a link that calls a Javascript function before the page has finished loading, the function may fail if it depends on functions kept in external Javascript files (that have not yet loaded). 

Because of the complexity of Facebook pages, there is often between 1 and 5 seconds when a user may click one of my links even though the page hasn’t technically finished loading. If this happens, the user will either see a very cryptic error or nothing will happen. It’s a completely unacceptable user experience so I was forced to come up with the workaround below.  

var sliderCreated = false; // can only create one slider per-page with this
function createSlider() {
    if (!sliderCreated) {
        try {
            new slider(document.getElementById('output'), document.getElementById('count_slider'), 0, 100000, 250, document.getElementById('salary_min'), , 500);
            sliderCreated = true;
        } catch (error) {
            // Errors may get thrown if the fbslider.js wasn't loaded quickly enough, so we wait a second
            setTimeout(function() { createSlider() }, 1000);
        }
    }
}

This will detect any errors thrown from the creation of the slider (which relies on an external Javascript file). If an error is found, then the attempt to create the slider will occur again after a second. This has the advantage that even if the second attempt fails (if the network is very slow), it will continue to retry with a delay between each attempt.

A variable (sliderCreated) is used to detect when the slider has been successfully created. In case the link that creates the slider is still clickable after the slider is created, this variable guards against multiple sliders being spawned on the page.

It is left as an exercise to the reader to add an upper limit to the retries if required.