It doesn’t happen very often, but I was pointed to an online article recently that dates back almost a decade. The Pitchman is a story by Malcolm Gladwell about ”Ron Popeil, who invented a better rotisserie in his kitchen and went out and pitched it himself.”

“Like most great innovations, it was disruptive. And how do you persuade people to disrupt their lives? Not merely by ingratiation or sincerity, and not by being famous or beautiful. You have to explain the invention to customers– not once or twice but three or four times, with a different twist each time. You have to show them exactly how it works and why it works, and make them follow your hands as you chop liver with it, and then tell them precisely how it fits into their routine, and, finally, sell them on the paradoxical fact that, revolutionary as the gadget is, it’s not at all hard to use.”

“in every respect the design of the product must support the transparency and effectiveness of its performance during a demonstration - the better it looks onstage, the easier it is for the pitchman to … ask for the money.”

While Ron and the article are focused on gadgets and devices, the above holds true for any kind of innovation that you’re trying to convince people to buy, or in the case of a free service, just to use.

I’ll have a better fix up in the near future, but for now, the ruby gem “Nokogiri” crashed out on me on Joyent’s Solaris platform. Its error was a little cryptic unless you’re familiar with libc. I can’t reproduce it here because unfortunately it’s lost in the sands of scrollback buffer.

The crux of the problem was the reference to a missing vasprintf function. Nokogiri has an extension implemented in C that speeds it up with native functions. The extension is compiled when you do ‘gem install nokogiri’ and will compile even though the vasprintf function is missing in your environment.

vasprintf is available in Linux, but not in Windows or Solaris. Nokogiri already have a workaround in place for Windows, but they don’t yet detect the issue in Solaris so the error still occurs. I plan to submit a patch to detect this automatically. If for any reason you need to get nokogiri working immediately on your Solaris 11-based Joyent accelerator, I did the following after installing the nokogiri gem:

cd /opt/local/lib/ruby/gems/1.8/gems/nokogiri-1.2.3/ext/nokogiri/

vi native.c

[remove the #if XP_WIN line and its corresponding #endif]

vi native.h

[remove the #if XP_WIN line and its corresponding #endif]

make

make install

FWIW, on Linux and Mac OS X nokogiri just works.

BCP is a method for quickly copying data into SQL Server or Sybase databases. It’s much faster than running INSERT statements, so if you have a large amount of data, it’s a tempting option.

However, the trade-off for the speed is that when importing data, it bypasses a large amount of error checking and handling. As a result, any error messages you do get from the database server can often be cryptic or can even appear to be unrelated to what you’re trying to accomplish!

As well as the command-line BCP tool, JBCP is a pure Java library that reimplements the BCP protocol for Java developers. Unfortunately it’s still relatively immature, and you may find that Sybase gives the following error message in response to almost all JBCP issues:

Bad row data received from the client while bulk copying into object 1874356894 in database 4

This can be caused by several things. The most common is simply that the table definition has not been setup correctly to allow rows to be BCP’d into it. One way to resolve the issue is to ensure that the table you are copying into has ‘lock allpages’ as part of its definition, for example:

CREATE TABLE bcptest (some_key int not null, mydata varchar(50) null) lock allpages
go


Other issues can be caused by using univarchar columns that exceed 255 bytes. This means that all univarchar columns you attempt to copy into must be no larger than univarchar(127). A valid workaround for this issue is to split large column definitions into several smaller columns, and then use an UPDATE statement to reassemble the data from the split columns after it has been uploaded.

To help googlers, here is the complete stack trace I was typically seeing from JBCP before I created my table with the additional lock allpages clauses.

java.sql.SQLException: Bad row data received from the client while bulk
copying into object 1874356894 in database 4. Received a row of length
72 whilst maximum or expected row length is 15.
[19/12 10:46:37.160] [TID:lee2BCP1]:Upload of 1 result: FAILED
       at
net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
       at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:3149)
       at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2587)
       at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:734)
       at net.sourceforge.jtds.jdbc.TdsCore.bcpBatch(TdsCore.java:2394)
       at net.sourceforge.jtds.jdbc.BCP.bcpBatch(BCP.java:1134)
..snip..


One final warning for using JBCP - you don’t always even see the problem by default - some exceptions are supressed causing silent failure! There be dragons!

rubygems makes package management a nightmare. This is demonstrated every time you try and setup a new server using your operating system’s ruby/rails packages. It gets fully demonstrated when you try and upgrade to Rails 2.2 on a Mac. You can go read the full details of the nightmare if you’re so inclined.

The summary is that rails 2.2 removes the mysql driver and so you have to install a native driver using rubygems. Normally this would be a single command, but MySQL is a mess on Mac OS X. If you have a working MySQL installation on a Mac, chances are you’ve used MAMP and/or installed it yourself.

After about two hours of pain, I finally came across this forum post which has the horribly hacky but wonderfully simple idea of copying the mysql driver that worked in older versions of rails into your rails 2.2 app! This finally worked! On my Mac, this meant running the following command:

 cp /Library/Ruby/Gems/1.8/gems/activerecord-2.1.2/lib/active_record/vendor/mysql.rb ~/Software/myapp/lib/

When I restarted my rails app, it actually worked! Joy!

It’s worth noting that it’s probably worth pushing through the pain to install the native mysql gem on your production servers, but this technique is a winner to keep development on the go.

I’m pleased to announce the first public beta release of an exciting new service from Crossbone Systems.

Head Hunting is a revolutionary new way to recruit employees at a fraction of the cost of other methods such as recruitment agencies and big job sites. As a Facebook application, Head Hunting allows you to hire new employees with a little help from your friends.

This week the service has launched into “public beta”, which means anyone can post and search job listings for free.

Read the rest of this entry »