I recently created a new slicehost and thought I’d try the latest Ubuntu release with it. Unlike brightbox hosting, you only get a plain boring ubuntu installation out of the box on slicehost (and various other VPS-based hosting services).

A bit of aptitude

Ubuntu/Debian make it really nice and easy to install packages, but this time it still took me a couple of hours to install everything.

For my (and your) future reference, here’s all you need to create a full rails/mysql/apache platform on Ubuntu 9.04:

apt-get install rubygems ruby1.8-dev mysql-server-5.1 libmysqlclient15-dev apache2 libopenssl-ruby libxslt1-dev libcurl4-gnutls-dev build-essential apache2-prefork-dev libapr1-dev libaprutil1-dev libmagickwand-dev git-core subversion

gem install rails --version=2.2.2
gem install rake mysql passenger


cd /to/your/app
rake gems:install


Be sure to follow the instructions that appear after installing passenger, and then that’s it! Make sure you set the version number of rails that you need, or remove the –version parameter if you just want the latest stable release.

(Note: the last line assumes you’ve added all the gems your application requires to config/environment.rb. You could make the above list of packages even more minimal, but they cover plenty of common gem dependencies).

Going the extra mile

If you need to setup an environment to create screenshots programmatically, you’ll need a virtual X server, firefox and selenium (which is Java-based). That’s also fairly easy:

apt-get install xvfb firefox latex-xft-fonts sun-java6-jdk gsfonts-x11 sun-java6-fonts
gem install selenium-client

Do you have any must install packages when creating a new server? Please add them in the comments below.

It doesn’t happen very often, but I was pointed to an online article recently that dates back almost a decade. The Pitchman is a story by Malcolm Gladwell about ”Ron Popeil, who invented a better rotisserie in his kitchen and went out and pitched it himself.”

“Like most great innovations, it was disruptive. And how do you persuade people to disrupt their lives? Not merely by ingratiation or sincerity, and not by being famous or beautiful. You have to explain the invention to customers– not once or twice but three or four times, with a different twist each time. You have to show them exactly how it works and why it works, and make them follow your hands as you chop liver with it, and then tell them precisely how it fits into their routine, and, finally, sell them on the paradoxical fact that, revolutionary as the gadget is, it’s not at all hard to use.”

“in every respect the design of the product must support the transparency and effectiveness of its performance during a demonstration - the better it looks onstage, the easier it is for the pitchman to … ask for the money.”

While Ron and the article are focused on gadgets and devices, the above holds true for any kind of innovation that you’re trying to convince people to buy, or in the case of a free service, just to use.

I’ll have a better fix up in the near future, but for now, the ruby gem “Nokogiri” crashed out on me on Joyent’s Solaris platform. Its error was a little cryptic unless you’re familiar with libc. I can’t reproduce it here because unfortunately it’s lost in the sands of scrollback buffer.

The crux of the problem was the reference to a missing vasprintf function. Nokogiri has an extension implemented in C that speeds it up with native functions. The extension is compiled when you do ‘gem install nokogiri’ and will compile even though the vasprintf function is missing in your environment.

vasprintf is available in Linux, but not in Windows or Solaris. Nokogiri already have a workaround in place for Windows, but they don’t yet detect the issue in Solaris so the error still occurs. I plan to submit a patch to detect this automatically. If for any reason you need to get nokogiri working immediately on your Solaris 11-based Joyent accelerator, I did the following after installing the nokogiri gem:

cd /opt/local/lib/ruby/gems/1.8/gems/nokogiri-1.2.3/ext/nokogiri/

vi native.c

[remove the #if XP_WIN line and its corresponding #endif]

vi native.h

[remove the #if XP_WIN line and its corresponding #endif]

make

make install

FWIW, on Linux and Mac OS X nokogiri just works.

BCP is a method for quickly copying data into SQL Server or Sybase databases. It’s much faster than running INSERT statements, so if you have a large amount of data, it’s a tempting option.

However, the trade-off for the speed is that when importing data, it bypasses a large amount of error checking and handling. As a result, any error messages you do get from the database server can often be cryptic or can even appear to be unrelated to what you’re trying to accomplish!

As well as the command-line BCP tool, JBCP is a pure Java library that reimplements the BCP protocol for Java developers. Unfortunately it’s still relatively immature, and you may find that Sybase gives the following error message in response to almost all JBCP issues:

Bad row data received from the client while bulk copying into object 1874356894 in database 4

This can be caused by several things. The most common is simply that the table definition has not been setup correctly to allow rows to be BCP’d into it. One way to resolve the issue is to ensure that the table you are copying into has ‘lock allpages’ as part of its definition, for example:

CREATE TABLE bcptest (some_key int not null, mydata varchar(50) null) lock allpages
go


Other issues can be caused by using univarchar columns that exceed 255 bytes. This means that all univarchar columns you attempt to copy into must be no larger than univarchar(127). A valid workaround for this issue is to split large column definitions into several smaller columns, and then use an UPDATE statement to reassemble the data from the split columns after it has been uploaded.

To help googlers, here is the complete stack trace I was typically seeing from JBCP before I created my table with the additional lock allpages clauses.

java.sql.SQLException: Bad row data received from the client while bulk
copying into object 1874356894 in database 4. Received a row of length
72 whilst maximum or expected row length is 15.
[19/12 10:46:37.160] [TID:lee2BCP1]:Upload of 1 result: FAILED
       at
net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
       at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:3149)
       at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2587)
       at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:734)
       at net.sourceforge.jtds.jdbc.TdsCore.bcpBatch(TdsCore.java:2394)
       at net.sourceforge.jtds.jdbc.BCP.bcpBatch(BCP.java:1134)
..snip..


One final warning for using JBCP - you don’t always even see the problem by default - some exceptions are supressed causing silent failure! There be dragons!

rubygems makes package management a nightmare. This is demonstrated every time you try and setup a new server using your operating system’s ruby/rails packages. It gets fully demonstrated when you try and upgrade to Rails 2.2 on a Mac. You can go read the full details of the nightmare if you’re so inclined.

The summary is that rails 2.2 removes the mysql driver and so you have to install a native driver using rubygems. Normally this would be a single command, but MySQL is a mess on Mac OS X. If you have a working MySQL installation on a Mac, chances are you’ve used MAMP and/or installed it yourself.

After about two hours of pain, I finally came across this forum post which has the horribly hacky but wonderfully simple idea of copying the mysql driver that worked in older versions of rails into your rails 2.2 app! This finally worked! On my Mac, this meant running the following command:

 cp /Library/Ruby/Gems/1.8/gems/activerecord-2.1.2/lib/active_record/vendor/mysql.rb ~/Software/myapp/lib/

When I restarted my rails app, it actually worked! Joy!

It’s worth noting that it’s probably worth pushing through the pain to install the native mysql gem on your production servers, but this technique is a winner to keep development on the go.