Rails, MySQL, and Text Encoding

My first project using Ruby on Rails and the excellent Refinery CMS is nearly complete. One especially irksome issue that cropped up during the course of the project had to do with text encoding.

This particular project was a fairly complex site for a local band here in Colorado Springs and included an online events listing, online store with support for digital downloads, etc., etc.  Anyway, I noticed that after I added a new event to the band calendar and visited the ‘events’ page, I would get the Rails “Oops! Something went wrong” message.  A closer examination of the error log revealed the following:

ActionView::Template::Error (incompatible character encodings: ASCII-8BIT and UTF-8)

Well, that’s strange.  I then remembered that some of the event information text I entered into Refinery had been copy/pasted from another site, and that upon closer examination of the text, I realized that it was not encoded in plain ASCII, but contained a few ‘exotic’ characters only available in UTF-8 (for a great introduction to unicode, read Joel Spolsky’s article on the subject).  I verified this problem by pasting in the same errant text into a page on this site (jamesadam.me is also running on Rails 3.0.11 and Refinery — UPDATE: I’ve since migrated this blog to WordPress), and got the same result. I couldn’t understand why this was happening, as I had configured my rails app, apache server, and mysql to work with UTF-8… or so I thought.

After much googling, I decided to replace the ‘mysql’ gem I was using with ‘mysql2’. Since I’m on Rails 3.0.11, I couldn’t use the latest version of mysql2 (>0.3), so I decided to use version 0.2.18, replacing the following line in my Gemfile:

gem 'mysql'

with

gem 'mysql2', '~> 0.2.18'

I then ran bundle install, and fired up my local rails server, only to watch it crash. Oops! I forgot to update the adapters in database.yml. So:

adapter: mysql

becomes

adapter: mysql2

Great! Now rails starts up, and after some testing, I know I can paste all the UTF-8 encoded text into the Refinery HTML editor I want, and it works!

So, time to deploy to the production server:

cap production deploy

I then pointed my web browser at the site, and got the pretty yet frustrating passenger error screen with the rather useless error message:

The application spawner server exited unexpectedly: Unexpected end-of-file detected

Gee, so helpful.. An examination of the Apache error log yielded something… also not so helpful:

*** Exception NameError in PhusionPassenger::Rack::ApplicationSpawner (uninitialized constant Gem::Deprecate) (process 16197, thread #)

Ok… in this case Google once again turned out to be my friend, and I discovered the source of this problem appeared to be with my version of Ruby Gems.  I was running Ruby Gems 1.8.15, so I decided to try downgrading to 1.8.10:

rvm rubygems 1.8.10

After restarting the app, it all works! If you’re reading this and are having the same problem, Hopefully I just saved you some time (and sanity).