"invalid byte sequence in US-ASCII" when issuing rake commands

Ruby on Rails
1

I recently upgraded my environment to Rails 3.2.13, Ruby 1.9.3p545, and rubyenv from rvm. In the process, I’ve also upgraded several gems (nokogiri, most painfully) and attempted to add devise to my application. I can get the rails server to start no problem, but rake tasks (e.g., rake db:migrate or rake routes) all seem to produce an “invalid byte sequence in US-ASCII” error message.

I’ve implemented the suggested fixes in several SO and Google Groups threads, adding

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8

at the top of my Gemfile and config/environment.rb files. I also issued these commands from my bash prompt:

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

and added this line to my migration file:

#encoding: utf-8

Still no luck. Here’s the error I get:

hikari-2:genlighten dean$ bundle exec rake db:migrate --trace
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `block in require': iconv will be deprecated in the future, use String#encode instead.
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.7.3
rake aborted!
rake aborted!
invalid byte sequence in US-ASCII
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/trace_output.rb:16:in `block in trace_on'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/trace_output.rb:14:in `map'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/trace_output.rb:14:in `trace_on'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:328:in `trace'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:183:in `display_error_message'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:169:in `rescue in standard_exception_handling'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:159:in `standard_exception_handling'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:88:in `load_rakefile'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:72:in `block in run'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
/Users/dean/.rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rake-10.0.4/bin/rake:33:in `<top (required)>'
/Users/dean/.rbenv/versions/1.9.3-p545/bin/rake:23:in `load'
/Users/dean/.rbenv/versions/1.9.3-p545/bin/rake:23:in `<main>'

There are no obvious uncommon characters in my code (that I’m aware of, anyway.)

Any suggestions for how to proceed?

Thanks,

Dean Richardson

2

Riffing on a suggestion in a SO post I found linked to some of the ones mentioned above, I went through each of my existing rake tasks looking for non-standard characters. I didn’t find any, but inserted #encoding: utf-8 at the top of several files that were seeding databases with placenames from non-english-speaking countries. It’s still not obvious to me which character was in which file, but in the end I got past the error message.

3

I was having the exact same problem when deploying from my home machine which is a macbook air. Ended up putting these lines on my ~/.bashrc and from time to time I have to issue a source ~/.bashrc to fix it.

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

And in my case it was a gem that was causing that so I couldn’t do much about it.

4

Encoding issues recently came up for me on an older project, in the end it was missing readline support when the Ruby version was built.

If you run ruby -r rbconfig -e 'puts Config::CONFIG["configure_args"]' in your command line you should see an option similar to this: '--with-readline-dir=/usr/local/opt/readline' in the output.

Checkout the link below from ruby-build wiki on how to tell it how to find readline Home · rbenv/ruby-build Wiki · GitHub

Hope this helps with your situation.