Sunday, July 1, 2012

Installing MySql on mac using homebrew

I spent whole Saturday trying to install mysql on my mac but kept getting errors about pid file and complains about the socket. Homebrew instructions hadn't been helping much. At the end I restarted the system and by some miracle I didn't get any more errors and mysql works perfectly now. All that hassle and I still don't know what went wrong with installation. Oh, well.

Monday, April 30, 2012

Indexing Mongoid models in external gems

We have some models that live in external gem. The problem comes when we want to auto create indexes in mongodb based on these models definitions.

For models not in external gem, the solution is straightforward - run standard mongoid rake task:
bundle exec rake db:mongoid:create_indexes

But for models that come from external gem, that standard solution does not work since mongoid simply does not look that deep for models.

By looking into sources of mongoid document it reveals a handful method create_indexes. So we can use that in our own favor: find all the models in gems ourselves and call create_indexes function on each one.

Here's an example of a final rake task that does this. Note, it does some stuff specific to our application as well.

  namespace :mongoid   do   
    def find_mongoid_models(parent_module)   
     models = { |c| eval("#{parent_module}::#{c}") }  
     models = { |m| !m.kind_of?(Fixnum) and m.included_modules.include?(Mongoid::Document) }  
    task :create_indexes do   
     puts "Initializing the app ..."  
     require File.expand_path('init.rb', File.expand_path(File.dirname(__FILE__)))  
     puts "Creating indexes for #{ENV['RACK_ENV']} ..."  
     models =  
     models += find_mongoid_models(Models)  
     models += find_mongoid_models(PromoServer)  
     models += find_mongoid_models(VC)  
     puts "found mongoid documents #{models.to_a}"  
      models.each do |m|   
       puts "Generating indexes for document #{m}"  
        rescue Exception => e   
         puts "ERROR. #{e}"  
     puts "Complete."  

Wednesday, February 29, 2012

Mongoid before_save

Recently migrated the rest of our postgresql data to mongodb. For long time we had something like a hybrid - some models in activerecord (tied to postgresql), some models in mongoid (tied to mongodb). The migration went pretty smoothly except one problrm with callbacks.

ISSUE. Mongoid before_save callback is not exactly aligned with rails ActiveRecord before_save. The difference is that ActiveRecord invokes before_save prior to running any validation. Mongoid however does model validation first and then does before_save.

SOLUTION. Instead of before_save we use before_validation callback.

Monday, January 30, 2012

SecureRandom in Jruby workaround

As part of our migration to JRuby, one thing that had to be taken care of was SecureRandom.
We use that ruby class to generate UUIDs.
In JRuby it is not supported at the moment.

SecureRandom.uuid fails in JRuby. There is an open ticket for that

Use java implementation for generating UUIDs.
require 'java'; java.util.UUID.randomUUID().to_s

Run JRuby Sinatra app on Heroku

We made it work on Heroku! After making a regular java webapp work on Heroku (see it is time to deploy a JRuby Sinatra app.
The solution is based on this article with additional fixes of our own:

The working Jruby app is running at
Codebase including configurations is on github
Locally run it by:
foreman start
or just run what's in Procfile for web process ( target/bin/jruby --1.9 -S trinidad -p 8080 )

Not resolved gems and java libraries

Changes made to pom.xml with additional configurations, and to script/jruby bash script.
- Make maven (via pom.xml) install gems into vender/bundle (this is where Heroku expects them to be)
- Make maven create local repository in target/repo. (so all the java dependencies are available)
- Updates in script/jruby to properly resolve gem folder, maven repository and java libraries.
- Added maven instructions to copy script/jruby to target/bin/jruby (by convention this is where appassembler-maven-plugin puts all executables)

Run java project on Heroku (preparing for jruby)

We've been trying to run JRuby app on Heroku and faced some challenges. At the end to descope the problems we decided to run a simple java app on Heroku as the first step towards deploying Jruby.

It turned out even running a Hello World was not as simple as to just follow the Heroku instructions from

All problems were fixed and the working app is running here:
The code with all the right configs (pom.xml):
In order to locally run it:
mvn install
PORT=8080 target/bin/webapp

Some paths and libraries not found.
You can compare pom.xml and pom.heroku.broken.xml. The diff is what we had to do to make it all work.
- We had to force maven to create local repository and include that into target/repo.
- Specify java source code level to 1.5

Wednesday, December 28, 2011

Good lessons learned after using mongoDB service (mongoLab)

We don't host mongoDB ourselves but instead we use a service provided by mongoLab. Great company, they manage mongodb servers and host everything on Amazon EC2.

A few lessons that we learned while using mongodb hosted service:
  1. MongoDB profiling. You have to do the profiling. which will watch your queries and help identify the slow ones. In many cases it will be due to missing indexes. Some profiling instructions from mongolab too
  2. Shared machines. In mongoLab, mongoDB server is shared between multiple users. So if you run some bad performing mongo commands, you will affect other tenants. We once run a re-index command as a blocking operation, and this ended up freezing the whole mongoDB server (it was a blocking operation), so all the queries from other users just queued up. In fact the collection being re-indexed was huge and never-ending, so mongoLab team had to restart the server.
  3. Migrating data. It's a huge pain! Downtime cannot be avoided. So you better think through your infrastructure beforehand to avoid data migration. E.g. if you know your data will grow over a few gigabytes in 2-3 months, then just buy proper plan or setup your mongodb servers yourself right in the beginning. (with enough memory).  Or if it's going over dozens of GB, setup sharding right away and select smart keys for sharding.