Metaprogramming magic in Ruby

The past week I’ve been playing with implementing LinkedList in Ruby, mainly Single and Double. Both of them include an Enumerable like module called Iteration. It provides methods like: each, count, map, select etc. Double also inclues IterationReverse which provides: reverse, map_reverse etc. As you can tell there’s a lot of code common between the two Iteration modules. The only real difference is each. Iteration#each starts at tail and iterates to head, while IterationReverse#each starts at head and iterates to tail. head being the first node added to list, while tail is last node added. (I know crappy class names right, we’ll fix it later). Late last night I started thinking about ways to DRY the two modules, this post is direct result of that.

First I renamed Iteration and IterationReverse to BackwardIteration and ForwardIteration. My original plan then was to extract out the common methods into Iteration, while implementing each in the respective modules.

module Iteration
  def map
  end
end

module ForwardIteration
  include Iteration

  def each
  end

module BackwardIteration
  include Iteration

  def each
  end
end

class SingleLinkedList
  include ForwardIteration
end

class DoubleLinkedList
  include ForwardIteration
  include BackwardIteration
end

The problem here is when you include BackwardIteration in DoubleLinkedList, it will override ForwardIteration#each.

Before we get any further, let’s write some specs. The below should be fairly self explanatory. We test #first which returns first added node, #last, first’s counterpart. #to_a iterates through list from head to tail, while #to_a_reverse, well you can guess. The last test was a regression test, which we’ll get to later.

describe 'LinkedList' do
  context SingleLinkedList do
    subject { described_class.new }

    it 'returns first character' do
      subject.first.should == 'F'
    end
  end

  context DoubleLinkedList do
    subject { described_class.new }

    it 'returns first character' do
      subject.first.should == 'F'
    end

    it 'returns last character' do
      subject.first_reverse.should == 'B'
    end

    it 'returns array from head to tail' do
      subject.to_a.should == %w(F o r w a r d)
    end

    it 'returns array from tail to head' do
      subject.to_a_reverse.should == %w(B a c k w a r d)
    end

    it 'does not override #each method' do
      subject.to_a_reverse.should == %w(B a c k w a r d)
      subject.to_a.should == %w(F o r w a r d)
    end
  end
end

What we need to do is to stop ForwardIteration#each from clashing with BackwardIteration#each, while also renaming the methods in BackwardIteration with _reverse added to them if ForwardIteration is defined.

First I tried to do this:

module BackwardIteration
  def self.included(base)
    if base.ancestors.include?(ForwardIteration)
      instance_methods.each do |method|
        base.class_eval do
          alias_method "#{method}_reverse", method

          define_method method do
            unbound = ForwardIteration.instance_method(method)
            unbound.bind(self).call
          end
        end
      end
    end
  end
end

What it does is it looks for ForwardIteration, if its defined, it changes its own method name, by adding _reverse to it. Then it forwards the method to point to ForwardIteration. Ruby lets you get a method as an object with #instance_method(method), you’ll have to bind the method to the object, i.e. put the method in the lookup chain, then you can call it. But it didn’t work as I thought it would, so I decided to try another way.

Below I use method_missing is catch methods with _reverse suffix. For those methods, I temporarily redefine each to point to BackwardIteration#each and then call the method. Once done, I redefine each to point to ForwardIteration#each.

module BackwardIteration
  include Iteration

  def method_missing(method, *args, &blk)
    if method.to_s =~ /^(.*)_reverse/
      define_each
      result = send($1.to_sym)
      undef_each

      result
    end
  end

  def define_each
    class << self
      define_method :each do
        e = BackwardIteration.instance_method(:each)
        e.bind(self).call
      end
    end
  end

  def undef_each
    class << self
      define_method :each do
        super()
      end
    end
  end

  def each
    'Backward'
  end
end

class DoubleLinkedList
  include ForwardIteration
  include BackwardIteration
end

Clearly the constant defining and redefining of each isn’t a very good idea, but this was just a learning experience and I had a whole lot of fun trying to get it to work. Hopefully the specs pass for you and you learned a little bit more about the dynamic nature of Ruby.

rSpec best practices

Most of rSpec best practices are found here:

Here are some I learned while working on algorithms.

  1. Use symbolic constants whenever possible.

When you decide to change the name of something or rename your class, you don’t want to hunt down every occurance of it. This happens way more often in specs when using bdd.

let is great for that purpose.

let(:one_array) do
  ['one']
end
  1. Use subject

For the same reason above. You define it once, and use it through in your specs.

subject do
  list = described_class.new
  list << 'one'
  list
end

it 'returns size' do
  subject.size.should == 1
end
  1. Define subject explicityly.

rSpec is smart enough to detect it most of the time from describe, but you should define it anyway. Makes for easier reading.

  1. Use described_class

It refers to the argument passed in to describe. Again, same principle.

describe Algorithm::DoubleLinkedList do
  subject do
    described_class.new
  end
end
  1. Use context to organize specs related to a method

For the above project, I had three contexts based on the size of the list:

  • zero_nodes
  • one_node
  • multiple_nodes

But, I had to jump too much to find the related specs, so now they’re organized by methods instead. Now after adding/changing a method, I can just jump to the appropriate context and change the specs.

In the above project I probably reorganized the specs at least three times to make it easier to deal with the complexity. A good indicator about the health of your specs is how confident you feel changing something in the project. And also how much you dread having to work with the specs.

Customizing zshrc prompt to have git info in Ruby

I’ve been using Gary B’s dotfiles ever since I saw his PeepCode screencast, especially like his zshrc prompt, which looks like:

bin(0min|master?) $

bin - folder name 0 min - time since last commit master - name of current branch ! - there untracked files in the repo.

Its pretty helpful since I use terminal a lot, but one problem is the speed when switching to new repos, especially ones not opened for a long time. I didn’t mind it that much, especially since I found no good alternatives, but after looking at PeepCode’s latest blog post, and seeing they used Ruby, I decided to switch. PeepCode was nice enough to include the dotfiles, so I mostly just adapted that. My prompt now, looks something like:

bin(1hr|master!)☺

Its mostly the same except the time is more descriptive. The older one showed just minutes, which after 2 days since didn’t make sense anymore. This one is a little faster than the older one and in Ruby. It also shows if branch is behind or ahead of remote in terms of commits with ↑ and ↓. I’m not very proud of the code and it is unnecessarily slow at some points, but it works and has a smiley, which I also stole from PeepCode.

I pushed them to my dotfiles repo, if you’re interested. See zshrc and ruby script.

P.S. Both the prompts use color. They’re not visible since I just copied and pasted.

Moving away from Singleton pattern for Configuration

Whenever I wanted to stored some config options for a gem, I go with the singleton pattern. Usually this involves creating a class called Config and letting users configure that with configure block. Think I learned that from Mongoid, don’t remember.

require 'singleton'

module ShinyGem
  class Config
    attr_accessor :options

    include Singleton
  end

  class << self
    def config
      Config.instance
    end

    def configure
      yield config
      config
    end

    def options
      config.options
    end
  end
end

I can do this just once and forget about it. It is easily available anywhere ith Config.instance.options and I can modify it further down the line by opening the block again. The advantage of this approach is only one instance of Config can exist, which is also its disadvantage.

For gems like OpenAuth2, there could be technically be dozens of different api wrappers all using OpenAuth2 under the hood. Can you imagine what would happen then? Panic and chaos in the streets! I didn’t want to get mobbed down the line, so I decided to move away from singleton pattern.

The main requirement here is once config has been setup, it should be available everywhere internally, without me having to pass it each time to “. I decided to make Config a class and pass an instance to Client the first time around. So something like:

config = OpenAuth2::Config.new do |c|
  c.provider = :default
end

# Client
client = OpenAuth2::Client.new(:config => config)

Here’s the code for it.

module OpenAuth2
  class Client
    def initialize(hash={})
      @config = hash[:config]
    end
  end
end

From the client instance I wanted to be able to get access token from Google/Facebook.

client.access_token

Once again, its implementation. Note how config is based here to “ as well.

module OpenAuth2
  class Client
    def access_token
      @access_token ||= AccessToken.new(:config => config)
    end
  end
end

AccessToken stores config in an instance variable as well. The user has to set config up just once and the rest is done internally. And I can switch out the Config instance from Facebook to say Google or just create another Client instance. There could now be dozens of api wrappers in the same environment, all using OpenAuth2. I know, pipe dream indeed.

Easy plugin system for your gem in less than 100 lines

For my OpenAuth2 gem I wanted to let users customize the endpoints and various other options. At first I had a simple configure block, something like:

def configure
  yield self
  self
end

Here users could define options, even procs. Unfortunately every OAuth2 provider seem to have its own take on the implementation and needless to say this leads to lot of headaches. Which is why I decided to hide away the options of big ones like Facebook, Google etc. This way users can set provider = :facebook or provider = :google and not worry about what endpoints or the different options they all use. Also, I wanted to let users define their own providers. The end goal is this:

module OpenAuth2::Provider
  class Facegle < Base
    def options
      {:host => 'http://facegle.com'}
    end
  end
end

OpenAuth2::Config.new.provider = :facegle

The last line tells Config the name of the class to look for options under.

class Base
  attr_accessor :client_id

  def initialize
    @client_id = :default_client_id
    super
  end

  def set_defaults
    options.each do |key,value|
      instance_variable_set("@#{key}", value)
    end
  end
end

When you call provider=, it calls set_defaults which iterates through the hash and sets them to instance variables. That’s one part of it.

Another part is directing the method calls to the right object. const_provider does that job. Here’s rest of the code.

class Config
  Keys = [
    :client_id, 
  ]

  attr_accessor *Keys, :provider

  Keys.each do |key|
    define_method key do
      @provider.send(key)
    end
  end

  # ... some code removed

  def provider=(name)
    @provider = name
    const_provider.set_defaults
  end

  private

  def const_provider
    provider = @provider.to_s.capitalize
    string   = ['OpenAuth2', 'Provider', provider].join('::')
    @provider  = string.constantize.new
  end
end

This piece of code directs client_id to whatever provider is in @provider.

Keys.each do |key|
  define_method key do
    @provider.send(key)
  end
end

One problem here is we can’t override anything that is not set in provider class. What we can do is look for the method locally and if not found, send it to @provider.

Keys.each do |key|
  define_method key do
    instance_variable_get("@#{key}") || @provider.send(key)
  end
end

Here’s the rest of the code: https://gist.github.com/1656550. Amazing how much could be done with less than 100 lines of Ruby code.

Extract out a block into a proc

Say you’ve something like:

[1,3,5].map do |x|
    x + 1
  end
  # => [2,4,6]

What if you want to reuse the block? You can use a proc to do that.

even_to_odd = proc do |x|
    x + 1
  end

Now you can pass in even_to_odd to the method with &.

[1,3,5].map(&even_to_odd)
  # => [2,4,6]

What if you don’t have an array, but just a single object and you still want that behavior? Instead of passing even_to_odd to the block, you pass in that object to even_to_odd.

even_to_odd.call(1)
  # => 2

Now don’t go extracting out every block into a proc, only when you want to be able to reuse that block.

Getting rspec's "let" method to work in pry

Backstory: Was writing a google calendar api wrapper gem, since I just worked on a project that required it and also there are no existing gems and very few examples available. Needless to say that led to lot of hair pulling. Guess rubyists don’t care about google api. Anyway, I got a prototype up and running using bdd, and was switching over to pry for manual testing. I use lot of let methods in my specs and this one was no exception. One of them was:

let(:token) do
  "access_token"
end

I copied and posted that in pry and I got NameError: undefined local variable or method let' for main:Object. Well duh, let is a rspec method. Usually I’d copy and paste the string alone, but I had couple more let methos, so I figured I’d just define let in pry. It seemed simple enough, it takes a symbol as argument and sets that to the value in the block. Stupidly I did:

def let(name)
  name = yield
end

Yes, except name is a local variable with scope restricted to just that method body and therefore useless for my purpose. I then changed name to an instance variable and that worked.

def let(name)
  @name = yield
end

pry(main)> @name
=> "access_token"

Expect then I’d have to manually change them all and what’s the point in that. Local variable and instance variable is out of question, so the next option is to define them as a method instead.

def let(name, &block)
  define_method(name, &block)
end

That looked good, I turned the implicit block into an explicit one with &block and passed that along to define_method. Except I got NoMethodError: undefined method define_method' for main:Object. When pry starts, it starts under main and that has no define_method method. Say you’ve a Post class and you cd into that, then you can do define_method, since that is an object.

pry(main)> class Post
  pry(main)* end  

  pry(main)> cd Post

  [8] pry(Post):1> define_method(:hello) {'hi'}
  => #<Proc:0x007f84daa5f2b0@(pry):11 (lambda)>

  [9] pry(Post):1> Post.new.hello
  => "hi"

Just blindly I tried calling define_method on Object class and that almost worked.

def let(name, &block)
  self.class.define_method(name, &block)
end

=>NoMethodError: private method `define_method' called for Object:Class

So I guess it is a private method, no problem, we can use send instead.

def let(name, &block)
  self.class.send(:define_method, name, &block)
end

let(:token) do
  'access_token'
end

[10] pry(main)> token
=> "access_token"

And that’s how you define let to work in pry. This is a fairly simple example, but what was interesting to me was the trial and error process. Figured someone else would benefit from it, rather than just the answer.

EDIT:

Add the method to .pryrc in my root dir and its available every time pry starts now. I think pry has some special syntax for adding methods, but plain `def let` works fine, for now.

Faster specs with DummyTestingPattern

I’m developing a project for a client that involves a lot of libraries, worse, slow loading libraries that makes doing fast tdd a real pain. One way is to use something like spork, which I do use in Rails, but for a standalone ruby library, it seemed like a overkill, so I decided to use DummyTestingPattern. No, that’s not a real name and yes I just made it up. Btw, I learned this from watching Gary B’s screencasts at destroyallsoftware.com.

Originally the structure was something like this:

# lib/blade.rb
  require 'mongo_mapper'
  require 'stalker'

  class Blade
    def retire(name)
      Stalker.enqueue('retire', :name => name)
    end
  end

  # spec/blade_spec.rb
  require 'blade'

  describe do
    it do
      Stalker.should_receive('enqueue').with('retire', :name => 'leon')
      Blade.new.retire('leon')
    end
  end

If you’re wondering, I watched BladeRunner last night and hence the naming. Great movie, btw.

The problem with the above code is I don’t care about actual Stalker functionality. I just want to make sure Stalker receives the job, not that it actually processes it. Stalker could be an empty module and the above code would still work and we wouldn’t have to incur the cost for loading Stalker each time. So that’s what we’ll do.

# lib/blade.rb
  require 'mongo_mapper'
  require 'stalker'

  module Blade
  end


  # lib/blade/runner.rb
  module Blade
    class Runner
      def retire(name)
        Stalker.enqueue('retire', :name => name)
      end
    end
  end

  # spec/blade_spec.rb
  require 'blade/runner'

  # dummy
  module Stalker
    def self.enqueue(name, hash)
    end
  end

  describe do
    it do
      Stalker.should_receive('enqueue').with('retire', :name => 'leon')
      Blade::Runner.new.retire('leon')
    end
  end

Note we’re not loading the `blade.rb` file, instead we’re directly loading `runner.rb` and stubbing out Stalker. You can do the same for MongoMapper and save all the trips to the db, which btw, is another speed killer.

Question driven learning

I can never sit still enough to finish a class or read a book cover to cover. Yes this could be attributed to our ever decreasing attention span thanks to 100 word articles and top 10 lists or maybe its just my personality, destined to jump from subject to subject, never finish anything. For a while I did feel bad about this and rightly so. After 2 years of learning to program, all I had to show for myself are bunch of unfinished projects, plenty of half read books and a whole lot of unanswered questions.

Let me take a sidetrack and ask you this: when was the last time you were really involved in something? I mean truly experiencing “flow.” The only times I’ve felt that was when I was curious about something, especially the times I was trying to find the answer to some question or trying to solve something. (Mind you I’m talking about programming, but it applies generally as well.) As soon as I find the answer, I lose interest in it, hence the plethora of half baked projects which no longer interest me, mocking me with its halfedness (is that even a word?)

A couple weeks back after trying to understand my failures, I came up with “play based learning” system. The idea here is to not learn the things that you should you need to learn to become a better programmer (I wasn’t a CS major in college and I’ve always felt uneasy about it), but rather just play with it. Find something interesting and take it apart. I even had a flashcard with list of things that I wanted to learn/understand better. That worked great for a while. But soon I got bored.

Once again I tried to figure out what went wrong. I think I just aimed too big this time. I know I’m on the right track, because I felt happy when I was doing it. (There are just way too many crap indicators to tell if you’re being productive or not. The only thing that matters is are you happy when and after doing a particular task.) Also, there weren’t questions that I wanted answers to, but rather general topics.

So I decided to throw out the old flashcard and write a new one. This one had just three entries:

1. How does evented loop actually work internally?
2. What exactly happens when you send a GET, POST request?
3. What was status quo before REST?

As you notice they are all in form of questions. They are the things I was curious about at this moment and I could figure it out before end of today. I’ve decided to throw out the flashcard every night and come up with more questions everyday. This way the material will never get stale, i.e. I won’t lose interest in it.

At the end of the day the goal is to have knowledge of the right tools and the ability to use them properly to get the job done. What does it matter how we go about acquring that knowledge?

Gmail tips

Few tips on using Gmail:

Learn the shortcuts Press Shift + ? and you’ll see a lightbox open up with all the shortcuts, but I’ve found this to be very unusable. I recommend using this list here, where you can search.

Some of my most commonly used ones:

c – compose g+ i – Go to inbox g + a – Go to all message # – Delete a message / – Search n – Next message p – Previous message j/k – navigation – up/down e – archive enter – open message x – select conversation l – labels r – reply

There are plenty more, but those are the ones I use mostly.

Forward all email to one location I’ve three active Gmail accouts, personal, work and automated. First two are fairly obvious, the third is for signing up to websites I don’t want to give my personal info to. This way I don’t get spammed in my personal email. All three forward to a single account that I check. Haven’t visited the other two in probably weeks. Gmail also allows you to send email from that address, so you can always send email to yours from your work email.

Turn on Google Labs There are dozens of Labs and most are very useful.

Use labels + filters I think labels is my favorite Google labs feature. Currently I’ve ‘Reference’, ‘Waiting’, ‘Work’ and few named project labels on my list. By itself labels is pretty useful, if tedious, but combine that with filters, they become incredibly useful.

For example, all my work emails are tagged “work” automatically since one of my filters catches all email coming to my work email and tags it work. Same for all outgoing email from my work account, it automatically gets tagged. I also have couple work based filters, but I don’t really find them that useful.

Also all my outgoing email gets tagged “Waiting” indicating that I’m waiting for a response from someone.

Hope you found them useful.