On Distributed Development

 So, I have been doing distributed development at Bang the Table, for the past year. It’s a company with a very strong work from home culture. We even had a virtual birthday party for one of our directors over Skype video chat –

Birthday party at Bang the Table

Telecommuting / working remotely / working from home – these are various terms for an increasingly popular mode of working. In this model (of working) people use the amazing ability of the internet to communicate over vast distances instantly and modern collaborative software tools to effectively work together without actually being in the same physical room, office or even city.

There are some pretty powerful advantages to such an arrangement –

1. No need for any commuting, one can stay close to one’s family (a huge factor if you have children) and flexible working hours. It is a more relaxed way to work. I have been working in such an environment for about a year now, and with a new baby, the lack of commute and the freedom to pop downstairs and help out in case my wife needs an extra pair of hands has been awesome!
2. Another major advantage (at least for the company), is the reduced office infrastructure costs. In places where getting an office building thats a reasonably close to where your employees live is difficult, this can be a major boon. Also, not forcing your employees into having to commute is a major plus both for employee morale and the environment.
3. One has the opportunity to hire and work with people from a larger pool rather than being limited by a single geographical location, like a city or state or even a country. This can be very powerful advantage, sometimes finding all the right people you need in a single location can be a major problem.
4. An “advantage” that is often touted is that the geographical separation allows for better efficiency, since, theoretically, one could have employees coming online and starting their workday as others end theirs – making use of the entire 24 hours on the clock for work. I believe, at least for software development, this is a fallacy and attempts to leverage this so called “advantage” is a big reason for the failure of certain companies to successfully leverage this model of working – but more on this later…

In the software development field we have been working in such a distributed model for a long while now and I have seen this model evolve, influenced both by the advances in the technology and tools available for collaboration as well as the evolution of the software development methodology itself. I’d like to take a few lines to write about this evolution and distinguish between what I consider to be “distributed development” model and what I call “outsourced development”.

Image via Wikipedia

In the outsourced development model  – a software project is broken up into separate well defined phases, each phase having definite outcomes with concrete artifacts (usually in the form of documents). This means that (in theory) one can easily move around parts of a project between various teams who would then rely on the artifacts produced in the previous phase to execute the next phase of the project. Other than a short period between phases, where one team handed over the project to the other, there would be very little actual interaction between teams. This form of software development follows the waterfall methodology and while it is a good fit for outsourcing, it is, in practice, prone to problems, that, for the most part arise due to a lack of communication between the various stakeholders of the project and the members of the team(s).

What I call “distributed development” (there is a more generic definition on wikipedia that is not limited to software development) is a refinement on the outsourced development approach that attempts to leverage technological advances in internet connectivity and better communication tools to try and minimise the disconnect introduced by working remotely in software teams. Distributed development places emphasis on communication and interaction between all the members in the virtual team. The idea is to reduce the feeling that we are working in different physical spaces and that everyone is off working on some discrete task on their own. This kind of thinking is in-line with the agile philosophy which emphasizes individuals and interactions over processes and tools.

At Bang the Table, we get pretty close to what I consider to be an ideal distributed development model and I’ve come up with a few guidelines are a result of my experiences working here and from what I learned listening to a Hanselminutes podcast about working remotely –

1. One can use many tools and technologies to do your job. But regardless of what you use, patience and trust are the biggest success factors in this kind of working. I initially put this point lower down in the list but on consideration decided that this is the most important tip. Without trust the whole thing is an exercise in futility. One needs to learn to trust in the skills of the developer to do his or her bit and the patience to listen and accept that problems and delays might happen. The best way to build trust is by having open and rich channels of communication and meeting and interacting with everyone in person.
2. Reliable and fast internet is an absolute must and its not that easy to get – at least where I come from. Ask if the person you are working has a backup power supply and a good internet connection. In terms of power supply, the need for one depends a lot on the place the person is working from – if it’s a big city then usually a UPS that provides an hour of backup should be fine, unless there is chronic power problem. In smaller towns and such, they might need to have something more substantial like an inverter or a portable generator.  Modern laptops with their longer battery life and portability are a major help since they allow one to ignore smaller outages and coupled with the appropriate wireless or cellular data connection can even allow you to simply shift your base of operations to other places with power and internet.
3. This guideline, I think is going to be a bit controversial and is the reason I think that 24 hours work day idea I have mentioned is not really a good idea. If you have a team that spans countries and (in my case) continents, it is very important to have good overlap between your timezones – at least 4 hours of overlap is needed in my opinion, since this will allow you to actually collaborate rather than be limited to a short meeting at the beginning or end of your day. After all, developing software in a team is, in my experience, a collaborative activity, requiring lots of discussion (and argument)  and having your stakeholder at hand while you work through a problem is very useful. No amount of documentation and pretty pictures can substitute someone  being there, clarifying and providing context for you.
4. You need to have an open channel of communication during the time of overlap while you are working – this can be as simple as an open IRC (Internet Relay Chat) channel. We use Skype at work and I think the ability to call or video chat on demand make it far superior to IRC. Another option to explore is to use something like a Google+ Hangout while you are working – obviously this will mean the both you need to have reliable high bandwidth internet. Paid options for software tools are there as well like Microsoft Lync which may be a more integrated solution if you have a Microsoft based IT infrastructure with Microsoft Exchange (I got this from the podcast – haven’t used this myself since we don’t use MS Exchange). There are variety of communicators and tools out there to tryout – have a look at this link where Scott provides a deep dive into the communications tools available for remote workers.
5. Schedule regular meetings and get togethers in the real world. If this means that someone needs to travel somewhere from time to time, consider that the cost of doing distributed development. I would consider this an investment in the team’s productivity and efficiency. When everyone has met in person and worked together on issues, you build up mutual trust and respect. In my year of working at Bang the Table,  I think we have had get togethers at least once every quarter. Sometimes we just got together and spent a few days simply working together with no other agenda, sometimes it was a conference or a training.
6. Distributed development tends to work well with relatively flat and simple organizational structure. The idea is that everyone should feel equally invested and responsible for the software project. The challenge of course is that this particular setup does not scale very well and can be problematic for large organizations.
7. A good online project management tool is a must to ensure you are heading in the right direction. We are constantly trying out new tools for this part of our development. I have found JIRA to be good for support and task based work while, I am liking Trello for more open ended new development.
8. At work we use a Rackspace server as a development server and for testing. This is a machine on which we stage our commits from our local machines and have our testing done on. Having a machine to deploy your code on also forces you to test it in the right environment and also brings deployment considerations into your design and development schedule.

It’s been a fascinating experience for me – working from home and I think I have learnt a lot about the vagaries of distributed development. In fact, I have been experimenting with a more distributed approach to pair programming with a couple of friends of mine.

We used Vim in a GNU Screen session to set up a development environment that we could share between us and then while one person typed we used Skype to chat with each other. We followed the classic test driven mechanic – one person writes a test and the next person passes the test cycle. The experience was remarkably powerful, since I could see when the editor was being manipulated by the other person, the thought process going on as the code was being typed in. We built a small Python program – I came from that experience with a real appreciation of what can be done with some of the simple (and powerful) unix tools already out there.

I came across another tool Tmux which is similar to screen and is used a lot in the Mac when I was listening to this podcast on the Changelog. There is a very nice screencast showing how to use Tmux I found recently, you can checkout.  I also came across this awesome website called pair.io that spins up a server for you to use in your pair programming sessions.

Related articles

• Hanselman Blog – Remote Work
• Hanselminutes – The Plight of the Remote Worker with Pete Brown
• The Changelog – Tmux with Brian Hogan and Josh Clayton
• Pair Programming with Tmux – a screencast
• Bang the Table
• On working remotely
• Wemux – Tmux enhancement for remote pairing

Advertisement

A Ruby on Rails glossary.

So it’s been about a year since I started working on the Ruby language in the context of the Rails framework. It’s been awesome,  the combination of Ruby’s developer friendly language style and the incredible productivity of the Rails framework makes this a killer framework for web applications. Building the basic code is simply a matter of typing in some commands that generate all the scaffolding of code needed for a basic website. The rest is usually a matter of tweaking and adding to the generated scaffold.

Image by dullhunk via Flickr

Like every web technology the Ruby on Rails framework has its own set of idioms, that one needs to get familiar with and since I am new to Ruby, for me this includes some Ruby idioms as well. I saw a post today that lists a glossary of Rails terms and I thought I’d do my own. So here goes –

1. Application Servers – Thin, Mongrel, Passenger, Unicorn, Pow – These are application servers that run the Ruby web application and respond to the requests. They are generally integrated with a web-server like Apache or Nginx, though Mongrel is technically capable of running on its own as a web-server (though since it is not multi-threaded this configuration is not useful for anything except for very light loads or on a development machine).
2. Active Record – Active record is the ORM tool used in the Rails framework to abstract the database handling. It is an implementation of the active record architectural pattern described by Martin Fowler thus – “An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.” The Active Record code is available as a separate gem that is downloaded as a dependency to Rails.
3. Active Resource – A simple way to think of active resource is that it’s like active record but for REST based web-services. Active record provides a wrapper around REST actions providing a transparent proxy between a business entity and REST based web-services.
4. Active Support – This is a set of utility classes and standard library extensions that are useful in Rails application development. It includes classes for dealing with caching, dates, time zones, text, etc. Like active record and active resource, active support is available as a separate gem that is installed as a dependency of the Rails gem.
5. Bundler – Bundler manages the gem dependencies of a ruby application in a repeatable way across multiple machines. It automatically looks for and installs / updates all the gems needed by the application when it is deployed on a machine. It can do this is a repeatable way across multiple machines and deployments.
6. Capistrano – Capistrano is a tool for executing command on groups of servers over SSH. It is used mainly in deploying web applications and is capable recognizing different types of server environment (like staging and production, etc).
7. Capybara – This is an advanced version of the Webrat tool. It lets you do acceptance testing of applications using various drivers (Selenium and Rack are built-in) by simulating a real user hitting your web application.
8. CoffeeScript – Coffee-script is a little language that compiles into java-script. It has rapidly become very popular and has been included in Rails 3.1 as the default method of writing JavaScript code in a Rails application. I am personally a bit on the fence here but I like its terse syntax.
9. Cucumber – Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable DSL and serves as documentation, automated tests and development-aid – all rolled into one format. Cucumber along with RSpec are main tools used for the BDD methodology favored in many Rails projects.
10. DSL(Domain Specific Language) – A domain-specific language is a language dedicated to a particular domain or the representation of a particular problem or solution. DSL’s are more common than you think, some good examples in the Ruby being – Cucumber, Rake, Haml etc.
11. ERB (Embedded Ruby) – ERB is a simple and powerful templating system for Ruby. It is part of the Ruby core and it allows one to place ruby within other files (any text document).
12. FactoryGirl – This is a gem that can be used for creating object factories in an automated fashion. This is useful in testing or even for populating an application when deployed
13. Gem – At its most basic form, a Ruby gem is a package. It has the necessary files and information for being installed on the system. There is an online repository for Ruby gems at http://rubygems.org/ For me this is the biggest find and literally the “hidden gem” of my Ruby learning experience has been – Ruby Gems.  It’s like having an app-store for your development needs, where most of the best gems are freely available to you!  The number of awesome, useful gems actually available to ruby developers means that we have a huge advantage over other frameworks. A testament to the power of Ruby gems has been that Microsoft has started its own package management library – Nuget which, seems to be simply a straight lift of the Ruby Gems concept, except of course they have incorporated it into the Visual Studio as an extension. Nuget was released by the same team that created the Microsoft version of Rails – ASP.Net MVC so you can see the Rails influence…
14. Gherkin – Gherkin is a DSL  that emerged from the Cucumber project. It was designed to succinctly describe behavior in a form that can be acted upon by a tool like Cucumber. It is described as business readable DSL and can be used in any project that has need of such a tool. The lexer and parser for the language has been create and the library is available in several frameworks including Ruby, JVM, Javascript (Node.js) and as a .NET dll.
15. Git – Git is a free and open source, distributed version control system. It was written by Linus Torvalds to replace the old source control system for Linux and has become one of the tools of choice for a lot of Ruby projects. Git is typically used along with Github which is an online source control repository that leverages Git.
16. Haml – Haml is an HTML abstraction language. Haml functions as a replacement for inline page templating systems such as PHP, ASP, and ERB, the templating language used in most Ruby on Rails applications.
17. Heroku – Heroku is a cloud based platform as a service (PaaS) which was one of the first to have Ruby and Rails support. One could build a Rails application and directly deploy it on Heroku without having to set up a web-server. The platform has evolved to support multiple languages including Java, Scala, Node.js, Clojure and Python.
18. jQuery – jQuery has become the JavaScript framework of choice for web developers everywhere. It is a fast, framework that leverages CSS style selectors to allow developers to traverse, animate and handle events in the HTML DOM while abstracting browser specific quirks as far as possible. It has become so popular that it displaces Prototype as the default JavaScript framework used in Rails 3 applications.
19. Memcached – A distributed open source caching system, memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. It is a generic store and can be used in any situation where caching is needed and is available in multiple languages and technologies. It is used in a lot of Rails applications even though today, other object stores like Redis are getting popular.
20. Merb – Like Ruby on Rails, Merb is an MVC framework. But, it a lighter and more barebones version. I haven’t used Merb – I’d love it if someone could give a more apt description of what Merb is and how it compares to Rails and Sinatra.
21. Open classes / Monkey Patching – Open classes is a feature of the Ruby language where you can simply open and add methods to existing classes. Active support library uses this feature to add a lot of its helper functions on the Ruby core classes like string. A popular name for this particular practice is “Monkey Patching”.
22. Paperclip – Its a complete (and easy) file attachment library for active record. It provides validation features based on the properties of the file like size, type, etc. The files themselves are saved in the filesystem. It has some basic image manipulation features as well, like creating thumbnails etc. There is a dependency on the ImageMagick library (an awesome image manipulation library) in order to use Paperclip.
23. Prototype – Prototype is a JavaScript Framework that aims to ease development of dynamic web applications. It was originally the default JavaScript framework bundled with Rails.
24. Rack –  Rack is a gem that provides a simple interface between a web-server and the Ruby web framework. Rack is used in various Ruby web frameworks, like Rails, Sinatra and Merb.
25. Rake – Rake is the ruby version of make but much more developer friendly. It provides capabilities that are similar to make but it uses the Ruby syntax instead of XML files or makefile syntax to define tasks.
26. REST(Representational State Transfer) – REST is a set of principles that define how Web standards, such as HTTP and URIs, are supposed to be used. Well, actually REST as a concept is more abstract than that – its architectural style that can be applied to distributed systems. REST in the context of Rails, mainly is used to refer to the way Rails exploits REST concepts in the way that Rails web applications are structured.
27. RJS(Ruby JavaScript) – It is a ruby helper used to generate JavaScript that are used to decorate HTML pages. It is used mainly in AJAX calls, and in situations where you wanted to conditionally emit JavaScript. The general opinion though was that it sucked and was avoided in favor of the unobtrusive style of jQuery.
28. Routing – Routing is the aspect of the Rails web application that deals with the translation of a URL request to a specific action in the application. In Rails, it is the job of the router to recognize URLs and dispatch them to a controller’s action. It can also generate paths and URLs, avoiding the need for hardcoded strings in your views.
29. RSpec – RSpec is a Behaviour-Driven Development tool for Ruby programmers. RSpec helps you do the TDD part of that equation, focusing on the documentation and design aspects of TDD. Like Cucumber RSpec uses a DSL to simplify design and implementation tests.
30. Ruby Debug – Just like it says on the tin – it is a debugger for ruby applications. Like the other tools it is gem that can be installed and can run your ruby application in debug mode.
31. RVM(Ruby Version Manager) – This is an awesome tool from Wayne Seguin that is used to install, manage and work with multiple versions of the Ruby programming language on your machine. It even allows you to segregate your gems so you could work with multiple versions of the same gem for different applications.
32. SASS & SCSS – These are CSS abstraction languages (I believe SCSS is a reference to the version that supports CSS3). They are compiled down to CSS. They allow you use variables, modules and include other files. I am not very familiar with them and would welcome more information from readers.
33. Sinatra – From the website – “Sinatra is a DSL for quickly creating web applications in Ruby with minimal effort.” Like Merb, Sinatra is a minimalist web framework. Again, like Merb, I haven’t used Sinatra at all and would welcome some comments to flesh out this entry.
34. Test::Unit – This is the default testing library available with Ruby. It is from the xUnit family of testing frameworks and is a part of the core Ruby libraries. It is mostly not used in favor of more powerful BDD style tools like RSpec.
35. UJS(Unobtrusive Javascript)– “Unobtrusive JavaScript” is a general approach to the use of JavaScript in web pages. The general principles of this approach includes –
    1. Separation of behavior from the markup – The HTML in a document is meant to describe the structure of the document and should not contain programmatic elements in it. This is similar to the principle of separation of the structure from the presentation of the document when talking about the separation of CSS and HTML.
    2. Should not pollute the global namespace – UJS code should add as little as possible to the global object or global namespace of the environment in which it runs.
    3. Degrade gracefully – If the browser rendering the document does not have the capability needed the code should be able to degrade gracefully without breaking the rest of the application.
36. Vim – IMO Vim, the text editor that just celebrated its 20th anniversary in 2011 has experienced a resurgence of sorts with the rise of Ruby and Rails. In fact there is a large number of Vim plugins that have been created to make developers even more productive and happy on this text editor. Indeed after Textmate (the other favored tool for Rails developers) I think Vim has the biggest following among Rails developers and the fact that its available even on Linux and Windows only makes it cooler…
37. Web Servers – Apache, nginx, Lighthttpd , Thin – In the Rails world web servers options include Apache, nginx and lighty(lighthttpd) of which Apache is by far the most popular. I have included Thin in this list as well since it includes both a web-server and an application server. With advent of PaaS providers like Heroku, this particular aspect of web application development is becoming abstracted out, just as the database is getting abstracted on the other end.
38. Webrat – This is a headless browser that is used in web application testing. It lets you run in browser test without the associated overhead of actually instantiating a browser. This does not execute any JavaScript but is great for doing tests of simple websites without much JS interactions.
39. Will_Paginate – This is the most popular pagination library out there for Ruby on Rails. It alos supports other Ruby web frameworks like Sinatra and Merb. This gem has a wide variety of formatting options and can accept an active record object or even a simple array. It is quite configurable and can be easily extended to do AJAX pagination as well.

So, that’s all the terms that I can think of so far, I’ll try to add to this as and when I come across more terms. Nowadays, web development is more than a single framework, even one like Rails.  I have found the Rails community is more open and embracing of different ideas and frameworks – even to the extent of moving away from Rails and using lightweight frameworks like Sinatra. IMO that makes this glossary all the more relevant and fun.

As I have mentioned in the article I don’t profess to be an expert about or have even used all the tools and technologies listed above. I would love to get your comments and suggestions on what could be added.

And last but not the least – I wish you all an awesome 2012 – may all your efforts bear fruit this year.

Update (Jan 12 – 2012) : Since there has been some interest in this post, I have added a few more entries and provide more information where I can – stay tuned :-)

Google Android and the CLI

Today morning I was reading about Brad Adams – Going Google and ruminating about what he wanted to do there when several earlier articles I had read suddenly came together and I had an inspiration that I thought I’d blog.

It would by really cool if Brad and Tim Bray (who recently left Sun and joined Google Android) get together and implement the CLI on the Android OS.

Brad is one of the architects of the CLI and has been one of the main driving forces behind the development of the .NET framework and it’s adoption. He is currently looking at what he wants to do when he starts at Google. He mentioned that he thinks the cloud plus devices is one of the dominant trends of the future and I agree.

I think the Android OS is an important part of this future and currently I am frustrated that there is only Java support for developing in it. Tim shares my frustrations and he is looking at getting other languages supported in the Android OS.  He is looking at Ruby right now – it’s open source and a dynamic language and it makes sense . But I feel the CLI (which is an open ECMA specification) is a great fit for the Android OS because it can be used to as a basis to quickly support a lot of languages.  Besides as a .NET developer myself, I think that having the capability to develop Android applications in C# or IronPython is a far more palatable proposition than doing it in Java :-)

I saw this article by Miguel de Icaza where he puts out an idea to incorporate the CLI into the browser engine so that we could use languages other than Javascript in our client-side scripting (my take on that is here). It occurs to me that it should be similarly possible to bring it into the Android OS as well.  There are currently efforts to port Mono that could be used as a starting point.

So what say guys – can we get the .NET CLI in Google Android ?  Become the opposite of the Apple iPhone and embrace developers instead of driving them away :-)

Quietube – Readability for videos

I was thinking a few weeks back, while wading though yet another banner ad heavy website to get to a video, about asking the folks behind Readability for something similar for videos. I filed away the thought for later and soon forgot about it (I think this is why I am not a millionaire ;-)).  Anyway, while going through other blog posts on Readability the other day I came across this one from a fellow Readability enthusiast from Brazil – right at the bottom was the link to this gem.  Et´ Voila ! Solution found – this is why I love the internet :-)

Quietube is exactly what I was thinking about – its Readability for internet videos !
Even the installation is similar –

1. Go to the Quietube installation page.
2. Drag the bookmarklet on to your bookmarks toolbar or add it to your bookmarks.
3. When you are in a one of the big internet video websites (YouTube, BBC iPlayer, Viddler, and Vimeo are supported) – click on the bookmarklet and you can see the video without any of the comments and distracting fluff.
4. A link is also available at the bottom of the page enabling you to send this view of the video to a friend or save it for further viewing.

The tool is built by BookTwo and from the blog post it seems its a side project inspired by Readability. The tool is currently limited, in that it does not detect embedded videos in other websites (and the error message seems a bit cheeky ;-)) but they are working on improving this. Another problem is that there seems to be no mechanism to report issues to the group like Readability. On the whole this seem to be a useful tool for those who want to quietly watch a video without all the bells and whistles :-)

web 2.0, software, tool, bookmarklet,

Readability – Transform the way you read on the web !

I love reading – I read books, articles, magazines, circulars, just about anything I can lay my eyes on… Naturally, I am a voracious reader in the Internet as well – I have subscribed to a huge number of RSS feeds, and  have a constant stream of Twitter tweets on articles to read.

The problem, I face when reading on the internet is, eye fatigue – I have astigmatism and the varying fonts faces, sizes and colors is a strain – even to normal people with perfect sight ! Couple this with the increasing number of advertising on the top and on the side of articles and you have a situation where the actual content is increasingly obscured. Font sizes are getting smaller and smaller and content is getting squeezed more and more by advertising and marketing clutter.

Of late, I have been using a tool that I found on the web (I saw it on Twitter but I have unfortunately been unable to track who sent it to me :-( ) that has literally transformed my reading experience. It was created by a company called Arc90 and is a Bookmarklet which can dragged onto your browser bookmarks. When you click on it – it  reloads the current page loaded in your browser stripped all extraneous fluff.  Simply the content of the page is shown.

As a web-developer and general internet addict ;-), I have seen a lot of tools and gadgets and I like to think I have a good idea of what works – as soon as I saw this tool I knew this would be awesome. The concept is simple, powerful – elegant.  It is one of those things
that seem so obvious  – when you feel that way – you know you are seeing a work of genius.

What you need to do to get this gadget is  –

1. Go to the Readability installation page
2. Choose from the simple options given there how you want your content to be displayed – the options are simple and straightforward giving you a choice of reading style, font size and margin width (I chose eBook style, large size and narrow margin). A sample text is shown with the formatting you choose so you can play with the options.
3. Once you are satisfied, simply drag the big button marked “Readability” onto your bookmark toolbar (or you can right click the link and choose the bookmark option) and your done !
4. Next time you are on a web-page with a compelling article being obscured by garish ads and decorations, simply click on the Bookmarklet and the page reloads with the content formatted exactly as you want it.

When I first installed it they had fewer options and every once in awhile it would not work – but they have been going from strength to strength with new options and it’s rare to see it not work.

This tool has certainly transformed how I read on the web – I can’t recommend this tool enough!!  So what are you waiting for – go get it !! :-)

PS: It won the NY Times Pogy award for best idea of 2009

web 2.0, software, tool, bookmarklet, font