Monday, December 31, 2012

Debugging code in IntelliJ for OpenMRS Jetty

Yesterday, while working on OpenMRS I needed to debug some Java code. I could not easily find steps for debugging OpenMRS with IntelliJ on Google, so I decided to write down my findings.

I used IntelliJ's Remote JVM option to connect to OpenMRS Jetty server.

Step1: Configure Jetty to open a remote debugging port:

One needs to set the following in MAVEN_OPTS so that Jetty opens a remote debugging port for IntelliJ to connect to:



I updated the MAVEN_OPTS in my home directory's .zshrc file (since i use ZSH shell on Mac). You may need to set the MAVEN_OPTS variable appropriately for your environment (Windows/Linux/etc) so that it is correctly set, before starting Jetty on command line.

The value "51696" that I have mentioned above is the port where I want the debugger to run. The memory options have been set because Jetty gives an OutOfMemoryException with OpenMRS if the memory is insufficient (and by default, it seems to be insufficient - so you are better off setting the memory values too).

Now, start Jetty normally using "mvn jetty:run". You will notice the following line being printed by Jetty to indicate that debug port has been opened.

Listening for transport dt_socket at address: 51696

Step2: Configure IntelliJ to connect to the remote debugging port of Jetty:

In IntelliJ, go to "Run" menu, and select "Edit Configurations". Select the "Defaults" option on the left menu, and click on the "+" on the Right side, to create a new configuration. In that Select "Remote" option, and enter options as "localhost" and port as "51696" (same as the port number in step 1 above). Put "Name" as anything you like. I have set it as "OpenMRS Jetty".

OpenMRS Jetty Remote Debugger in IntelliJ IDE


That's it! Now you can debug using this "OpenMRS Jetty" configuration. On clicking "Debug" you should see the debugger connected to Jetty, and breakpoints will start working.

Note: If you want to perform Debugging on Eclipse, then basic process would be the same. Step1: No change. Step2: In Eclipse choose Remote JVM Debugging option (similar to IntelliJ).

There are other ways of debugging OpenMRS too, especially if you run Jetty from inside the IDE. But, for me, this method was simpler.


Sunday, December 16, 2012

Vagrant -- An interesting approach to setup development environments FAST!



If you have never heard of Vagrant, or are interested in understanding what is Vagrant, then this blog should be useful to you.

The Need for Vagrant

Recently, I was involved in organizing the GraceHopper Hackathon in Bangalore where we expected about 50+ women developers to come and contribute to Humanitarian software projects like OpenMRS, RapidFTR, and others. Since all participants would mostly be new to the projects, and we had a single day to get them going, and contribute -- we wanted to ensure that people spend as less a time on setup of their dev environment as possible.

I got introduced to Vagrant, because at the GraceHopper Hackathon, one of the projects - RapidFTR, provides people with the ability to setup their environment using a Vagrant box -- and that's when I realized -- this is interesting!

Setting up a development environment for many projects now-a-days isn't a matter of simply running a script or downloading a piece of software. On most projects, the average setup time to get all up and running is usually a day, and sometimes more. Most often, time is spent on setting environment variables, handling software version conflicts, or general configuration. Not to mention, that once you are done, doing this for the next project would be even more challenging.

Vagrant: The Solution

Vagrant provides an elegant solution to this whole mess.

In simple terms, Vagrant allows you to provides a "box" which has everything pre-installed and setup for you -- and all you do is "start" the box on your machine and you are set. This box is essentially a Virtual Box VM, which is running in invisible mode (background mode) on your machine, and has the whole development environment for your software setup inside it. But, instead of coding inside the Vagrant VM, you code on your laptop -- and do your builds, compilations and server startups, etc inside the Vagrant VM.

Vagrant essentially shares a folder on the "box" which is also available on your laptop, and this is the folder that contains the code of your project. Any modifications to that folder by your laptop (host), or the box (guest), will be reflected to each other.

Basically this means that your laptop owns the code, so you use your favorite powerful IDE and laptop environment to modify code, and you use the vagrant environment to "run" the code.

The advantages you get when using a Vagrant box are:

  1. You don't install stuff on your machine, and mess your machine up with all the weird libraries and settings. All you have on your machine is the "code". Rest is inside the VM.
  2. You have exactly the same development environment as everyone else in your team.
  3. The development environment is created once (Vagrant box), and used everywhere on all Operating Systems. This is because the Box is essentially a VM, which is running on top of Virtual Box. And Virtual Box VMs can be run on any host operating system.


Is it different from using a VM?

Some people ask the question -- why Vagrant Box? Why don't I just setup a Virtual Box VM and everyone can work on that VM. That is also the same -- isn't it?

Well the answer is -- "Almost".

If you create a VM, and then ask people to work on that VM on their machines -- you are in for trouble most of the time! In most cases a VM on your machine is slow. It is especially painful when you have to do things like editing, coding, etc. Plus that also means you have to learn the shortcuts of the VM to make you productive on that VM.

When using a Vagrant box, you aren't working in the VM. You are working on your own laptop, and only connecting to the VM in "ssh" (command-line) mode. In effect, you are actually just firing a few commands to VM -- like build, start server, etc and the rest of the work is being done on your own machine. For you, the Vagrant box is just like a "server" to which you connect and tell what to do.


A Vagrant VM is usually viewed as a light-weight command-line only VM which usually runs a Linux OS (Lucid32 by default, but you can use Ubuntu or anything else).


Vagrant also allows you to configure port forwarding, so that when you type a URL in your browser like http://localhost:3000, you get connected to your server application running inside vagrant -- just as-if the server was running directly on your laptop.

Steps to setup a Vagrant Box on your machine:

When you have a Vagrant Box based Development environment setup (like RapidFTR did), here is all you do to get yourself setup:


  1. Install Virtual Box for your laptop. If you have a Mac, then you install Virtual Box for Mac (and so on). This will act as the environment inside which Vagrant box will run.
  2. Install Vagrant for your laptop. Once you install vagrant, you will see that on command line "vagrant" command will become available.
  3. You download the code into a folder on your machine (say from Github or your network code repository).
  4. You "add" a vagrant box to you laptop using the "vagrant box add" command. This command will need the path to your vagrant box VM. Alternatively, the box can be downloaded and added automatically from the internet as part of the vagrant up command (in next step).
  5. You then do a "vagrant up" to start your box. This vagrant "up" command is done from your code directory, and the code directory will contain a "Vagrant" configuration file with configuration details on the Vagrant box to run. If the box isn't found on your local machine, vagrant will attempt to download the box from the URL provided in the vagrant configuration file. (This means you didn't use the box add command to add the box locally - as was mentioned in the previous step).
  6. Once the vagrant box is up and running, then you ssh into the Vagrant box using the "vagrant ssh" command (or putty on windows) -- to build/start/stop your project application.


That's it. Now you are free to modify code on your local machine, and see it working via the vagrant box VM environment.

Chef your Vagrant

Vagrant also allows you to run chef scripts as part of your vagrant up command. The chef scripts will perform the installation and setup of software inside your Vagrant box. This way -- the power of infrastructure automation and provisioning provided by chef, can be combined with the power of a decoupled development environment provided by Vagrant.

Monday, August 20, 2012

Pretty Fractal Trees using HTML-5 Canvas & Random Numbers

Last week I sat down to see how I can generate a fractal tree on a web page. It turned out to be a very interesting exercise using HTML5 Canvas and simple Math trigonometric functions. Using random numbers for changing the length of the tree branch, and the angle of branch split -- I was quite successful in generating very realistic looking trees!

Each time you render the HTML, the trees look quite different. I can see how Game creators can create interesting looking forests for "heroes" to travel in using such techniques.

HTML 5 Canvas with a Fractal Tree





















Here is the code (also posted on github):



Sunday, July 29, 2012

Recommended Readings for Lateral Dev Hires at ThoughtWorks -- An Unofficial Opinion

All thoughts mentioned in this post are in no way endorsed or validated by ThoughtWorks. These are solely my opinions (and a few co-workers). You are still the best judge of your capabilities, and needs. 

Quite often experienced folks joining ThoughtWorks have a common set of questions: What should I read to prepare myself for ThoughtWorks? What kind of technologies does ThoughtWorks work on? Will I go on a Ruby or Java or Mobile project?

I doubt there is any good and precise answer to these questions. But, there certainly are some practices that I feel our in the DNA of ThoughtWorks. Agile & Extreme Programming practices drive most of this. Unfortunately, Agile means totally different things to different people outside ThoughtWorks. And, quite often, nothing will prepare you for the radical changes that are going to come to your way of working at ThoughtWorks. 

So, I thought, why not try come up with some recommended readings for Dev Lateral hires (experience range of 3 - 10 years), to introduce them to some of our development practices, coding styles, unit testing beliefs, etc. I asked a few of my colleagues - namely Unmesh, Chirag, Sunit and Aman on what they felt should be the minimum recommended reading list for Lateral ThoughtWorkers. This is what we all came up with. 

I may have used my discretion to sanitize the final list, therefore if you don't like the list -- blame me! :) 

Highly Recommended Book Reading

Refactoring, Improving the Design of Existing Code (Martin Fowler)
This book introduces you to refactoring, code smells and patterns to help you refactor your code into something much more readable, maintainable and extensible. If you are new to refactoring, this will be an eye-opener. Note that examples in this book are targeted towards Java developers, but core concepts are valid in almost any serious programming language. Couple this reading with IntelliJ Refactoring shortcuts to realize how easy it is to perform useful re-factorings by Java programmers. At ThoughtWorks, the most popular Java IDE is IntelliJ. Its refactoring capabilities are beautiful.

Test Driven Development (TDD) By Example (Kent Beck)
TDD - A fundamental practice that has been very highly suggested by proponents of Extreme Programming (XP). This book (again in Java) helps developers in understanding how to think Test-First, and how unit testing affects the design and development of your code. It is a simple, tiny and very nicely written book that gets-to-the-point. Warning: You are not supposed to "read" the book, but follow it as-if Kent Beck himself is pairing with you. If you do not code along with this book, you may as well not read this book. If you are joining ThoughtWorks, you will benefit tremendously by understanding TDD.

Extreme Programming Explained (Kent Beck)
An excellent introduction to the values and practices of Extreme Programming, and our preferred flavour of Agile. It will give you an insight into how we work, and why we do things the way we do. A must read -- whether you are a developer, QA, BA, PM or any technically connected person who is joining ThoughtWorks. The book is deceptively simple. Understanding this book will explain to you the meaning of "being" Agile, versus "doing" Agile.

Clean Code: A Handbook of Agile Software Craftsmanship (Robert C Martin a.k.a Uncle Bob)
Coding is our bread and butter. Well, we prefer it nice and clean! Clean Code is taken very seriously at ThoughtWorks, and the techniques, tips and recommendations mentioned in this book are invaluable. This book is packed with code examples (in Java), and isn't something you read over a plane journey, but something you couple with an IDE, coffee and a leave-me-alone-to-do-as-I-please environment.

Other Recommended Book Reading

Even though I have mentioned the highly recommended ones, there are still some other books, that I just had to let people know they should consider reading. Each of these books are as much a gem as the ones above, except that if you think like an economist, who wishes to maximise his returns at ThoughtWorks, then maybe you will start with the list above, and later digest the ones in this section.

The Pragmatic Programmer: From Journeyman to Master (Andrew Hunt, David Thomas)
A collection of very practical & solid advice on many many aspects of Software Development. Reading this book will make you feel you are being handed out information and knowledge developed by the authors through years of hard labour, failures, learnings and successes. Quite a collection!

Agile Software Development, Principles, Patterns and Practices (Robert C Martin a.k.a Uncle Bob)
Uncle Bob introduces Agile practices, and shows how teams need to design and develop code so that Agility can be attained. Even if you didn't work in a team that followed the Agile methodology, you definitely need to know these design principles. Most of them are fundamental to how good software should be created.

Effective Java [2nd Edition] (Joshua Bloch)
If you are a Java programmer, then you have to read this book. This book is filled with pearls of wisdom on how to use the Java programming language way more effectively than you thought possible. I still pick it up this book time to time, to refresh my Java knowledge. Please read it.

Continuous Delivery: Reliable Software Releases Through Build, Test and Deployment Automation (Jez Humble, David Farley)
How does one make software delivery and releases --  painless and a non-event. What processes, practices and tools should teams follow so that one-click delivery is possible. Turns out, it isn't so easy - but then it isn't rocket science either. All ThoughtWorks projects aim towards providing Continuous Delivery, and this book will give an insight into our methods of achieving those.

Implementation Patterns (Kent Beck)
Kent Beck takes the sum total of all his experience in programming, Smalltalk & Java, and categorizes it into a set of Values, Principles and Patterns (with lots of good examples) for us to realize. This book will make you a much better programmer -- one that others will be able to appreciate.

Recommended Reference Links on the Internet

There is a wealth of information on the Internet, undoubtedly. Although I do feel that books seem to deliver a more "complete" experience and understanding -- since they take you through a complete story line with a full narrative and total context. The articles on the internet seem to provide a window to possibilities - and there are few folks who publish the "complete picture". Although, the situation has been remarkably improving in the last two or three years. One has to be very judicious and thorough when reading / learning stuff on the internet. As a reference guide - the internet is priceless. There are always exceptions to the rule. Always.

Some recommended reference links on the internet that will help the new ThoughtWorker: 

I would use these links as a starting point, or a seed, to lead me into further research, based on what I find interesting and parts where I feel I need a better understanding.

Welcome to ThoughtWorks!

Sunday, July 22, 2012

POKA YOKE - Applying Mistake Proofing to Software


For years, automobile companies have utilized "Mistake Proofing" as a technique for ensuring high quality, high speed manufacturing -- especially in cases of mass scale production. This is also known as Poka-Yoke (in Japanese) and was adopted and formalized as part of the Toyota Production System. This blog attempts to raise awareness (with examples) for the need of Poka Yoke in Software Design and within the Software Development process.

What is Poka Yoke?


The essential idea of Poka Yoke is to Mistake Proof the manufacturing process so that workers in a plant cannot make mistakes easily, or if a mistake is made, it is detected and corrected quickly. 

There are two types of Poka Yokes:

Control Poka Yoke: A Control Poka Yoke is one where the process is designed in such a manner that one cannot make a mistake.

For instance, a car manufacturer might want to put special heat resistant bolts in the engine assembly. For this, the bolts can be made to be of a certain dimension such that it only fits the engine assembly, and nowhere else. Therefore, one cannot use the special bolts in any place except the engine assembly, and one can also not inadvertently put the wrong (non-heat resistant) bolt in the engine assembly

This is a form of control poka yoke, where one cannot go wrong, or make a mistake.

Another interesting example of a control poka yoke are the Gas connections in an Emergency Room at the hospital. To ensure that someone doesn’t connect the wrong gas pipes to the wrong outlets, the Pin configuration of the connections are designed to be unique for each Gas outlet (a standard called the Pin Index Safety System).

Mistake Proofing via different Pin Configurations
Warning Poka Yoke: A warning Poka Yoke is one where the moment someone makes a mistake, the person is quickly notified of the mistake, so that corrective action can be taken.

An example of a Warning Poka yoke is the Car Seatbelt Warning indicator. If one forgets to put on the Seat Belt, then it will beep to warn you of the fact that you forgot to put it on.

Check out the following catalog of Poka Yoke examples from John Grout’s website:

The Need For Poka Yoke in Software

There are very compelling reasons to use Poka Yokes in Software Design for the benefit of end users, and also in Software Development Teams for creating high quality software in shorter time-frames.  

In Software Design

Beautiful software is designed in a manner that it is intuitive and transparent to the user. It meets the need in a fluid manner, assisting the user to do just what he/she would like to do with it, without making the user jump through hoops and millions of clicks. It should be like an extension of the body, which one uses sub-consciously. 

To create such Beautiful software, it is imperative that it is designed with the user behavior, psychology and needs at the core of all thought. When the user naturally moves through the software with ease, we can perceive it as a form of Control Poke Yoke, that ensures that one doesn’t make costly mistakes while using the software.   For instance, a simple control Poka Yoke feature is the “Auto Save” feature of Gmail, which ensures that a user’s email is auto-saved every few minutes, so that loss of the internet connection doesn’t make the user lose his/her data. This is an indication of detailed thought gone into understanding the users behavior and needs.

We will look at few more examples of Poka Yoke in software design in the following sections.

In Software Development Teams

The need for Poka Yoke in Software Development teams is higher than ever before. It is quite well established in the industry that the cost of fixing a defect amplifies 10 to 100 times depending on how far the defect has gone unnoticed in the development process. Couple that with the need for integrating with tens of systems nowadays, and the problem gets compounded manifold.  Delegating software development to large number of mediocre or inexperienced developers, isn’t helping the cause.  Introducing a poka yoke, that catches defects early, makes Discovery and Diagnosis of a defect ridiculously easy. Of course, to design a Poka Yoke, one needs to be aware of possible mistakes that could potentially occur -- and that comes from feedback, retrospectives and burnt fingers across a project or multiple projects.

The point of a Poka Yoke is: Making mistakes is OK, just don’t make the same mistake again and again.

Quite frequently on large projects, or long running projects, or on projects with multiple and remote  teams -- communication becomes a severe issue. In such projects it is very difficult to keep all folks “on the same page”. One finds people sending emails in BOLD, or writing and maintaining huge Wiki pages with FAQ and ToDos on how to perform a certain task. Unfortunately, no one has the time to read or maintain these documents, and errors slip through the cracks.

For instance, on one of my Travel projects, to perform test bookings, we had to use live GDS inventory. So we were given instructions on attributes of a test booking. These went something like: Make bookings at least 6 months in advance, and don’t book during Christmas, or New Year season, don’t book ABC and XYZ airlines, cancel the ticket within 20 minutes of doing the bookings, etc.

Now, even though people followed these rules, quite a few folks would forget to follow at least one or more of these rules. As a result, one time a large airline sent our Travel booking agency a huge bill for messing up with their ticket prices because apparently some test program did a huge number of bookings & cancellations for their flights -- and caused their ticket prices to go up, resulting in shooing away of “real” customers. Now, this can be easily avoided by putting a Poka yoke that would reject any bookings sent via our service that didn’t meet the criteria. And -- that’s really quite easy in software!

In today’s world, the array of tools and techniques available to us are immense. We must use these tools smartly, so that we concentrate on more pressing issues, and let poka yokes, automation scripts and smart checks catch common, and well understood project issues.

Instead of sending multiple BOLD font emails, create a Poka Yoke -- to ensure things don’t break again, and mistakes and errors don’t slip through.

Qualities of a Good Poka Yoke

A good Poka Yoke must meet the following qualities:

  • Early: A good poka yoke must be early in the process, so that it can provide quick feedback -- and help in detecting mistakes the moment they occur.
  • Precise: It should be precise, so that it is easy to diagnose and identify what mistake occurred.
  • Simple: The poka yoke should be simple -- to develop and maintain. This is quite important since one doesn’t want to spend time and effort in maintaining poka yokes, and complex poka yokes will have a fairly high chance of becoming erroneous. Having a buggy poka yoke is worse than having no poka yoke at all.
  • Light: The poka yoke needs to be unobtrusive and transparent. If a poka yoke itself becomes an overhead to the process, then it will drive the developers/users crazy, and they will find ingenious ways to avoid it all together. For instance, think about how a developer will feel if he/she has to run a 70 minute pre-commit script before each and every check-in!

Examples of Poka Yoke in Software Design (for End Users)

  1. Gmail Attachment Check: If one uses the words "I have attached", but does not attach any document to the email, Gmail will give a warning saying you used the words I have attached, but didn't really attach any document. Are you sure this isn't a mistake?
  2. Wrong Bob/ Missed Bob Check in Gmail: Depending on which set of people you commonly email together, Gmail will warn you (or offer suggestions) to include the folks you may have missed. This will ensure you don't mistakingly leave out someone, or add someone you didn't want to email.
  3. Password Strength Indicators: When you sign up for an account on most websites, you are nowadays displayed a password strength indicator - which gives you feedback on the quality and strength of your chosen password. You may think that "Passw0rd" is an awesome password, but that's a mistake you are making! Websites have data collected over millions of users which can tell them things like what are common and easy to break passwords. They utilize this data, along with sophisticated software to create Password Strength indicators so that naive users don't mistakingly set simple, easy to guess passwords.
  4. Cmd-Q Warning In Chrome: Quite often Mac users make a mistake of pressing Cmd+Q to close a TAB instead of Cmd+W. Google Chrome on Mac can warn users when they press Cmd+Q, to help them from inadvertently closing all their windows.
  5. Spelling suggestions in Google suggest: Google will auto-suggest spelling corrections depending on what you might be searching. Helps users from making inadvertent mistakes.
  6. Undo Feature is nowadays present in most production quality successful software. Undo as a way of quickly "fixing" mistakes has become second-nature for most software users, that without this feature, most of us operate with a Save-paranoia. Gmail provided an Undo Send feature to provide a safety net for times when someone clicks Send by mistake.
  7. Double Entry Box: Most websites & software where one needs to enter a critical bank account number, or a password create option, users will notice that they are asked to enter the same value twice (with paste option disabled). This is to ensure people haven't made a mistake while entering the value, and that both boxes hold the same value.

Examples of Poke Yoke in Software Development

  1. Unit Tests + Pre-commit + CI + Build Radiators: Unit Tests are one of the strongest and most effective means of Mistake Proofing software development. They are precise and early in the development stage. They catch regression mistakes during refactoring, or bug fixing immediately. Unit Tests, coupled with a Continuous Integration Server (like Go, Jenkins, etc), a pre-commit script, and a Build Radiator are an excellent poka yoke mechanism to inform developers quickly about a mistake having occurred.
  2. IDEs and Compilers: IDE's indicate issues in code while you code. They will catch mistakes around incorrect type casting, generics, exception handling and provide you with possible fix options. Compilers will act as strict control mechanisms and will disallow any code that doesn't meet the syntax requirements (especially in case of Statically typed languages).
  3. Architectural Controls [ACs]: Some architectural poka yoke control examples in software:
    1. [AC] Hiding HttpSession: In some web projects folks would create a custom framework for development teams where a potential-to-misuse object like an HttpSession would be made unavailable. Instead, a custom "Session" object would be available which only has specific methods exposed like the getSessionID( ) method, so that application code could use the Session for its original purpose -- which was to identify the user's session, and not to act as a bag of data for passing around to pages and methods. It also ensures that there isn't much overhead in keeping session data synchronized across multiple machines in a cluster - since the ability of developers to stuff anything they like in the session object has been taken away. This will force the developers to look for data structures appropriate to their need for storing user specific application data.  
    2. [AC] Context aware injections: Custom frameworks can also be written to explicitly ensure that people don't perform an operation which is incorrect / invalid in the current application context. For instance, we would not like to perform updates during a GET Request (remember REST?). In such cases, the framework can inject appropriate implementations which do not support update method calls, so that a developer doesn't make a modifying call in the GET context.
    3. [AC] Running Under Least Privilege (RUPL): The paradigm of Running Under Least Privilege is popular at many levels in Software Development. Operating Systems now create processes which operate at Lowest Level of Privilege, so that processes cannot inadvertently wipe out a memory area which doesn't belong to them, or they do not mistakingly come under a virus attack and wipe out system secure space. Database connections at application level are given only read/update/delete row access on tables, rather than dba / admin access to ensure that the application cannot overwrite/drop tables by mistake. RUPL ensures that a process or a program is given only as much control as they need to avoid costly screw-ups.
    4. [AC] Circuit Breaker: Michael T. Nygard introduces this concept as a programming pattern in his masterpiece book ReleaseIt! The essential idea is similar to an electric current circuit breaker which trips open whenever the current load is high. The Circuit Breaker trips open whenever it detects that the call to an external web-service, database, etc took too long to respond, or timed-out. Once a circuit breaker is in Open State, it will immediately return an exception to any caller that attempts to connect to the external service. This has the advantage of protecting the external service from an overload of requests while it is attempting to recover, and also prevents the caller application threads from getting blocked. For more details read the concept and a sample Java AOP based implementation here. Circuit breaker acts as a poka yoke by preventing both the callee and the caller from unknowingly blowing up in case of a failure. 
    5. [AC] Types with Immutability: Passing a type (like a Money object), instead of passing primitives (like numbers) gives developers control over what can happen to the data as its passed along the various software layers, and what kind of operations can be performed on the data. Adding immutability to these types also ensures that an intermediate layer cannot inadvertently modify an internal data element while it is passing the type around.
  4. Password Log Check: To ensure that someone has not mistakingly logged sensitive user information like Credit Card Number, CVV or user password to a log file one can write a script to scan log files on the automation test machines. In automation tests, teams usually use only a small set of username/password/accounts/test-data, and hence checking for these accounts in log files, can help uncover if someone has mistakingly left a debug statement in code that prints sensitive data to the log. This would be a Warning Poka Yoke. A control poka yoke would be one where somehow the log API would disallow logging any variable/string which contained the words "credit card number" or "password" or "pwd", etc.
  5. Localization Test for Menu-Keyboard Shortcuts: Quite a few desktop softwares have menu options with Keyboard Shortcut keys (accelerators) which are shown underlined. The idea being that when someone presses "Alt+Character", then that menu option will get pressed. There are 2 requirements of these shortcut characters: First, the character must be present in the menu item, and Second, it must be unique in the menu. This is all fine when its designed, but quite often the keyboard shortcuts get messed up when they go through a localization process for other languages. Quite often translators will assign shortcuts which will clash with other menu options, since they don't fully understand which all menu options are shown together (especially in the case where menu changes dynamically based on roles). The solution in such cases is to write a poka yoke script that will check all menu options in a particular language to ensure there isn't a clash. This kind of bug detection is very cumbersome manually.
  6. Localization Message Bundle Checks: Localization in Java for instance is done through message bundlers (or properties file). Quite often translated files can have errors like translation of a specific line missed, or key misspelled, or key missing, etc. Such errors cannot be quickly caught through language testing. Instead a simple script that compares English Locale properties file against each Language property file can easily catch most of these errors through simple comparisons, and save precious heart aches later. These scripts can be run as part of the localization check-in to catch translation file errors immediately. Use of good translation tools can also eliminate these problems quite effectively, acting therefore as control poka yokes.
  7. Hiring the Right People: A very effective poka yoke to mistake proof your software. Something only few companies like ThoughtWorks get right :) 

Pragmatic Mistake Proofing

One needs to be pragmatic about mistake proofing to ensure it is effective, and doesn't irritate the hell out of its users. The guideline to deciding whether a Poka Yoke is needed is to look at feedback from the field. If an issue occurs quite frequently, and people seem to stumble upon it too often, then its most likely a candidate to apply a Poka Yoke. Also, if the blast radius of an issue going out in the field is very high, like logging passwords in clear text, then too, it makes sense to mistake proof the issue.


Conclusion

Poka Yoke techniques have been in software for sometime. The point to keep in mind, is to be aware of the fact that whenever there is something to warn to people -- instead of writing long emails, and bold font Wiki documents, one should pause and introspect. Ask the question: Can I re-design the system/ component/process such that mistakes cannot be made. Or, can I put in a check in software, such that if mistakes occur, then they can be caught quickly. If yes, then consider the Poka Yoke. 

Note: The idea of Poka Yoke in Software was presented by Dhaval Doshi and me in ThoughtWorks Bangalore xConf (July 2012) Slideshare Link. Based on positive reviews at the xConf we decided to write a blog on this topic since we could not find good enough resources on the Internet that spoke of Poka Yoke in Software. Thanks to fellow ThoughtWorkers Unmesh Joshi and Chirag Doshi for providing feedback and few suggestions on this topic.  

Sunday, April 29, 2012

Web Vulnerabilities - Phishing, Cookies, XSS and CSRF

Last week I gave a presentation on "Web Application Vulnerabilities" as part of our weekly Dev meetup at ThoughtWorks. The presentation was aimed at covering some vulnerabilities and risks that plague Web based applications, and to make folks aware of risks and possible mitigation options. In specific, topics covered were Phishing, Social Engineering, Cookies and Cross Site Request Forgery (CSRF or XSRF). As a continuation to the presentation, I felt that writing a blog on it would crystallize the information I had collected and make it available to a larger audience.

Note: As per conventions, Malice = Malicious user. Alice = Unsuspecting user.

Phishing: Phishing is a form of Social Engineering attack, where-in Malice tries to trick people into handing over their sensitive data and passwords. The premise is that its much easier to trick a person into giving their information, than attempting to hack a website. So, for instance -- if on an IM chat, someone shares a link to a Photo sharing web site, which directs a user to a login screen that looks like Google Accounts login page -- with a message "Please enter your Google credentials" to login --- and Alice enters her Google password, she would have exposed her sensitive data to someone else! This is called a Phishing attack. The internet is strewn with sites explaining how to create such fake websites. As an example check out this Link. The author of this page, has listed steps on how to create fake pages for a website, and then at the end has attached a zip containing pre-created fake pages of many popular websites for download!

So, as Web Application developers, it is imperative that we are aware of such risks, and can help our customers prevent Phishing vulnerabilities on their sites -- because information on hacking, phishing and other attacks is now easily available to any malicious attacker.

To address the problems of Phishing, one can develop solutions along two lines:
  1. Prevention: Solutions to help users of a website identify a Phishing attack, or to make it difficult for an attacker to create a Phishing website.
  2. Protection: Solutions and strategies around Protecting user data, money, or other sensitive information, even if user information like login credentials have been compromised via a Phishing attack.
Here are some examples of how we can achieve the two P's mentioned above:
  1. SSL/Digital Certificates: Whenever a user visits an SSL secured website (i.e. over an HTTPs connection) on their browser, some browsers will display the name of the organization that owns the website by inspecting the digital certificate presented by the site. Also, if the certificate has issues, is expired, etc, then the browser pops up a warning. If a user checks the PAD lock icon to ensure the authenticity of the website, then the user can be sure he is not submitting his data to some malicious user. All websites of any standing will use SSL/Digital Certificates nowadays to help users identify their websites. This has an additional advantage that all communication between the browser and the user is encrypted -- to avoid a man-in-the-middle attack. So, this strategy is a "Prevent" strategy -- though, unfortunately not sufficient enough. Most successful Phishing attacks have targeted sites that are SSL enabled, and most naive (and sometimes even smart) users do not check to see if the page is SSL enabled or not - leave alone verifying certificate owner, etc. Furthermore, getting a Signed Certificate for a website isn't complicated or expensive, and hence even though the page is served over an encrypted connection, its "destination" could very easily be malicious. Hence, SSL based security is good, but no longer sufficient to prevent Phishing attacks.
  2. Personal Image or Message: The idea here is to display a user-selected image or message on sensitive transaction pages. For instance, a Bank may display a user-chosen image (or from a set of pre-selected images), on the screen where user enters his transaction password or his profile data. This way, for a Phishing site, it would be difficult to have a large attack vector -- since it needs access to each user's personal image or message. Quite a few banks, including "Verified by Visa" utilize this technique -- and is mandatory for all online credit card transactions in India.
  3. RSA or Two-Step Authentication: Most banks and websites which handle sensitive data usually resort to a Two Step Authentication mechanism, where one of the passwords is randomly generated and unique for every login. It is also known as "OTP" or "One Time Password". The idea here is to ensure that even if a user unknowingly enters his credentials on a Phishing website -- since the secondary password has been generated using RSA or some random password generator, it is not valid for the next login request. Hence, the "password" is only valid for one-time use, and is useless in the hands of an attacker. This is a very popular and effective strategy to defend loss of critical user data. Its downside is user inconvenience, since the users need to carry or have access to a Random password generating device or software. With the advent of smart phones, this has become less of an issue. To avoid user inconvenience, some websites -- like Facebook and ICICI, will request you to enter a OTP only if they detect a change in user's co-ordinates -- like change of MAC address (due to the user logging in from a different machine), geo location (different place), etc.
  4. Log Referral Websites: Every HTTP request comes with an attribute called the "Referer" which contains the name of the website that originated the request. For instance, if on attacker.com, there is a link to example.com, then the request received by example.com whenever Alice clicks on the link on attacker.com will contain its referer attribute set to "attacker.com". Similarly, if a website example.com contains images, then request for images to be displayed on the browser will contain referer as "example.com". It is quite likely, that a fake bank login page (on a Phishing Website) will redirect its user's to the original website after a user has entered his credentials on the fake page, so that the user does not realize that he had visited a fake website. If the original bank website, logs the "referer" attribute on all its incoming requests, then it can easily determine a phishing attack, when a large number of redirects are coming from a single website -- which isn't one of its own domains. The bank can also utilize this information to display a Warning message to its customers that they may have just been victims of a Phishing attack, and they should call help desk immediately to have their accounts secured. Unfortunately, again - this isn't a fool proof mechanism since many applications, proxies and firewalls can empty out the "Referer" attribute -- for user privacy, etc. Still, its a good idea to have referer logging in place in a website.
  5. Google Safe Browsing API: Google provides a Safe Browsing API that developers can leverage to check whether a given URL is considered unsafe, or a Phishing website. Google Chrome and Mozilla Firefox use the same data for warning their users about malicious websites while browsing. For example, this API may be used by a web site which allows people to post URLs on its web pages (like maybe a forum, etc). Since user's can post malicious URLs on the website, one can use Google's safe browsing API to check if a URL is marked as malicious or not in Google's database. There are two types of APIs available: one allows you to submit the URLs to a Google Service, and get a response which indicates each URL in the request is safe or not, or alternatively you can download the Malicious URL database, and perform a local comparison. For more details check out: https://developers.google.com/safe-browsing/
  6. Phishing Detection Browser Plugin: Some websites will allow a user to install a browser plugin that can detect whether the user is visiting a real banking website or not. This is not such a popular option, but still something that can be considered. There is a lot of research going on in the area of Fake website detection, so that one is able to detect whether a website is "similar" to another website. Originally, to detect "similarity", algorithms would compare HTML snippets of websites to see if they match, but newer malicious sites have been able to circumvent such detection -- by using Flash and Images to generate pages which look similar to the real website but contain totally different HTML snippets than the original website. So newer algorithms are being developed which perform image comparisons and use a weights based score to detect polymorphic Phishing. For reference check out: http://www.cs.rice.edu/~wx6/publications/phishing.pdf

Cookies: Developers should have good understanding of Cookies and their underlying behavior, since a lot of XSS and CSRF attacks on websites are built around exploiting the way Cookies are used by websites, and are managed by browsers.

Cookies were originally designed in 1995 by Netscape as a way of storing "Shopping Cart" information of a user on the browser side. Since HTTP is a stateless protocol, developers wanted a way of being able to recognizing a user without having to ask "Who are you?" on every page request, and also have the ability to store some user specific data on the browser. A cookie can be used by the server to ask the browser to store some information -- given a "Name" and a "Value", like "UserSessionID = 1234", and the browser will create a Cookie for the current website, put this information inside it and store it locally on the user's machine. When the next page request is sent by the user for the same domain, the browser will automatically submit all cookies to the server along with the request, so that the server can "identify" the user or use the data in the cookie to give a personalized experience to the user. In response, the server can ask the browser to update some cookie values, or create more cookies.

The fact that many Websites use Cookies as a way of "identifying" a user is exploited by many XSS attacks -- since a malicious user can simply submit a different user's cookie, and the website would mistake the malicious user for a genuine user.

Some points to note about Cookies:
  1. A Cookie is always associated with a domain. The domain of a cookie is the website which requested the browser to create the cookie. For security reasons, a browser will never submit a cookie of one domain to another domain. All cookies for the current website are accessible using Javascript via the document.cookie object, but again, only for "current" domain. In Google Chrome browser, one can see which cookies a website has stored using Developer Tools (Ctrl+Shift+C or Cmd+Shift+C on Mac), in the "Resources" Tab.
  2. A "Session Cookie" is a cookie whose lifetime is the "session" of the browser, and will be auto-deleted once the browser session is destroyed. This is the default behavior of a cookie, if the server has not specified Expiry or Max Age for the cookie. The point to note is that a browser session is destroyed only when the browser window is closed, and not when the browser Tab is closed. Hence, the cookie is still present, even if a user closes the browser's Tab. Websites which track user's session via cookies, will make use of Session Cookies to store user identifiers.
  3. A "Persistent Cookie" is a cookie whose lifetime is defined by the "Expires" or "Max Age" attribute of the cookie. Such cookies will be kept by the browsers until they expire. Even when the browser session is destroyed, these cookies will be stored on the user's machine, and the next time a user visits the same website, the browser will automatically submit the cookies to that website. A browsers can choose to delete a persistent cookie in case it needs space, or a user decides to remove all cookies from the browser, etc. Persistent cookies are used by websites to remember personalization data -- and also by the "Remember Me" functionality on many websites.
  4. A "Third Party Cookie" is a cookie whose "domain" is different from the "domain" being currently visited by the user. For instance, if a user visits www.example.com, and this website shows advertisement on the right side of the page, for which it creates an iFrame and makes a call to www.some-advertisement-server.com, then the Ad server may choose to send cookies for its domain, which will also be stored by the browser. Therefore on visiting one page, a user's browser now stores cookies for two different domains. This cookie, which has been stored by the Ad server is called a Third Party Cookie. The reason the Ad server chose to store the cookie is that, if the next time a user visits another website, which also has a tie-up with www.some-advertisement-server.com, then the cookies already stored for www.some-advertisement-server.com will be automatically submitted to the Ad server, and then Ad server will be able to give personalized ads to the user based on the knowledge that the user visited site A, and now site B. A Third Party cookie is also a type of Persistent Cookie, except that it was created by a different site than the one which the user visited. Third Party Cookies have raised a lot of Privacy concerns, and subsequently some countries have now come up with legislation which requires a website to get a user confirmation before storing any Third Party Cookie. Check out "Cookie Law" http://www.cookielaw.org/
  5. Multiple Tabs in a browser, will share the same cookies -- and therefore if a user visits the same website on two different Tabs, the same cookies will be shared by both Tabs. For instance, try logging into GMail from one Tab, and then access GMail from the other tab, and you will see that you are automatically logged into the same user session that you opened in a different Tab. This is because as far as GMail is concerned, its the "same" user, since the browser sends the "same" cookies for both Tabs. Interestingly, for most browsers, even if you open a new window, and enter GMail, you will again get logged into the same account, since for most browser's a New Window does not really mean a New "Session". But, if you open Google Chrome in Incognito Mode, then you get a different session from the non-Incognito Mode, which is why you can now login as a different user on GMail -- since Session cookies are not being shared across these two Sessions of Chrome.
  6. A cookie is only associated with a domain, not with a protocol or port. Therefore, a cookie which was created as part of an HTTP connection with the domain, will also be submitted for HTTPs, FTP, and other such requests to the same domain by the browser. If a cookie was created as part of a port 80 connection, or port 1000 or some other port, it will still be submitted to all requests to the domain -- irrespective of the port. Therefore, all websites on a domain must be trusted, since cookies from one will be accessible to the other as long as the domain name matches. If a router is compromised (DNS Spoofed), then browser will unknowingly submit cookies for a domain to a malicious server, since it would think it is submitting data to the right domain.
  7. Cookies can also be scoped to a sub-domain. For instance, bar.example.com can set a cookie for bar.example.com, or for example.com. If a cookie is set for a sub-domain, then it will only be sent for requests directed to that sub-domain. But, bar.example.com cannot set cookies for foo.example.com.

Detailed specification on cookie behavior can be read in RFC-6265

Interesting Side Note:

Besides cookies, there are other ways to identify a unique user (or shall we say browser). Check out: http://panopticlick.eff.org/ This website will tell you how unique is your browser's fingerprint, based on browser version, OS, patch level, fonts installed, screen resolution, plugin, and settings. It is quite surprising how easy it is to identify unique browsers based on these parameters.


Cross Site Scripting (XSS) and Cross Site Request Forgery (CSRF)

XSS is a broad class of attacks where Javascript injection into web pages is used to perform malicious actions -- from defacing the website to extracting personal details and cookie information and submitting it to a different website (Cross Site). Most XSS attacks happen when a user can input javascript code into web form elements like text boxes, or when a website is designed to take user inputs and stores them onto web pages (like forums, user reviews, collaborative editing websites, etc). Since an HTML page consists of a mix of CSS, HTML Markup, Javascript and user inputted data, it is difficult for a browser to distinguish between a user input text, and javascript execution code. Malicious users utilize this vulnerability of a browser to enter javascript code into form fields like name, address, etc to make the browser execute javascript which will alter the behavior of the website.

One example of an XSS attack would be when Malice on his profile page of a legitimate website, puts a malicious javascript code which reads the current user's cookie, and submits it to a malicious website. When Alice logs into the legitimate website, and views Malice's profile, the browser executes the malicious javascript while rendering Malice's profile page, and submits the cookie data to the malicious website -- without the knowledge of Alice. Depending on the type of website, Malice now has access to Alice's personal information, and may also be able to impersonate Malice on the legitimate website, using Alice's cookie -- until Alice logs out. XSS attacks can have very serious consequences to unsuspicious users.

To prevent XSS attacks, it is important to sanitize and check all user inputs, so as to ensure that malicious javascript has not been entered by the user. This may seem easy (for instance disallow any script tags, etc), but in practice is quite difficult, and requires the use of standard well-tested HTML sanitizing libraries like those provided by OWASP Security APIs. If you would like to see examples of how Hackers circumvent javascript filtering in websites, check out this well written explanation of the Samy Worm exploit on MySpace http://namb.la/popular/tech.html

CSRF is a class of attacks where a Hacker utilizes the trust that the website has on a user's credentials (cookie), to make the browser request actions on the behalf of the trusted user -- which the trusted user would otherwise not authorize. For instance, let's say Alice is logged into her company website (alice-company-website.com) in the browser, and on a new browser tab, Alice is checking out a social networking site. While browsing the site, Alice visits Malice's user profile page, on which Malice has posted an image link such as

<img src="http://alice-company-website.com/mailReport.html?report=companyReport&amp;mailID=hacker@some-website.com" />
Then the browser will submit a request on Alice's behalf while rendering Malice's profile page -- which will send a request to the server to email the company confidential report to Malice's email address, without the knowledge of Alice. This happens because when the browser requests the page on alice-company-website.com, the browser will submit all cookies of Alice's company website, and the legitimate server will have no idea that the request was not really approved by Alice (but was instead forged by Malice on behalf of Alice). This form of an attack, where on visiting one website X, a request is forged and sent on another website A, while the user is logged into website A (or while the cookies for site A of the user are stored in the browser session), so as to execute an action not really authorized by the user is known as CSRF. A CSRF attack can also be done via an IM link, or an email link, as long as the link is clicked while the user is logged on to the legitimate site in the browser.

This form of vulnerability does have a small attack vector, because Alice needs to be logged into for the attack to be played out. It also requires that the website has non-confirmation based actions, like GET requests with side-effects which can be exploited by the attacker, since Alice would not be "authorizing" the request. Websites which have the "Remember Me" feature aggravate this form of attack, since the user's session containing cookie is stored for longer periods of time.

Protection options for Websites to avoid XSS and CSRF attacks:
  1. If a website has a 2-step authentication, or a transaction password for authorizing all sensitive actions, then it becomes quite difficult for an attacker to perform CSRF without involving the unsuspecting user.
  2. Sanitize all user inputs to disallow malicious (or any) Javascript code.
  3. Ensure that there are no GET actions on the website, which have side-effects or modify the state of the user's session.
  4. Use CAPTCHA techniques for verifying sensitive actions.
  5. Encourage users to open a separate Browser "Session" (like Incognitio Mode of Google Chrome), for performing sensitive web interactions, and only work with one website at a time in one browser session.
  6. Make use of CSRF token technique to generate a unique URL (for GET Requests) or hidden parameter (for POSTs). In this technique, the server generates a unique Hash (http://en.wikipedia.org/wiki/HMAC) based on User's Session ID, Server Key, and URL Action and attaches that as part of the POST's hidden input field. This ensures that no one can sent a forged request, since in such a case, one will need to have access to the user specific hash generated by the server. For more details read: https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet
  7. Cookies support two additional flags for security purposes. One is the "HttpOnly" flag, and the other is a "Secure" flag. Marking a cookie as "HttpOnly" will ensure that the cookie cannot be read via Javascript, and will only be used by the browser to submit the cookie for HTTP or HTTPs connections. That will ensure that a malicious XSS Javascript cannot read the user's cookie. Session cookies should always be marked as HttpOnly for safety purposes. Marking a cookie as "Secure" will ensure that the browse will submit the cookie only over a secure TLS/SSL connection, and not over an unencrypted channel. This will help against a Man-In-The middle attack or in case of DNS Spoofing where the spoofed website does not support SSL (even though it points to the same domain).

Since a lot of the attacks exploit the fundamental design of how the Web works, it is important for all Web Developers to understand various risks, vulnerabilities and attacks which a website can face -- to be able to design systems which are safe and cannot be compromised easily.

Saturday, April 7, 2012

Making your Terminal look pretty on a Mac

I got a 15-inch MacBook Pro (8GB RAM/256GB SSD) last week. Yippie!

My first requirement was to have my terminal look as snappy as my previous Linux one did. So, after a little googling, and checking out my Linux settings, I finally made my Terminal (iTerm) look pretty. Here are the steps:


  1. Open Terminal.
  2. Type the command "vim ~/.bash_profile". This will open the hidden file ".bash_profile" from your home directory. I am using vim to edit the file, but you could instead open the file in any editor of your choice. If this file does not exist, create it. We will be putting our configuration settings in this file, which will be read by iTerm Terminal whenever a new window is opened.
  3. Paste the following lines into this file: 

  4. # enables color in the terminal bash shell 
    export CLICOLOR=1 
    # sets up the color scheme for list
    export LSCOLORS=gxfxcxdxbxegedabagacad 
    # enables color for iTerm 
    export TERM=xterm-color 
    # Color Prompt 
    export PS1="\[\e[36;1m\]\u@\[\e[32;1m\]\w> \[\e[0m\]" 

    # sets up proper alias commands when called
    alias ls='ls -G'
    alias ll='ls -hl'

  5. Save you file and exit. 
  6. Close/Exit your Terminal, and open a new Terminal Window. 
  7. You should be seeing something as pretty as the screenshot below.




Note:
  1. If you would like to make more changes to how your prompt shows up. Read this link:
    http://www.funtoo.org/wiki/Prompt_Magic
  2. If you would like to change the LSCOLORS attribute, so that directories show up in some different color, then read this blog link:
    http://blog.imzaid.com/color-terminal-bash-on-the-mac-iterm-included

Saturday, February 25, 2012

8 Months at ThoughtWorks Pune

I joined ThoughtWorks, Pune in June 2011. 8 months ago. Previously I had worked with Persistent Systems, IBM Software Labs and Performix (a start-up). I would like to pen down my ThoughtWorks journey so far. Correction. Incredible journey.



For sake of readability, let's list down the things I feel are awesome and different about ThoughtWorks, and why I catch myself wondering why the hell did I not apply to ThoughtWorks earlier. For the record, I joined ThoughtWorks as a Senior Consultant, and my role here is Programming, Programming and yeah Programming. :) Lets see.. what this means.

1. People: The most awesome thing about working here is definitely the People. Each and everyone has a special talent. Everything else that follows is just a nice side effect of smart people doing things they love. If you are looking for a UI guru -- there are many, if you are looking for a Linux guru -- again there are many, if you are looking for someone with experience in Mongo DB -- turn around and go to the 3rd table, if you are looking for someone with experience or interests in Statistical data modelling -- drop him an email -- he should be visiting us next week for one of the many technical meetups at ThoughtWorks, if you are looking for someone with awesome drumming capability (yes music!), well you can find him too. Talent seems to be a complementary feature of all great programmers. I guess, you can't hone just one skill. Perfectionism and curiosity become a habit. At ThoughtWorks, the recruitment process is stringent, and we are careful about selecting folks who can build, fit and contribute to the ThoughtWorks culture of excellence, fun, and social responsibility.

2. Pairing: Now, this one is very interesting. We pair for all our development work. That means, every code that is written is written by two people sitting side by side, connected to a huge monitor and a pair of keyboards/mouse. Sounds unbelievable to most folks. Its true for ThoughtWorks worldwide. What this means is that obviously silly bugs are caught while programming, it also means I can't get lazy. If I skip a unit test -- or write code that is sub-optimal, the other guy will be like - Dude!! Then I'd go.. you know what.. why don't you drive (means type). The other awesome benefit of Pairing are the little tidbits you pick up from your pair. For instance, I used to use :wq for saving a file in vim. Well one day my pair said.. hey try using "Shift zz". Its faster. And since then its been a bye bye to ":wq". The net output of pairing is -- both folks learn the best from each. That's quite something! If you are weak in Javasript, your pair can help.. and in the end you know stuff you didn't know 10 mins ago. Next time you can teach that to someone else. Also, its bye bye to gtalk, facebook and twitter while pairing.. which means hours of solid productivity, with no distractions --- and to top it, you don't really feel exhausted because you have a pair and a friend to keep chatting with the whole time! And, then there is the concept of Pair Rotation. What this means is that every 2-3 days, you rotate pairs. This helps in ensuring everyone learns from everyone, and everyone learns mostly everything about the code base -- so there are no bottlenecks and module-owners. Everyone owns the code, and everyone can fix anything. Of course needless to say, we still have our strengths in certain areas, but we still know sufficient about other areas as well.  If you feel Gym membership is costly, try open heart surgery instead. Same is the case with Pairing. Its totally worth it!

Note: Even if you just read this post till here, and have never done Pair Programming, I suggest, for now you leave this post alone, and hurry along finding out more about Pair Programming because that's the single most beneficial decision you can take (other than choosing to join ThoughtWorks of course).

3. Linux (a.l.a Unix): Well, I spent 9+ years of my life with a Windows Laptop, thinking - well, Linux is for the die-hard geek, hacker, or for running mission critical server applications. I was familiar with Linux, and tons of its avatars -- thanks to multi-platform testing at IBM. Guess what --- if you are a developer in ThoughtWorks, and you don't work on a Unix/Linux  machine --- something is wrong with you, or you must be writing Windows all over again. In the 3rd week at ThoughtWorks, I wiped Windows off my 8GB Dell Latitude laptop, and installed Ubuntu. All our development happens on Linux machines. And believe me, for a developer, Linux is God-sent. The amount of tweaking, customization, productivity enhancers, and shortcuts you can enable in Linux.. makes Windows feel like crutches. And, yeah -- it helps having folks around you who have mastered such tools. Makes it way easier for you to hop on for the ride.

4. Transparency and Feedback: These words take on their real meaning at ThoughtWorks. There are no walls in ThoughtWorks Pune office. Just one big hall, with large tables (one for each team). No cubicles, for anyone. No exceptions. This means -- you know everyone, you see everyone, you say hello to everyone, and you can hear anyone. Turns out, it's quite easy to work in such an environment. You want to know what's happening on project Z, go and walk up to anyone in its team and he or she will bring you up to speed. You want to attend the leadership meeting, just walk in -- you are most welcome. You want to know if you are gonna be sent abroad on an assignment this year -- go say hello to PSM, and he'll tell you who all are selected and why. You delivered a presentation, and it was not-so-good, people will come and tell you what you could do better. If you rocked, they will tell you that too. You suck in Web Development, no problem, they will tell you, and help you find resources to improve, or better still, pair with you on such tasks. You want to know what's the future projects pipeline -- check out the internal my.thoughtworks portal. You want to know which bid we lost out on, and why, check out my.thoughtworks. Its all out there for you to see, read, learn and share. We trust you, and as long as you live up to the trust, things are much easier to manage and communicate.



5. Social Impact Projects (SIP): Besides the plethora of open source project options, we also have a number of Social Impact project options available for voluntary contribution. This gives me an opportunity to learn new technologies, while making a useful contribution to social causes. The great thing about SIP projects is they seem to be invariably built using the best and newest of open source technologies, cloud platforms, and the works. Most SIP customers want us to build, deploy and manage the whole solution, with no silly opinions like "It must use ESB", or bloated middlewares, etc -- only because someone higher up spent lots of money buying bloatware which now can't be justified. SIP projects are nimble, agile, and very satisfying to work on. ThoughtWorks has special folks identified for driving and helping volunteers contribute to SIP. The momentum on SIP in India is huge. To know more:
http://www.thoughtworks.com/jeff-wishnie-rohit-bansal

6.  Comfort, Fun and Food: Thanks to Extreme Programming (Kent Beck), not only do we NOT have psychotic processes (Agile rocks!), we also have tons of outings, meet-ups, and fun! In fact, after I joined ThoughtWorks, my wife was disgusted with the number of team outings I went out for. Teams are made by getting a bunch of folks together, and successful teams are made by getting a bunch of passionate friends together. ThoughtWorks is made of these successful teams. And successful teams celebrate.


To sum it up: My wife has joined ThoughtWorks last month! :) :)



[Update: 25/July/2013] A really nice video on life at ThoughtWorks: http://www.youtube.com/watch?v=owrMkfR5a4Q

[Update: 07/Aug/2013] Added snaps I took of Pune office.