Emacs absolute basics

There is an awesome editor out there which are used by great programmers and then there is an OS which is also getting used as an editor by the not so geeky 🙂

Hugely inspired or copied from Vim Katas


I will write a line and hit C-j to insert a new line
I will write a line and hit C-a C-j to insert a new line above

I will navigate from beginning of line to end of line using C-a and C-e
I will go to the top of the document by hitting M-<
I will go to the first line of the document using M-g g 1
I will go to the thirty fifth line of the document using M-g g 35
I will delete the first line by using M-< C-k
I will paste it back using C-y

I will delete the next line using C-n C-a C-k
I will delete the entire line using C-S-backspace
I will paste it back using C-y
I will repeat previous command using C-x z

I will undo using C-_
I will undo again using C-_

I will select a line using C-a C-SPC C-e
I will remove everything using C-w
I will undo using C-_

I will go down a line using C-n
I will go up a line using C-p
I will go left 1 using C-b
I will go right 1 using C-a
I will go up four lines using M-4 C-p
I will go up four lines using C-u 4 C-p
I will go up four lines using C-u C-p
I will go down four lines using M-4 C-n
I will go down four lines using C-u 4 C-n
I will go down four lines using C-u C-n
I will go down eight lines using C-u C-u C-n

I will select 2 lines below this using C-n C-a C-SPC C-u 2 C-n
I will select 4 lines below this using C-n C-a C-SPC C-u C-n
I will select 3 lines above this using C-p C-e C-SPC C-u 3 C-p

I will save using C-x C-s

I will navigate by word using M-f
I will navigate by many words using C-u 3 M-f
I will replace words using M-%
I will undo those changes using C-_
I will delete this line using C-S-backspace and undo with C-_
I will delete these words using C-u 3 M-d
I will undo using C-_
I will save and quit with C-x C-c

Agile Tour Visits Hyderabad

For the first time Agile tour visited Hyderabad last weekend on October 1st. The event was held at the awesome HICC.

This event comes in the back of Agile Hyderabad 2011 conference which was a well organized one and had great speakers like Vibhu Srinivasan and Jesse Fewell.

The theme for Agile tour Hyderabad was Agile First Steps. As the theme indicates, the conference delved into the basics of Agile, especially the “what” bit — understanding what agile is and what benefits it provides. Understandably, the talks didn’t venture much into the “how” of it. On a personal note, it was a little disappointing to have such a theme as I was pretty much comfortable with the basics of Agile after reading books on Scrum, Extreme Programming, Lean Software Development and of course practicising xDD and little bit of Scrum. But I was also looking forward to meet the agile community in Hyderabad.

The event kicked off with a key note by Mr. Vivek Jain, Director, Pega Systems, on “Introducing Agile Work in a Product Enterprise”. Basically the talk was about how Pega System adopted Scrum and have seen exponential growth in revenue numbers and have successfully launched a couple of products after that. They had adopted Scrum hardly 2 years back after doing waterfall/chaotic model for a decade or long. The talk was well delivered but I have a peeve for the way it was delivered. I thought they spend lot time explaning Pega Systems, their revenue and market analysis etc. rather than focussing more on nitty-gritties of how really Scrum works at the ground level or at least what their experience has been.

Then we moved to the 2nd key note of the day by Madhur Kathuria. Madhur is the only Certified Scrum Coach in India, and I have heard him talk at Agile Hyderabad 2011. He is an eloquent speaker and really engages the audience well in his talks. So I was expecting a very good session and Madhur really delivered with his keynote on “Why it’s hard to Scrum in India”. He touched on the cultural factors which stand against us (Indians) when it is coming to do Scrum. I still remember the points he delivered on that day on top of my mind. I think some of the factors he touched upon were

  • The Cultural Factors – where we Indians hesitate to say NO
  • Chalta Hai Attitude & Jugaad – basically we tend accept status quo and just accept to fate “it’s like that only”
  • The Service Focus – where we accept anything that client tells us to do blindly
  • The Power Index – where the perceived distance to power center is pretty large when it comes to Inida.

Overall Madhur did a great job and by interspersing games between the talks and there by engaging all the participants the session received a good response.

After the keynote was the time for Just a Minute sessions where anyone can walk up to the stage and talk for a minute or two about their pet topics. I talked about the lack of focus on Engineering Practices in Scrum, especially in Indian context how bad it may hurt the success of agile projects. The message I was trying to drive was – even if we do Scrum, do standups and plan our sprints etc. if we are bad at writing code, if we do not do proper TDD, Refactoring, Continuous Integration and so on, we are basically just sugar-coating our old bad habits with the guise of Scrum and in a couple of years down the line we could be staring at a system which would be tough to maintain, enhance and understand much like the Big Ball of Mud. Weren’t those the problems which waterfall had and agile tried to solve when it came to limelight in the early 2000s?

Here is a picture of me doing my JAM session.

Arun's JAM

The next 30mts was for Open Space where, any one could propose a topic and whoever interested in topic can form a group and discuss the same. There were around 8 topics up for discussion. I participated in topic “TDD” and had some heated debate with my fellow participants and I also propsed and anchored a topic on “What killed waterfall could agile”. Again it was based on my JAM talk about lack of engineering focus while practicing agile. The ideas were around lack of learning in technical side from Project Managers, lack of passion to improve by the upcoming developer community in India and of course the attitude that “if it ain’t broke, don’t fix” which is dangerous! And here is why, quoting from the Refactoring Workbook:

Build Testing into Your Practice

There’s an old adage (as so many are), “If it ain’t broke, don’t fix it.” (How many times has that last simple change caused an unexpected bug?) In programming, the downside of applying this adage is that the code just gets uglier and uglier.
Refactoring is willing to go against this rule through two mechanisms: safe refactorings and a supply of tests to verify that the transformations have been done correctly. Don’t neglect your tests.

In fact resonating my idea Jesse Fewell has touched upon the same topic in his latest blog past about his experiences with India. Read the section India’s talent is an impediment. Sad, but true!

The afternoon sessions were split into two tracks and I had a chance to attend the following topics

Learning Agile by Flipping Pennies, by Tushar Somaiya – the idea of the session was simple, there was a team of 4 (requirements guy, design guy, coder, tester) and their respective managers. The team had to flip a bunch of different coins based on some value assigned. First it was done in waterfall way, then it was done in a more iterative way with smaller releases and focussed priorities. After each round the timings were noted and we were shown how agile really reduced the time to market and how effectively people were engaged rather than sitting idle.

Culture and it compatibility in Agile Adoption, by Mohammed Shaik, PayPal – Shaik really spoke well from his experience of 10+ years on what sort of org culture works for Agile and what doesn’t work. He was really forthright in pointing out the short comings of Agile Projects being done in India and how they were just doing lipservice in the name of Agile.

Scrum and Lean: Multiply the Powers, by Om Band, SAP Labs – He demonstrated how SAP had adopted lean practices and meshed it with Scrum and using it in their day-to-day work. I thought this was the best session I attended, it really concentrated on the meat instead of focussing on SAP. The explanations of Lean principles were crisp and to the point. I really enjoyed this session, more so because I am very keen in Lean Startup concept as well.

Master the how, not just the what and why in Agile world, by Shweta Parashar, Xebia – This was a game of building Lego House using Scrum methodology. There were 3 teams of 7 members and each team had Scrum Master. We planned on the Back log, Sprint Plans and Sprint Tasks embarked on building the house by picking task by task. There was also a scrum board on which we posted our tasks using sticky-notes, and moved them from “To Do” to “In Progress” to “Done” time to time. I think the time given for the session was not enough to complete the task or the task was big (probably they could have chosen a less complicated lego toy), nevertheless it was an enjoyable experience. I thought even the presenters were doing it the first time and generally the teams struggled to build the house because there were a hell of a lot of pieces to be arranged. Our team did a fairly decent job, and only we could build something which was appreciated by our Product Owners (aka presenters). Perphaps my experience of playing with my son daily using blocks and lego toys helped us 😉

Overall

Yea, the cookies were real good, food was awesome, and the desserts were fabulous 😉

Coming to the sessions, as I said it was at a very basic level. So if you were aware of Agile Methodology and its basics there was nothing much to take home as learning — it was just reinstating the points back. Again more of it was tuned towards corporate audience in well-set companies (read Project Managers). As a Startup founder may be many of these points may not be applicable for my situation. But looking at the audinece profile, most of the members were Project Managers who never had done Agile in full spirit, they were also trying to figure out how things worked in Agile and many were confused whether what practice they were doing in their job was Agile or not. I could see that there were many misconceptions and misunderstanding floating around. Like one guy concluded that Waterfall with shorter iterations is Agile, or some one else was saying that they did TDD, where they documented the test-case in a Word Doc before starting coding or a Project Manager was mentioning how Agile reduced their release cycle from one year to 6-7 months.

And adding to the list is my pet peeve the target audience was just Project Managers not developers for this conference. I really hope that they had one track next year as to how they do agile technically as well like how you practically do TDD with xUnit or how you refactor the code and stuff like that sort. Again I may be wrong because in Indian context only managerial stuff may sell.

But overall I would rate the event was very good, it gave opportunity to meet a lot of people and surely I hope Agile Tour would come up with a big bang next year as well.

Some of the links from the event
http://www.facebook.com/media/set/?set=oa.267944886569832&type=1
http://www.isec.co/event_detail_info.php?event_id=5

Cucumber Scenarios : The Dilemma

Off late I am exploring BDD with Cucumber & RSpec in Rails.

The first step in BDD after understanding the customer requirements is to come up with the User Stories in a particular structure. Read this brilliant article from Dan North if you want to know more on the structure of User Story in BDD.

Let’s say I am creating a simple blog app, a sample Scenario for Creating a Post can be written in two ways as shown below. These scenarios are as coded for Cucumber.

A pithy way

Scenario: Create Post
Give I am a registered User
And I have signed in
When I go to Create Post Page
And Create a Post and Publish it
Then I should see the Post in the Index Page

or a Detailed way

Scenario: Create a Post
Given I am a registered User with name "Arun", email "foo@bar.com" and password "foobar"
And I sign in as "foo@bar.com/foobar"
When I visit Create Post Page
And I fill up Title as "Foo Post"
And I fill up Content as "Bar Contents for Foo Post"
And I publish the Post
Then I should see message "Post was successfully created."
And I should see post in the index page

The RSpec Book touches on this topic. It calls the first approach Declarative Style and second one Imperative Style. The Book also advises to use the former way if the project is more like a single man show where developer dons analyst/tester/coder hats and go for 2nd approach if the project has a bigger team and a business analyst for writing scenarios.

But its really a touch and go topic and its really difficult to gauge to how much in depth we have to go. The declarative seems to be pretty concise with loaded step definitions, but the imperative style gives a fair idea of what it takes to accomplish something in the system. So if we have a little bit informed tech savvy customer, probably second approach may add lot of value and clear lot of confusion. The imperative style is kind of walking the customer through the UI flows of the system.

But the conundrum with imperative style is if we already have good RSpecs for the Views/Controllers our RSpecs may read a lot like the imperative scenario and may look redundant.

So I can do rspec spec --format doc to get detailed documentation of the flows like the Cucumber imperative style. But of course it may not be good idea to share RSpecs with a Business User.

As the RSpec book says perhaps we need to mix judiciously both and perhaps experience only will teach to balance that.

The trigger for me to write this post was I was trying to create Scenarios for a sample project I am doing and just one hour I spend switching a scenario between declarative and imperative and was pretty confused where I would/should settle.

Phusion Passenger + Apache2 for RoR

Ok here goes my little notes on making a small nifty Rails App run on Apache2 with mod_rails provided by Phusion Passenger.

The installation of mod_rails is pretty straight forward. The installation steps are small and well documented here plus you have the amazing rails cast by Ryan Bates.

After watching the videos and reading the steps well I thought its gonna be a breeze I am going to wind up in 15mts and going to enjoy the night! Sigh! But things went wrong a 15mts exercise consumed close 4/5hrs of my time.

Ok here is where the fun is all about – I use RVM to maintain my installs. The first step in passenger installation is obviously to install the passenger gem. Now I felt it would be good to put the passenger in global gemset rather than any specific one and there by making it available to all. So I did

rvm use 1.9.2@global

Then
gem install passenger

Then
passenger-install-apache2-module

In my system after the above step it said some libraries were missing in the system (Ubunut Lucid Lynx), so I had to correct those by manually installing them with apt-get. After I got all the dependencies installed I again ran the passenger install and everything went fine.

At the end of the passenger-install step, it would spew a few lines of code which we need to put it on apache2.conf file. For me those lines looked like

LoadModule passenger_module /home/arun/.rvm/gems/ruby-1.9.2-p136@global/gems/passenger-3.0.5/ext/apache2/mod_passenger.so
PassengerRoot /home/arun/.rvm/gems/ruby-1.9.2-p136@global/gems/passenger-3.0.5
PassengerRuby /home/arun/.rvm/wrappers/ruby-1.9.2-p136@global/ruby

Now I added a Virtual Host to Apache to point to my sample Rails App by adding the following to the apache2.conf


<VirtualHost *:80>
ServerName http://www.depot.com
DocumentRoot /media/harddisk/work/ror/depot/public
<Directory /media/harddisk/work/ror/depot/public>
Allow from all
Options -MultiViews
</Directory>
</VirtualHost>

So that is it – restart the apache and hit http://www.depot.com in your browser you should see your rails app. But hey! wait won’t entering depot.com make the browser go and search the Internet rather than your local system. Well yes, so we have to make a small exception in the file /etc/hosts. In the file add a line

127.0.0.1 http://www.depot.com
127.0.0.1 depot.com

Ok all set! now again enter the URL to your browser biff! now got a new error

no such file to load -- bundler

The error was obvious it was not able to find the bundler. But from terminal I gave

gem list -d bundler

It was showing proper output, means bundler gem was there up and kicking. Mind you this command I was giving from the gemset which I was working on. Then I started googling this error and got many suggestions and tried one by one and spent nearly 3hrs and couldn’t make any break through. It was really frustrating.

Then finally help arrived from Hongli Lai of Phusion Team through the Phusion Passenger Google Group. She was pointing to me that ruby I am using for Phusion given in apache2.conf file was pointing to global gemset while bundler was installed in the other gemset. So I had replace the line in apache2.conf to pick up the ruby from the gemset which I was working on. So suppose foo is your gemset change PassengerRuby line in apache2.conf to

PassengerRuby /home/arun/.rvm/wrappers/ruby-1.9.2-p136@foo/ruby

Yes! This worked. Now if you are running a static site things would start working now. But my sample app had some tables so when I fired up app in browser it was complaining that tables were missing in the production.

So if you run into a similar situation just migrate the tables to production with rake.
rake db:migrate RAILS_ENV="production"

In my case I had some load scripts as well which was for loading some sample data to tables, for loading those data into production use the rails runner with following option

rails runner -e production script/load_xxxxx.rb

That is it, now I see my rails app working well in my Apache2 server with mod rails.

Ok! here is a small tip, in order to restart the Rails App after changes you need not restart the server, you can give this command from Rails Application Root

touch tmp/restart.txt

Passenger would see the timestamp of restart.txt and reload the Rails App if needed.

Cardinal Sins of Programming

What I am going to put here is nothing new or enlightening for a seasoned software developer, it’s just my trials and tribulations working through some heavily screwed projects over the course of my career.

So what are those software (mal)practices which sound death knell and ensue disaster? Well, here are my top recipes for embracing pain

1. Code Repetition – Remember DRY aka Do not Repeat Yourself. Even if you repeat a mere 3/4 lines of code in 10 odd places just pause and think over. Something ought to be fishy, try to put that repetition into a function, reuse the code. I have burnt my fingers fixing repeated lines of code in 10 to 100 odd places, even a full package of Java files being completely duplicated to another package. Just remain ruthless if you see duplication, it should never be there, don’t hesitate to select and hit del, even if your manager is going to fire you for that.

2. Intermixing logic – Or simply violating SRP (Single Responsibility Principle). Its really catastrophic to mix UI logic with business logic. It just makes the whole code very difficult to read/debug and even more difficult to change anything. Suppose you want a new UI for the same logic, you are screwed, you have to recode the entire business logic again or refactor the current code to separate out business logic from UI. In this aspect I need to point out a horrible technological offering from Java Stable – JSP. If not used carefully JSP spell death knell. It’s so easy to mix db calls/business logic and UI HTML into the JSP. But what we get out of this shortcut is a code mass which totally undecipherable, not at all debuggable and what more stack traces point to the dynamic servlet code line numbers of the JSP. I loathe JSPs and wish Sun never gave so much power to the naive user to intermix things and screw up life.

3. Conventions – How do you feel when you see code with variables test1, test2, test5, i, j, looper, arraylist, map1, list1 etc. Terrible and horrible. You read method half way then you wonder what this test1 actually is then you scroll up then find out what is getting assigned to it, then come down and re-read. The naming convention is not confined to Java variables or methods, even for HTML tags. Its really a bad idea to give tag ids as myForm or myButton. I faced really a real night mare when we decided to consolidate all JavaScrips littered over JSPs to a single JS and include it every where. In the end you have 10 functions with name submitForm. You are screwed, you have a real pain to correct them If your form is an Order entry form right name for form id would be orderForm and submit of that form would invoke submitOrder() method in JavaScript.

4. Looong methods – How long can a method be? 1000 lines, 4000 lines? Well I had opportunity to debug upto 10k lines of code in a method. My pea brain just could not hold on that and I just had to leave that and pray before I run those methods. I personally feel 50 is the max number of lines you can have, 50 is upper limit of tolerance, ideally a method need to be 10-15 lines of code in a well crafted software.

5. Exposing the internals – If you have class, don’t expose the internals of it to outside. If you maintain list of names as HashMap inside a class, don’t tell world that you are using HashMap. Just keep it with you and give neat interfaces to outside like
void addStudent(int id,String name)
Iterable getStudentNames()
void getStudentName(int id)

rather than using
HashMap getStudentMap()

In the latter case call can even modify the Map without you being aware of it and at later point if you are going to change the Map to a Student Class, you are screwed up because all those who are exposed to your internals need to be updated.

6. Building the code on ArrayList, HashMap – ArrayList and HashMaps are good, but should be the lowest level of implementation never exposed out through public APIs or methods. I had tremendous experiences where a HashMap contains 10 keys, 1st key returns another HashMap which intern has 10 keys and each of those keys have ArrayList, Array, String, Integer etc. Then there were wonderful data structures like an Array of ArrayList. These kind of things drive you nuts and crazy, unless you debug line by line and print line by line its never clear what goes where. Class is the most powerful abstraction you have and try to encapsulate things within the objects and pass around objects rather than Lists and Maps.

7. Think a little bit ahead – Yea its not good to code for requirements which you are not having today, but still for some cases it’s good to assume certain things and give that flexibility in the code. A case in point is internationalization, just don’t litter messages and boiler plate texts in Java/JSP/JavaScripts, have mechanism in place from beginning itself, even though you may just show things in English alone. Another aspect is db, it’s not good to assume one db and code it, code ought to be db agnostic. I had seen code where things were assumed to be always on Oracle and they take liberty of using sequences or sysdate (both Oracle specific) and one fine morning customer comes and tell you, well I want only SQL/Server. You are doomed.

8. Passion – (or lack of it) Majority of those who write code, do it for getting pay cheque at month end. Building code is cheap and easy. Anyone with a reasonable aptitude can do it. But crafting it to be readable, extensible, maintainable requires lot of pain, learning and rework. If you are not willing to take that pain, not willing to learn how to write testable software, then don’t just do it.

Guava/Google Collections – Unleash the power

I have been using Google’s core Java Library Guava (which includes Google Collections) extensively for the past few months in my project and man it’s an amazing and elegant framework to code with. Once you start using it, you can rarely get off it for its power and nimbleness.

The biggest use of it is reduce the Lines of code and avoid unnecessary looping through Java Collection Objects be it for filtering or searching or sorting. And what more it gives ability to do some amount of functional programming like in Java by passing around Predicates and Functions.

Some of basic classes which comes with Guava are

Splitter – which can split a String a produce a Iterable out of it. If you use Java String split you end up getting an array of String objects. But getting an Iterable out of it is really cool thing, especially if you get some request parameters which are comma separated you want to extract the individual parameter values.

String pattern = “Foo,Bar,Boo”;
Iterable pattLst = Splitter.on(“,”).split(pattern);

The above is a naive example, you have option to use an regex pattern, ignore empty strings etc.

Complimenting the Splitter is a Joiner, which joins the elements of a Collection for you, by joining with a pattern you provide.
Suppose you have list of values in a List which you want to concatenate with “,” and insert as a String to the db, Joiner will do the job for you in a single line rather than writing 5-10 lines in the conventional approach.

Iterable nameLst = new ArrayList();
nameLst.add(“Foo”);
nameLst.add(“Bar”);
nameLst.add(“Boo”);
String names = Joiner.on(“,”).join(nameLst); //voila!

The real power comes with Collection APIs. There are different components in it – Google Collection provides entirely new data structures like MultiMap, BiMap etc. Also it provides powerful ways to filter, search, sort and manipulate any Java Collection Object.

Let us explore the Predicate Interface, which facilitate in filtering of any Collection.

Suppose you have a class Student.

class Student {
String name;
int marks;
….
}

Further you have a list of students in a List

ArrayList studentLst = new ArrayList();

Out of the student list if you have to filter the Students with marks greater than 80, you can create a Predicate for it.
We can write a function which creates the predicate and it can used anywhere.

So in class Student we can put a static function.

public static Predicate getStudentMarkPredicate(final int mark)
{
Predicate studentPredicate = new Predicate() {
public boolean apply(Student input) {
if ( mark >= input.getMark() )
return true;
else
return false;
}
};
return studentPredicate;
}

The above function creates a Predicate, the predicate is defined such that it will return true for any input Student if his mark is greater than or equal to the given Mark.

So now if you want to filter the studentLst to get the Students who have scored above 80, you can do it with this small elegant code below

Iterable studentsAbvEighty = Iterables.filter(studentLst, Student.getStudentMarkPredicate(80));

Well you can pass any arbitrary mark above a get the list of students matching them.

Providing even more gist is the Predicates class which has some predefined Predicates like and, not, or, in etc. These are real power tools for you search and filter multiple lists elegantly.

Suppose you have two Lists
List studentLst; //master list
List failedStudents;//failed students

Now if you need to filter the passed students from studentLst you can do it with a single line of code like this
Iterable passedStudents = Iterable.filter(studentLst, and(in(studentLst),not(in(failedStudents))));

Look how readable the above code and no clunky mechanism of looping. The above is a very simplified example, but you can imagine the power by chaining ands/ors and nots and ins to produce a complex filter.

Another powerful class is the Ordering which can be used for Comparator Chaining a produce a multi-column sort. I would explore that in the next post.

So the key take away is if you are doing Java Development and haven’t used Google Collections, your moment has arrived for it now. Explore and have fun!

Can it be really Agile?

We will continually refine the design of the system, starting from a very simple beginning. We will remove any flexibility that doesn’t prove useful.

One of the many prophecies of Agile Development Process and in particular XP. I have been trudging the trail of spick-and-span product development for last 1year at least. Although org guidelines fixate a firm waterfall model of development, I have been reading quite a lot on agile processes. Every now and then I tend to cast some aspersions on agile way of development. One thing is it doesn’t scale well on big teams and big organizations. A case in point is development of a module in ERP system. Suppose I am doing Order Management (OM) module; I know upfront that OM depends on Shipping, Pricing, Invoice modules to pick a few. So I need to clean up my interface area with these modules upfront and declare them. My data model for interface with them has to be fixed. I should anticipate the future requirements and design the interface. For a change later in cycle can be really costly and moreover the cost has to be borne by the impacted team and not me.

One of the things which is not always true is that cost of change may not rise dramatically over time as XP suggests. In practical scenarios a cost of change can be significant.

Cost of Change

XP makes us believe that cost of change can be flattened if we follow a good test-driven programming. So as per XP daily builds and running automated tests is just enough to absorb this cost of changes. But alas! the reality is much more stark in contrast. After all software development is not just building automated unit test and running it daily to prove code is unbreakable. The underlying design should be flexible enough to accept the changes with minimal impact. Yes that is what XP also says, I know that. We in software world are in constant race against time. So suppose we just design things keeping in mind minimal things which just solves current requirement in hand and a week before deadline of delivery a requirement pokes up which is sort of destabilizing the underlying data model, what can we do? How many managers will have the gumption to take up an overhaul of data model late in the cycle?

I tend to take the argument “We will take away any flexibility that doesn’t prove useful” with a pinch of salt always. If the flexibility is like providing many-to-many relation between two tables compared to one-to-one relation existing today, I accept it. I know the cost of changing from one-to-one to many-to-many is manageable. Suppose I am coding a reporting tool on top of a warehouse, for my graphs I won’t provide all flexibilities like formatting X-Axis, Y-Axis, data points (with different colors, marker shapes etc) if the requirement is not there today. I know the uptake of this late in cycle is not going to destabilize the existing code. If I anticipate a requirement which is going to destabilize my whole code, I believe I should have thought about it at the beginning even though the requirement was not there at that point of time. A late cycle infusion of change can be jittery for the developers, for the management and taking undue pressure on all the parties concerned. As in our case if I am doing Business Intelligence software on top of a warehouse; I guess the design of the warehouse (called dimension modeling) should be done upfront considering all the requirements that are there today or that can be there tomorrow. If we leave holes of flexibility in that which we tend to think can be accommodated tomorrow its sure recipe of disaster. For a late cycle change like adding a new dimension to the warehouse can be impacting the guys who write code to build the warehouse (ETL), the guys who do reporting on top of the warehouse (analytics) and it can be too complicated to incorporate it.

XP can work very well, if I was doing a Library Management System for my College Library. The domain area is quite clear, boundaries are drawn and I don’t expect many dependencies with other systems. XP sulks in a product development environment where you are trying to connect disparate systems and frameworks and are trying to build a solution which you think customer is going to derive value. In a generic product development environment I guess it’s hard to get a customer to sit by you and validate each and every step you take. Well I know this is not a problem of XP, and it also means XP is not the panacea for all software development woes.

Another interesting aspect in XP is Collective Ownership of code. Yes its really worthy if we don’t build silos of specialists and instead carve out jacks of all areas. This is very much practical in the Library Management project in College. In practical cases, each module requires some core strength from the designer or architect involved. I don’t mind if we are just shuffling developers around modules and make them code. But I do believe there should be a owner/architect/designer for a module in a project, who know the historic decisions that have been taken and more importantly why they were taken and shape up the module in elegant way during the due course of time. If we are going to shuffle owners every month, it’s a recipe of acrimony. For in big enterprise systems it’s impossible for a single guy to grasp the complexities and nuances of each and every area make clear decisions. A case in point the warehousing and analytics solution, the ETL itself is so complicated that a guy who has worked in 3 months in analytics module may not find it a cake walk to suddenly switch over one day and chip into ETL and incorporate design changes into it. So the key is having an owner, a pillar for each module who decides how to incorporate changes without setting in entropy. And obviously programmers can be made to code as per specs in different modules. Here again its practically impossible for the person who is coding to take design decisions on the fly when the module he is coding involves 900 classes, each class spanning 100-500 lines. I would expect one guy who has fair deal of idea and handle to these classes and what it does before setting upon a design change.

Having said all these let me not be gushy about the agile way. Automated testing, Test before code, pair programming, adeptness to changes, 40hr work week are all too tempting and sinews of agile way to be hushed away in the background.

So I think it all boils down to the situation in which you are in and obviously it’s a balancing act and judicious mix of agile and waterfall methods which can sweeten your time during the travails of development.