My experiences with Waterfall Model of Software Development
May 11, 2009 
Agile Software Development is the latest fad. Everyone is jumping in the bandwagon and not without reason, I believe. But embracing change (aka uptaking agile) is not an easy task, when your surroundings are tethered to the archaic model of software engineering. It takes a lot to make people believe in the new way, inspire them, show them results. It’s no mean task.
So how do we develop software today? It is simple – the traditional waterfall way.
Who does what?
Business Requirements – Product Management
Functional Design – Product Management
Technical Design – Developer
Code – Developer
Unit Test – Developer
System Test – Testing
The figure is quite self-explanatory, a classic waterfall model.
Business Requirements – In this stage, the product management group comes up with requirements for the software. All requirements are grouped into a document which is the input for next stage. Say requirement can be – Ability to enter new orders in the system, when the order is entered, the system must auto-generate the order number.
Functional Design – This would elaborate on the requirements. So here screen mockups are made, interfaces/integration points with other systems are identified, open APIs are identified, reports, workflows et al are also etched out. So this effectively becomes the bible for the developer to start the design. Also in this document are mentioned, the product use cases and functional test cases.
Technical Design – Starting with functional specs, developer goes on to design the UI pages, business logic, API, reports etc. and in the end comes up with a document. And unit test cases should also be identified and documented here. Code – now things are easy, you know what to do, how to do? It’s just the matter of converting them into a workable code. So again the development team pitches in and cranks out the code.
Unit Test – what every developer hates to do?
System Test – Out of developers’ hand, the code moves into the hands of testers who start banging the system with all sorts of test scripts prepared out of the functional designs.
Here are some of the traits of the environment which is making us follow this methodology
• Large teams spread across geography
• Very huge product, with a generic behavior. What I mean here is the product is made to cater not one particular client or industry. It has a broad foot print and designed to work across many verticals. So idea is to embed as many features and flexibility as possible.
• Lot of integration points. One system integrates with at least 3-4 other systems for a complete business flow. So these integrating systems are again as complex as this system, so they would expect their integration points fairly very early in the game. That is why all this interface and API are decided during functional stage itself.
• Management get good artifacts to track the progress after each stage a 200 page word document
• Different management chain/organization strategy for the three roles – product manager, development, testing. So each of them have their own plans and time lines and schedules. So its quite natural that by a deadline testers expect the code to be ready so that they can start banging their head on the system.
• Each system can have millions of lines of code
In reality this methodology is working and riding the business, so it is fine as long money is coming to your coffers 
But it does make life painful for all stakeholders.
The first two stages are the fiefdom of one team, what they say is what is etched in the stone. But it depends on the competence of the guy in the business realm. If it’s a new domain or without much customer reference or the team is without industry exposures then things get stuck here in the requirement phase. A typical symptom is Analysis-Paralysis, where the whole team is bewildered as to what to progress, what system to build, and there are lurking doubts whether this is going to be accepted in the field? So what they typically do is incorporate all sets of wild requirements into the product and directly shoot in the feet. YAGNI (You are not going to need it) is not the mantra WNAOI (We need all of it) is the mantra. So all sets of weird requirements can creep in. But say if the domain is pretty straightforward and you have customer references at your disposal then this phase becomes a breeze and they can come up with the requirements and functional designs in a jiffy.
From there things get tricky. A functional design once gets completed gets review from developers and he heads on for making the next word document in pipeline the Technical Design.
Typical Time-allocation across the project would be something like this
So as you see always the poor developer guy is time-pressed to deliver the software, by the time he gets to know the requirements/functional designs and he reviews those documents and circles back with product managers to clarify it, his time would have started ticking away.
Typical difficulties faced by the developer
• Difficulty in imagining the whole system and coming up with the technical document
• Itching to jump into code by putting off the boring documentation job which is allotted anywhere between 3 weeks – 5 weeks time. It’s really difficult to hold the attention span for such a long time and imagine about code and write about it still not implement it. So what happens most of the times in TD and Coding starts happening hand-in-hand, many a time code first and document next. I feel it’s good to do that instead of coming up with a document lock-stock and barrel. But as the time progresses the deadline pressure looms and coding will be happening at a frenetic pace, but not documentation and design. In this frenzy of meeting deadlines detailed Unit Tests are thrown out of the window.
At the end we will have a
• Unstable code, which works very reasonably or sometimes very badly
• No documentation of design or limited and not up to date design documentation
• No unit test scripts
• Not much standards adherence to standards – what I meant was coding standards.
There are quite few standards enforced by the build system, like explicit checking of cursors/connections whether they have been closed. Using parameterized binding when passing arguments from Java to PL/SQL etc. Apart from that there is not much standards like
• Doing proper OOD
• Using Design patterns
• Inline comments etc.
Many a times what happens is that guys with procedural background comes in and writes very neat Procedural programs in Java, without realizing the power of objects. So the end result is quite guessable, system works but partially and starts bombing in lots of places.
So the originally envisaged System Test Plan schedules are goes hay wire and more and more rounds of System Tests are carried out to make the system stable.
As you see, the code moves from an unstable state to reasonably stable state thanks to multiple rounds of testing. So when it goes live in the customer place, you can expect many more bombs exploding giving sleepless nights to developers.
The biggest nightmare is for the new guy joining any team in this system. He is left on his own with no proper documentation to swim through the labyrinthine code mess to figure out what is happening. That’s the real pain point. So is this the classic waterfall – well I don’t think it’s so bad, it’s just a chaotic way of developing software. If this system is working it can be attributed to one simple reason the people – smart dynamic and energetic people who relish on thriving on this complexity and pushes himself hard to meet customer expectations and meet the deadlines.
Functors and Autos
August 2, 2006 
Joel had an interesting post today. The programming language he is talking in that post is obviously JavaScript.
JavaScript is an amazingly powerful language. Most people just don’t realize that and since it allied with the no-brainer (at least that is what geeks in ES might feel) HTML, it hasn’t got the limelight in programmers mind as say a Java or Python.
Even I never had a pithy idea of the language capabilities till lately. Last year as a part of our project we had come up with some HTML prototypes and then I got a chance to deep-dive into JavaScript and frankly I was spellbound by the language capabilities. JavaScript is an OOPL! And you can write neat and clean OO Programs leveraging objects, inheritance and polymorphism. One of the trademark of JavaScript is that it’s a loosely typed language.
So you can have
var a = 1;
a = “Arun”;
a = new A;
Interesting? a can be a integer, string or an object of class A. No need for explicit declarations. Well some might argue that this kind of coding will lead to readability, maintainability problems. I think its just a matter of discipline and moreover you don’t expect your complete Business Logic inside a JavaScript.
Another interesting concept which Joel touches in his post is the functor. A functor is something like a function pointer (an explanation for a C guy). A functor is a function that can be manipulated as an object, or an object representing a single, generic function. So functors support functional programming and JavaScript too supports functional programming.
Here is an example of how you define a class in JavaScript
function Foo()
{
this.i = 1;
this.string = “Arun”;
}
obj = new Foo;
Foo() looks like a function and can be used like a function. Well Foo can also be used like a Class. So in the above example obj is the instance of class Foo. And another cool example in Joel’s post is that you can easily assign a function to a variable.
Coming to Java; our bread and butter language. Java being a very rigid language such luxuries are a far, far cry here. Why does Java doesn’t have Auto Variables like C# or Python? See an interesting post here.
I feel Java can also imbibe some of these cool features and become more dynamic, especially concept of Auto Variables as was illustrated in above example of var a in JavaScript.
An example problem which we faced in our project was like this
We have
class parent
{ void buildReport(params); }
class child1 extends parent
{ void buildReport(params); }
class child2 extends parent
{ void buildReport(params); }
class child3 extends parent
{ void buildReport(params);}
class child3 extends parent
{ void buildReport(params);}
child1, child2, child3 and child4 extend parent and implements the buildReport() function. Now this was a Template Pattern implementation. So buildReport() was my template function which internally called 4 steps, which I haven’t mentioned here.
Now the catch is the params. In each of those childs I need to pass different params. In child1 I need to pass a int. In child2 I need to pass String, In child3 I need to pass a object and in child4 an array.
I cannot have different signatures for buildReport() because that will break the Template Pattern as well as polymorphism which we cannot do away with.
Now the solution would be keep the params as Object. So what happens to int, use Integer() ?. Or pass it as Collection in a Vector or HashMap. And retrieve the relevant stuff from the HashMap depending on which child you are in?
We used the 2nd approach. I think in JavaScript it (params) would have been just a var. This is just one side. Now think of the same if a function buildReport() starts returning new types for each of the child.
So my calling program will become nasty with if else ladders checking instance of the class.
Like this
Parent p = new child3();
if ( p instanceof child1 )
int i = p.buildReport(); //child1 I expect int
else if ( p instanceof child2 )
String s = p.buildReport(); //child2 I expect String.
I think we have lost the usage of the Template Pattern here by building a if-else ladder in the main code.
Does auto solve this? Should Java really think of becoming a bit more lenient and becoming more flexible (loosely typed) like Python or C#.
These are just my thoughts. May be I am wrong also, let me know your thoughts if any. Meanwhile you can go and start learning JS and start getting bemused by it.
