Agile Software Development is the latest fad. Everyone is jumping in the bandwagon and not without reason, I believe. But embracing change (aka uptaking agile) is not an easy task, when your surroundings are tethered to the archaic model of software engineering. It takes a lot to make people believe in the new way, inspire them, show them results. It’s no mean task.
So how do we develop software today? It is simple – the traditional waterfall way.
Who does what?
Business Requirements – Product Management
Functional Design – Product Management
Technical Design – Developer
Code – Developer
Unit Test – Developer
System Test – Testing
The figure is quite self-explanatory, a classic waterfall model.
Business Requirements – In this stage, the product management group comes up with requirements for the software. All requirements are grouped into a document which is the input for next stage. Say requirement can be – Ability to enter new orders in the system, when the order is entered, the system must auto-generate the order number.
Functional Design – This would elaborate on the requirements. So here screen mockups are made, interfaces/integration points with other systems are identified, open APIs are identified, reports, workflows et al are also etched out. So this effectively becomes the bible for the developer to start the design. Also in this document are mentioned, the product use cases and functional test cases.
Technical Design – Starting with functional specs, developer goes on to design the UI pages, business logic, API, reports etc. and in the end comes up with a document. And unit test cases should also be identified and documented here. Code – now things are easy, you know what to do, how to do? It’s just the matter of converting them into a workable code. So again the development team pitches in and cranks out the code.
Unit Test – what every developer hates to do?
System Test – Out of developers’ hand, the code moves into the hands of testers who start banging the system with all sorts of test scripts prepared out of the functional designs.
Here are some of the traits of the environment which is making us follow this methodology
• Large teams spread across geography
• Very huge product, with a generic behavior. What I mean here is the product is made to cater not one particular client or industry. It has a broad foot print and designed to work across many verticals. So idea is to embed as many features and flexibility as possible.
• Lot of integration points. One system integrates with at least 3-4 other systems for a complete business flow. So these integrating systems are again as complex as this system, so they would expect their integration points fairly very early in the game. That is why all this interface and API are decided during functional stage itself.
• Management get good artifacts to track the progress after each stage a 200 page word document
• Different management chain/organization strategy for the three roles – product manager, development, testing. So each of them have their own plans and time lines and schedules. So its quite natural that by a deadline testers expect the code to be ready so that they can start banging their head on the system.
• Each system can have millions of lines of code
In reality this methodology is working and riding the business, so it is fine as long money is coming to your coffers 😉 But it does make life painful for all stakeholders.
The first two stages are the fiefdom of one team, what they say is what is etched in the stone. But it depends on the competence of the guy in the business realm. If it’s a new domain or without much customer reference or the team is without industry exposures then things get stuck here in the requirement phase. A typical symptom is Analysis-Paralysis, where the whole team is bewildered as to what to progress, what system to build, and there are lurking doubts whether this is going to be accepted in the field? So what they typically do is incorporate all sets of wild requirements into the product and directly shoot in the feet. YAGNI (You are not going to need it) is not the mantra WNAOI (We need all of it) is the mantra. So all sets of weird requirements can creep in. But say if the domain is pretty straightforward and you have customer references at your disposal then this phase becomes a breeze and they can come up with the requirements and functional designs in a jiffy.
From there things get tricky. A functional design once gets completed gets review from developers and he heads on for making the next word document in pipeline the Technical Design.
Typical Time-allocation across the project would be something like this
So as you see always the poor developer guy is time-pressed to deliver the software, by the time he gets to know the requirements/functional designs and he reviews those documents and circles back with product managers to clarify it, his time would have started ticking away.
Typical difficulties faced by the developer
• Difficulty in imagining the whole system and coming up with the technical document
• Itching to jump into code by putting off the boring documentation job which is allotted anywhere between 3 weeks – 5 weeks time. It’s really difficult to hold the attention span for such a long time and imagine about code and write about it still not implement it. So what happens most of the times in TD and Coding starts happening hand-in-hand, many a time code first and document next. I feel it’s good to do that instead of coming up with a document lock-stock and barrel. But as the time progresses the deadline pressure looms and coding will be happening at a frenetic pace, but not documentation and design. In this frenzy of meeting deadlines detailed Unit Tests are thrown out of the window.
At the end we will have a
• Unstable code, which works very reasonably or sometimes very badly
• No documentation of design or limited and not up to date design documentation
• No unit test scripts
• Not much standards adherence to standards – what I meant was coding standards.
There are quite few standards enforced by the build system, like explicit checking of cursors/connections whether they have been closed. Using parameterized binding when passing arguments from Java to PL/SQL etc. Apart from that there is not much standards like
• Doing proper OOD
• Using Design patterns
• Inline comments etc.
Many a times what happens is that guys with procedural background comes in and writes very neat Procedural programs in Java, without realizing the power of objects. So the end result is quite guessable, system works but partially and starts bombing in lots of places.
So the originally envisaged System Test Plan schedules are goes hay wire and more and more rounds of System Tests are carried out to make the system stable.
As you see, the code moves from an unstable state to reasonably stable state thanks to multiple rounds of testing. So when it goes live in the customer place, you can expect many more bombs exploding giving sleepless nights to developers.
The biggest nightmare is for the new guy joining any team in this system. He is left on his own with no proper documentation to swim through the labyrinthine code mess to figure out what is happening. That’s the real pain point. So is this the classic waterfall – well I don’t think it’s so bad, it’s just a chaotic way of developing software. If this system is working it can be attributed to one simple reason the people – smart dynamic and energetic people who relish on thriving on this complexity and pushes himself hard to meet customer expectations and meet the deadlines.