Forget Requirements - Collaborate on a Solution Concept

Requirements in systems development have always been a difficult area. In the Standish Group Chaos Report issues with requirements always appear in the top 3 entries of reasons for project failure.

With this in mind there tends to be a management emphasis on "getting the requirements right", before committing to any form of development or implementation. Yet I've experienced numerous projects where hundreds, if not thousands, of man hours have been devoted to requirements, and still solutions have not met expectations. I suspect anyone reading this has also experienced similar projects. So why is requirements management so often poorly executed?

You often hear people talk about traceability, configuration & change control, use cases, process models etc, etc. Management will throw Process Improvement, Quality Teams and frameworks such as CMMI at the problem.

For me there are some 'home truths' about requirements which make the task. if tackled in the 'traditional way', near on impossible:

• The majority of IT programmes are driven 'top down' with very scant definition of what's required, usually some vague goals - if you're lucky.
• Stakeholders that will actually have to use the system are often not engaged until the end of the life cycle - if at all.
• Stakeholders and sponsors usually change during the project life cycle, along with their expectations and, therefore, the requirements.
• Users cannot often express their needs in terms that can be easily translated into system specification
• Management and users usually have no understanding of the constraints or capabilities of the technologies. They ask for features that are infeasible or uneconomic to implement or, at the other end of the extreme, they don't ask for features which would be simple to deliver because they don't realise they can
• Management ask for 'signed off' requirements documents, yet no-one ever reads them, let alone understands them.
• Business processes, rules and taxonomy are 'fuzzy', ill-defined and not agreed upon by stakeholders
• Stakeholders will keep changing their minds and usually come up with conflicting requirements
• Users and management usually cannot see a business process working any different to how it works now, resulting in lost opportunities for IT driven improvement.

For a good example, I was working in an Investment Bank on an Asset Management system. I remember a workshop where we were trying to detail the business rules of a particular financial instrument. When we got to the real nitty gritty of how these rules worked, the guy who was the SME in this instrument said "...the system calculates all that". It turned out in the end that very few people in the business understood the detail as it had all been encoded in a Mainframe system that had been there longer than their time in the company! Cue the development team spending man months reverse engineering 1000's of lines of ADABAS code!

I could go on, but you get the idea. Basically the traditional approach encouraged by the Waterfall life cycle and heavyweight methods such as PRINCE2, SSADM and, to an certain extent RUP don't deliver the goods in the majority of projects.

I believe a big part of the problem is a requirement can end up being anything from a high level business objective, e.g. the system shall reduce the claim process time by n%, to a specific system requirement, e.g all buttons shall be blue, and variations in between. In theory the requirements analysis process should weed these issues out. But it rarely does due to the simple fact that requirements are being captured in, what I call, a 'solution architecture vacuum', i.e. they can't be validated against any form of system implementation view that sense checks their feasibility. This process can continue until your project is overflowing with requirement statements and process models and the whole project ends up in Analysis Paralysis.

What's the solution? Well, there's a lot of talk in the industry about Agile, in fact so much so it's become an industry in itself and possibly well on its way to becoming an oxymoron. I have seen very few organisations truly embrace an Agile approach, mainly due to management culture and vested interests, but that's another article.

In my view if organisations want to improve their approach to systems delivery then they really need to drop the idea of requirements management altogether, at least in the traditional sense of doorstop URDs, SRDs, Use Cases, endless Workshops and incomprehensible Process Models.

A fresh approach is required that is focused, not on requirements, but the solution, right at the start of the project life cycle. A overview of approach is shown below.

The approach is, of course, Agile, but adding the concept of an Increment or Micro-Increment on top of an Iteration. Increments should be measured in days, yet still deliver some demo-able or executable software to stakeholders. Micro-Increments are important as they drive projects to meet short term goals that are focused on software delivery, even if it's a simple as a dumb HTML UI mock-up, this adds infinitely more value that lines of requirements text or use cases.

Inputs to the Solution Concept include:

• Available Technology Components - ensure you base your architecture on components and technologies you're confident you can readily develop and deploy. Look for maximum reuse, both in the small, e.g. Java persistence frameworks, and in the large, e.g. packaged COTS modules such as ERP and CRM
• Application Architecture Patterns - very few business systems are entirely new, in all probability elements of the solution you're trying to build have been built and proven. Don't waste time reinventing wheels, leverage these patterns
• Legacy Systems - this may be both systems that your solution will replace and systems you'll need to interface to or extract data from. It also includes manual systems, paper forms and any 'home grown' end user solutions, usually based upon desktop tools such as Excel and Access. Don't dismiss these by the way, I've repeatedly come across some pretty impressive solutions built by keen amateurs!
• Business Goals & Objectives - understand what the business is trying to achieve and what a successful system looks like. The more you can immerse yourself in the users problem from their perspective, the better chance you have off building a great solution. More often than not, you uncover whole areas of requirements that users have not even thought about.
• Programatics & Risk - ensure the budget and desired time scales are baked into the solution design at the start. There's no point designing a solution that's going to take 2 years when the stakeholders need something now! On the point of schedules, it's my view that if the solution is going to take longer that 9 months to go-live then you should either (i) reduce scope (ii) break the solution up into smaller elements or (iii) forget it! In my experience any information system that takes longer than 9 months to get deployed is likely to be pretty useless as the organisation will of moved on. The rule here is, the faster you can deploy solutions to production the better

Once you start to lay your hands on these inputs, the Increments themselves are all about getting stuff built! Yes you need to maintain some documentation, but keep it lightweight and value add.

• UI Prototypes - use cases are okay, but there's no substitute for putting, at least what looks like, a real solution in front of stakeholders. In my experience UI prototypes validate system requirements better that any process modelling or workshops could ever do.
• Demo-able Solution - if it's feasible to build some form of functional prototype within the bounds of an Iteration, then you should. Focus on the most complex or least understood area of the solution first.
• Architecture Prototype - sense check your technology stack, runtime topology and non-functionals as early as you can. Often these issues constraint the functionality than can be implemented. For example, you may be able to do some fancy stuff in the Browser with a Plug-In, but the Corporate firewalls block the port it uses. You want to find these issues out right at the start of the project before you comit to the architecture.
• Candidate Feature List - as you're producing these prototypes and getting feedback from stakeholders, you'll start to get 'real' useful requirements, that are in context of a system. I don't call these requirements, I call them system features as they are tied to the architecture. These features should be unequivocally understood by the development team how they work, what good looks like and potential approaches to implementation and test.

In my experience, once a project gets into a 'groove' of running Increments continuous throughout the life cycle of the delivery, then the whole process becomes self reinforcing through better defined features, improved prototypes etc. In fact, the process doesn't really change from inception to the final go-live, prototypes gradually move to alphas, betas, pre-release versions, release candidates then a final decision to promote a release candidate live.

So, stop talking about requirements, and start building Solution Concepts.

If you're looking for further useful info on this approach then I'd recommend:

• Eclipse EPF - an Eclipse project focused on a Open Source lightweight development approach based on IBM's RUP but stripped to the core.
• Feature Driven Development - promotes a project delivery approach called FDD based on features. There's also a book available on FDD.
• Introduction to Features - definition from Scott Ambler as to what a good Feature looks like.
• Agile Manifesto - and finally, keep this web page on your browser at all times to remind you what your job is!