SDLC Process Models

The simplest way to build software is to start coding and keep fixing what breaks, with no explicit stages at all. That approach, sometimes called code-and-fix, has no overhead and works well enough for a small program written by one person over a short period. It stops working as the software and the team grow. There is no way to tell how far along the work is, and no way to divide it among several people without them colliding. There is also no record of the decisions already made, so changes become harder and more expensive.

Software development goes through different stages such as requirements, analysis, design, implementation, and testing. These stages are collectively known as the software development lifecycle (SDLC). There are several approaches, known as software development lifecycle models (also called software process models), that describe different ways to go through the SDLC. Each process model prescribes a 'roadmap' for the development effort: the aims of the development stages, the outcome of each stage, and how the stages relate to one another.

Reaching users is not the end of the lifecycle. Deployment, operation, and maintenance are commonly counted as lifecycle activities too, and process models differ in how they partition and name them. Once the software is released, it has to be operated, kept working as its environment changes, and improved. What happens during that time feeds back into development: defects reported by users, the way the software is actually used, and requests for things it cannot yet do all become inputs to later work. Most software spends far longer in this state than it spent being built for the first time.

The sequential model, also called the waterfall model, views software development as a linear process, with the project progressing through the development stages in order. The name waterfall stems from how the model is drawn to look like a waterfall (see below).

When one stage of the process is completed, it produces some to be used in the next stage. For example, the requirements stage produces a comprehensive list of requirements to be used in the design stage.

A strict sequential model project moves only in the forward direction i.e., each stage is completed before starting the next. For example, once the requirements stage is over, there is no provision for revising the requirements later. In practice the model is often relaxed to let a stage send work back to the one before it, although doing so means redoing work that was already treated as finished.

This model can work well for a project that solves a well-understood problem, in which case the requirements can remain stable and the effort can be estimated accurately. Furthermore, as each stage has a well-defined outcome, progress is easy to track: it is visible from which stage the project is in. Progress within a long stage, which is where an overrun usually builds up, stays much harder to see.

However, real-world projects often tackle problems that are not well-understood at the beginning, which makes those projects unsuitable for this model. For example, target users of a software product may not be able to state their requirements accurately at the start of the project if they have not used a similar product before.

A second weakness is that feedback arrives late. Each stage's output is checked mainly by the stage that follows it, so integration and contact with real users come near the end. A mistake made while gathering requirements or designing therefore tends to surface at the point where going back to completed work costs the most.

The iterative model advocates producing the software by going through several iterations. Each iteration could go through all the stages of the SDLC, from requirements gathering to deployment.

Each iteration produces a new version of the product, building upon the previous version. Feedback from each iteration is factored into the subsequent iterations. For example, if an implementation task took longer than expected, the effort estimate for similar tasks in future iterations can be adjusted. Similarly, if a feature introduced in the current iteration was not well-received by target users, it can be removed or tweaked in the next iteration.

The two models divide a project along different lines. A sequential project is divided by activity: a stage is 'requirements' or 'testing', and it ends when that activity is finished for the whole product. An iterative project is divided into bounded cycles instead: an iteration runs through several activities and ends in a result the team can learn from. What each iteration is for is then a choice -- most often a slice of functionality, 'the part that does X', but it can equally be a component, a risky assumption, or a level of completeness across the whole product.

The iterative model can use a breadth-first or depth-first approach.

• In the breadth-first approach, an iteration evolves all major components and all functionality areas in parallel, producing a working product at the end of each iteration i.e., most features and most are updated in every iteration.
• In the depth-first approach, an iteration focuses on fleshing out only some components or some functionality area. Accordingly, early depth-first iterations might not produce a working product.

Iterating and incrementing are two different things, and most projects do both. To iterate is to rework something that already exists, in the light of feedback; to increment is to add to it. That is why the two are usually named together, as an iterative and incremental approach.

What an iteration delivers is called an increment: a usable improvement or addition to the product, not merely a new version of the code.

An iteration is only worth its overhead if it ends in evidence. Before starting one, decide what would show it succeeded -- a condition the result must satisfy, a test that must pass, or a demonstration to a target user -- and what decision the answer will drive. Without that, an iteration produces a new version and no new knowledge.

Example Taking a Minesweeper game as an example:

• breadth-first iterations will deliver a fully playable version early. These early versions may have primitive functionality, e.g. a rudimentary text-based UI, fixed board size, limited minefield layouts. This functionality (and the corresponding components) will then be improved in later iterations.
• an early depth-first iteration could deliver the full user interface (UI) but with no game logic at all. Alternatively, an early iteration could focus on just the logic for generating initial layouts of the minefield. Neither is a playable version of the game, but both can collect early feedback -- on the UI in the first case, on the minefield layouts in the second. That feedback then guides later iterations.

A project can be done as a mixture of breadth-first and depth-first iterations i.e., an iteration can contain some breadth-first work as well as some depth-first work, or some iterations can be breadth-first while others are depth-first.

Whichever shape the iterations take, an early one is a chance to find out you were wrong while changing course is still cheap. That makes the assumptions whose failure would cost the most -- an unproven technology, an unfamiliar user need, a performance target nobody has hit yet -- worth putting into an early iteration rather than a late one. Ordering iterations by risk in this way is the central idea of the spiral model.

As AI coding advances, producing a candidate implementation is becoming much cheaper than it used to be; deciding what to build and confirming that the result is correct have not. Within an iteration, that shifts the effort away from writing code and toward specifying and verifying. It does not reduce the value of being precise about what is wanted: a vague requirement that once produced a question from a teammate now produces a confident implementation of the wrong thing, quickly.

The agile approaches grew out of lightweight methods that were already in use. In 2001, a group of prominent software engineering practitioners -- among them the authors of several such methods -- met to articulate the values their approaches had in common. They were reacting against the documentation-driven, heavyweight processes used in most large projects at the time. The result was the agile manifesto.

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.
_{-- Extract from the Agile Manifesto}

The methods represented at that meeting, and later approaches built on the same values, are collectively called agile processes. Some of the key features of agile approaches are:

• Requirements are prioritized by user need, clarified with the whole team regularly (sometimes daily), and folded into the development schedule as they change.
• Planning and design stay light and keep evolving. Instead of a detailed design and a full project plan up front, the team works from a rough plan and a high-level design that evolves as the work goes on.
• The team shares responsibility for delivering the product, and reports progress openly to each other and to the user.

Many agile processes are in use today. Extreme Programming (XP) and Scrum are two well-known ones.

Agile approaches depend on conditions that are not always present: a customer available to give feedback continuously, and the ability to ship a change cheaply. Where those are missing -- a fixed-price contract with a signed-off scope, or software that must be certified before release -- an agile approach costs more than it returns.

The following description was adapted from the XP home page, emphasis added:

What makes XP 'extreme' is not the practices it uses but how often it uses them. Each one was already considered good; XP pushes each to the point where it happens continuously rather than in a scheduled phase:

• releases are small and frequent, rather than saved up for a milestone;
• code is integrated continuously, rather than merged near the end (continuous integration);
• tests are written before the code they test, rather than after (test-driven development);
• the design is refactored constantly, rather than in a cleanup phase;
• two programmers write the code together at one keyboard, so it is reviewed as it is written rather than in a scheduled review (pair programming).

That is the same argument iterative models make about the whole lifecycle, applied to individual development practices instead: shorten the gap between doing something and finding out whether it worked.

Scrum is a lightweight agile framework rather than a complete process. It fixes a small set of roles, events, and artifacts, and leaves the team to fill in the rest with practices of its own choosing. The description below follows the Scrum Guide.

A Scrum team has three accountabilities:

• The Product Owner, who represents the stakeholders and decides what the product needs next
• The Scrum Master, who is accountable for the team using Scrum well
• The Developers, a cross-functional group who do the analysis, design, implementation, and testing

A Scrum project is divided into short iterations called Sprints. Sprints are time-boxed (i.e., restricted to a fixed duration) at one month or less, and every Sprint in a project has the same length. One to four weeks is the common choice.

A Sprint contains all the work done in it, together with all its events. It opens with Sprint Planning, where the team selects the work and agrees on a Sprint Goal, and the Developers coordinate daily as the work proceeds. It ends with two distinct meetings: a Sprint Review, where the team and stakeholders inspect the Increment and decide what the product needs next, and a Sprint Retrospective, where the team inspects how it worked and chooses improvements. The next Sprint begins immediately after.

During each Sprint, the team creates a potentially deliverable Increment (for example, working and tested software). The work comes from the Product Backlog, a prioritized set of high-level requirements for the product as a whole. The items selected for the current Sprint form the Sprint Backlog.

Within a Sprint the Sprint Goal stays fixed, but the plan for reaching it does not. The team updates the Sprint Backlog as it learns more, and can renegotiate the scope with the Product Owner as long as the Sprint Goal survives. The Sprint must end on time; work that is not completed returns to the Product Backlog.

Scrum enables self-organizing teams, which rely on frequent and direct communication among all team members and disciplines rather than on documents handed from one to the next.

Scrum assumes that customers will change their minds about what they want (often called requirements churn) and that unforeseen problems cannot be planned for in advance. It therefore takes an empirical approach: instead of trying to define the problem fully up front, it maximizes the team's ability to deliver quickly and respond to requirements as they emerge.

The Daily Scrum is a short daily meeting in which the Developers synchronize their plans, surface whatever is blocking them, and decide what needs to be taken up separately. It is not a problem-solving meeting.
Example A common way to run it is for each member to say what they did since the previous Daily Scrum, what they plan to do next, and what is in their way.

No approach is best for every project; the choice depends on the project. These questions usually decide whether a project leans sequential or iterative:

• How well is the problem understood at the start? The less certain you are about what to build, the more you gain from delivering something early and learning from the response to it.
• How stable are the requirements likely to be? Requirements that will keep moving are expensive to freeze into an early document.
• How costly is a late change? Changing a web page after release is cheap; changing software already embedded in shipped hardware is not.
• Are users available to give feedback during development? Frequent feedback is what makes short iterations worth their overhead. Without it, the iterations still cost the overhead.
• Does anything outside the project demand signed-off documents? Contractual, regulatory, and safety-certification requirements can dictate stage-by-stage evidence regardless of what the team would prefer.
• How large and how experienced is the team? Coordinating many people, or people new to each other, needs more explicit structure than a small experienced team does.

These questions guide a choice; they do not compute one. Two reasonable teams can weigh them differently and both be right, and a project can combine approaches rather than adopt one wholesale.

Example Two projects, two defensible answers:

• Software controlling a car's braking system: the requirements are prescribed by safety regulations, changes after release are extremely expensive, and evidence for each stage has to be produced anyway. A largely sequential approach fits -- though a regulated project can equally run iteratively, producing the required evidence at each iteration.
• A new feature for a social media app: nobody knows yet which version users will prefer, and shipping a change costs little. Short iterations with real users fit.

The unified process was developed by the Three Amigos -- Ivar Jacobson, Grady Booch, and James Rumbaugh (the creators of UML).

The unified process consists of four phases: inception, elaboration, construction, and transition. The main purpose of each phase can be summarized as follows:

Phase	Activities	Typical Artifacts
Inception	Understand the problem and requirements Communicate with customer Plan the development effort	Basic use case model Rough project plan Project vision and scope
Elaboration	Refine and expand requirements Determine a high-level design e.g. system architecture	System architecture Various design models Prototype
Construction	Major implementation effort to support the use cases identified Design models are refined and fleshed out Testing of all levels is carried out Multiple releases of the system	Test cases of all levels System release
Transition	Ready the system for actual production use Familiarize end users with the system	Final system release Instruction manual

The diagram above visualizes a project done using the unified process (source: Wikipedia). The four phases run from left to right and each vertical column is a single iteration; the shading shows how much effort a given workflow, such as 'Requirements' or 'Analysis & Design', takes up in that iteration.

Phases and iterations are two different things, which is why a project can be phased and iterative at the same time: a phase says where the project is in its overall arc, while an iteration is one pass of work through several workflows. A phase is also not a development stage: each phase contains work from several stages.

The unified process is a customizable framework rather than a single fixed process. For example, the number of iterations in each phase, the definition of workflows, and the intensity of a given workflow in a given iteration can be adjusted according to the nature of the project. Therefore, the diagram above simply records a particular application of the UP rather than prescribes how the UP is to be applied. However, this record can be refined and reused for similar future projects.

A team's process is itself something that can be assessed and deliberately improved, rather than chosen once and left alone. Frameworks exist for doing that assessment, and they matter most where a customer wants evidence that a supplier can be relied on before awarding it work.

CMMI (Capability Maturity Model Integration), defined by the Software Engineering Institute at Carnegie Mellon University, is the best known of them. It defines five maturity levels, ranging from ad hoc to continuously improving, and criteria for determining which one an organization's process has reached.

The three approaches differ in how a project is divided, and in when it finds out whether it is on track.

• A sequential project is divided by activity. A stage ends when that activity is finished for the whole product. Progress is easy to track, but validation arrives near the end, so late changes are expensive. It suits a well-understood problem.
• An iterative project is divided into bounded cycles. Each iteration runs through several activities and ends in a result to learn from, so mistakes surface early -- at the cost of the overhead every cycle carries. It suits a problem that is still being understood.
• An agile approach keeps those cycles short and the customer close. It trades detailed up-front planning for responsiveness, which pays off when requirements keep moving and users are available to give feedback, and costs more than it returns when they are not.

The example models are particular ways of doing this: XP pushes individual development practices to happen continuously, Scrum fixes a small set of accountabilities, events and artifacts around short Sprints, and the Unified Process runs four phases, each as one or more iterations.

The exercises below cover several of these topics together.