Secure Software Engineering

This is a printer-friendly version. It omits exercises, optional topics (i.e., four-star topics), and other extra content such as learning outcomes.

What

Software now handles personal information, money, communication, transportation, education, health, and many other parts of daily life. A defect in such software can do more than inconvenience a user: it can expose information, allow unauthorized actions, or make an important service unavailable.

Security is therefore part of software engineering, not a specialist activity added after the software is finished; this approach is called secure by design. Every software engineer needs enough security knowledge to recognize common risks, make safer design and implementation decisions, and know when expert help is needed.

This textbook does not attempt to teach every kind of attack. Instead, it develops one reusable method that applies broadly.

Software security is the protection of a system and its stakeholders from misuse and harm, whether deliberate or accidental. A conventional defect might be triggered accidentally. In contrast, a security weakness may be deliberately searched for and exploited by someone who can choose the inputs, actions, timing, and sequence most favorable to an attack. Security analysis assumes that deliberate case, because an attacker exercises a feature far harder than an accident usually does. Accidental causes still need attention of their own, because a misconfiguration, an operator mistake, or a corrupted record need not resemble an attack.

Example We will use a university event-registration system as the running example throughout the related topics of this textbook. Students can view events and register themselves. Organizers can create events and view attendee lists. The system stores names, email addresses, registrations, and organizer privileges.

Testing whether an ordinary user can register for an event is necessary, but it is not enough. We must also ask whether one student can view another student's registration, whether a non-organizer can obtain an attendee list, and whether an attacker can submit enough costly requests to make registration unavailable to everyone else.

The security mindset

The security mindset means questioning assumptions and considering how a feature could be deliberately misused. When implementing a feature, ask both:

  • Does the intended workflow work?
  • What are we trusting, and what could someone do if that trust is misplaced?

Example Suppose a browser sends the following data when a user requests an attendee list:

eventId = 42
isOrganizer = true

The value eventId identifies the requested event. However, isOrganizer is merely a claim made by software under the user's control. Hiding the attendee-list button from ordinary users does not stop them from constructing the request themselves. A security-minded engineer treats the flag as untrusted and asks where the user's authority is established.

The security mindset is not the belief that every user is malicious. It is the recognition that software must remain safe when inputs are mistaken, unusual, corrupted, or deliberately hostile. The same controls often protect against both accidents and attacks.

Three basic security goals

Security is broader than secrecy. Protecting information from disclosure matters, but so do preventing unauthorized changes, preserving service availability, and ensuring that actions are performed by the right people.

Three of these are security properties, together called the CIA triad. Establishing that actions are performed by the right people is not a fourth property; it is a mechanism used to protect all three. The triad gives three useful questions to ask:

  • Confidentiality asks whether information has been disclosed only to those permitted to see it.
    Example An attendee's email address should not be disclosed to another student without a valid reason and permission.

  • Integrity asks whether information and behavior have remained correct, complete, and free from improper change or destruction.
    Example A student should not be able to cancel another student's registration or grant themselves organizer privileges.

  • Availability asks whether authorized users can obtain the service when they need it.
    Example The registration service should remain usable during a popular event's sign-up period.

A single incident can affect more than one goal.
Example A compromised organizer account might expose attendee details, alter registrations, and delete events.

A small vocabulary

The following terms let a team discuss security precisely:

  • Asset: something worth protecting.
    Example Assets include data, account privileges, service availability, money, physical resources, and reputation.
  • Stakeholder: a person or group that can benefit or suffer from the system's security decisions.
    Example Students, organizers, system operators, and the university are stakeholders in the event system.
  • Threat: a possible way an asset could be harmed.
    Example A student reading another student's registration is a threat to confidentiality.
  • Vulnerability: a weakness that allows a threat to be realized.
    Example Failing to check which student owns a registration is a vulnerability.
  • Attack: an attempt to exploit a vulnerability.
    Example Changing a registration identifier in a request to retrieve someone else's record is an attack.
  • Control or Mitigation: something that reduces the likelihood or impact of a threat.
    Example Checking ownership before returning a registration is a control.
  • Authentication: establishing which identity is making a request.
    Example Signing in as a particular student.
  • Authorization: deciding whether that identity may perform a specific action on a specific resource.
    Example Checking that this student owns the registration they asked to see.
  • Risk: the likelihood of a threat combined with the seriousness of its impact.
    Example A student reading another student's registration is a high risk: it is easy to attempt and it exposes personal data.
  • Misuse case: a short scenario describing how someone could deliberately use the system to cause harm.
    Example A student changes an event identifier to download another event's list.

These terms describe different parts of one situation. A valuable database is not a vulnerability. A possible theft is not an attack until someone attempts it. A control can reduce risk without eliminating it.

Security and privacy

Privacy is related to, but distinct from, security. Security asks whether information and capabilities are protected from unauthorized use. Privacy also asks whether collecting, using, retaining, and sharing personal information is appropriate in the first place.

Data that is never collected cannot later be leaked by that system. Strong access control cannot justify collecting data the system does not need.
Example The event system might need an attendee's name and contact address, but probably not their date of birth.

Security is risk management

Perfect security is not achievable; secure engineering is risk management. A useful security claim must say what is being protected, from whom, under what assumptions, and to what degree.

Example Encrypting database backups, with keys held separately, can reduce the harm caused by someone stealing a backup. It cannot protect attendee data from an organizer who is legitimately permitted to view it. A rate limit can make an automated denial-of-service attack more expensive, but a sufficiently large attack may still overwhelm the system.

Security also has costs. A control can consume development time, reduce performance, make a system harder to use, or create new failure modes. The goal is not to add every possible control. It is to identify important risks and choose controls whose benefits justify their costs.

Some names you will encounter

You do not need to memorize a catalog of vulnerabilities, but a few common labels are useful:

  • Broken access control allows someone to perform an action or access data they are not authorized to use.
  • Injection occurs when a system interprets untrusted data as code or commands.
  • Cross-site scripting (XSS) occurs when a web application causes untrusted content to execute as a script in another user's browser.
  • Vulnerable or outdated components expose a system to known weaknesses in reused software.

Each label has a matching engineering practice, and this textbook covers all four: enforcing authorization on every protected action, keeping untrusted data separate from commands, encoding output for the context it is placed into, and treating dependencies as part of the product.

The OWASP Top 10 is a widely used awareness list for web-application risks. It is a useful pointer for further study, but it is not a complete model of software security and should not replace thinking about the specific system in front of you.

Why

Software can pass its normal tests and still be insecure. Functional tests usually ask whether expected users can perform expected actions. Security also asks what happens when someone deliberately uses unexpected inputs, identities, permissions, sequences, and volumes.

Example Consider a registration page that correctly displays registration 381 to its owner:

/registrations/381

If changing the address to /registrations/382 reveals another student's record, the feature works on its happy path but has broken access control. The server checked that the requester was signed in; it did not check that this record belonged to them.

Authentication and authorization answer different questions: authentication establishes who is making a request, and authorization decides whether that identity may perform this particular action on this particular resource.

A user can be correctly authenticated and still be unauthorized. In most systems, authorization must be checked for every protected action, not just when the user first signs in.

Security failures cause real harm

A security failure can harm people who never chose to accept the risk. Exposed personal data can lead to harassment or fraud. Altered records can cause financial or academic consequences. An unavailable service can exclude users from something time-sensitive. Compromised software can also be used to attack other systems.

The engineer who writes a small part of the program may not see these consequences directly. Nevertheless, a missing permission check, leaked credential, or unsafe library call can bypass otherwise sound requirements, design, and testing.

Security problems also become more expensive after release. Correcting one line of code may be the easy part. The response may also require investigating what happened, recovering data, revoking credentials, updating dependencies, deploying urgently, notifying affected people, and rebuilding trust.

Attackers can exploit any undefended path

An attacker needs one usable path to an asset, while defenders must protect every reachable path that matters.

Example A team may secure the main web page but forget an older API, an import feature, an administrator script, or a default account. The forgotten path can be enough.

That asymmetry is why security work concentrates on reducing the number of paths, covering each of them consistently, and putting more than one control around anything that matters.

Security includes people and processes

A control that people cannot use correctly will often be bypassed.

Example If secure setup is much harder than insecure setup, developers will postpone it. If every harmless action triggers an alarming warning, users will learn to ignore warnings. If a password policy makes passwords impossible to remember, users may record them somewhere unsafe.

Software engineers do not control every human decision, and this textbook does not teach in depth. However, engineers influence interfaces, workflows, defaults, documentation, and operational procedures. Good security makes the safe action understandable and practical.