Software now handles personal information, money, communication, transportation, education, health, and many other parts of daily life. A defect in such software can do more than inconvenience a user: it can expose information, allow unauthorized actions, or make an important service unavailable.
Security is therefore part of software engineering, not a specialist activity added after the software is finished; this approach is called secure by design. Every software engineer needs enough security knowledge to recognize common risks, make safer design and implementation decisions, and know when expert help is needed.
This textbook does not attempt to teach every kind of attack. Instead, it develops one reusable method that applies broadly.
Software security is the protection of a system and its stakeholders from misuse and harm, whether deliberate or accidental. A conventional defect might be triggered accidentally. In contrast, a security weakness may be deliberately searched for and exploited by someone who can choose the inputs, actions, timing, and sequence most favorable to an attack. Security analysis assumes that deliberate case, because an attacker exercises a feature far harder than an accident usually does. Accidental causes still need attention of their own, because a misconfiguration, an operator mistake, or a corrupted record need not resemble an attack.
Example We will use a university event-registration system as the running example throughout the related topics of this textbook. Students can view events and register themselves. Organizers can create events and view attendee lists. The system stores names, email addresses, registrations, and organizer privileges.
Testing whether an ordinary user can register for an event is necessary, but it is not enough. We must also ask whether one student can view another student's registration, whether a non-organizer can obtain an attendee list, and whether an attacker can submit enough costly requests to make registration unavailable to everyone else.
The security mindset means questioning assumptions and considering how a feature could be deliberately misused. When implementing a feature, ask both:
Example Suppose a browser sends the following data when a user requests an attendee list:
eventId = 42
isOrganizer = true
The value eventId identifies the requested event. However, isOrganizer is merely a claim made by software under the user's control. Hiding the attendee-list button from ordinary users does not stop them from constructing the request themselves. A security-minded engineer treats the flag as untrusted and asks where the user's authority is established.
The security mindset is not the belief that every user is malicious. It is the recognition that software must remain safe when inputs are mistaken, unusual, corrupted, or deliberately hostile. The same controls often protect against both accidents and attacks.
Security is broader than secrecy. Protecting information from disclosure matters, but so do preventing unauthorized changes, preserving service availability, and ensuring that actions are performed by the right people.
Three of these are security properties, together called the CIA triad. Establishing that actions are performed by the right people is not a fourth property; it is a mechanism used to protect all three. The triad gives three useful questions to ask:
Confidentiality asks whether information has been disclosed only to those permitted to see it.
Example An attendee's email address should not be disclosed to another student without a valid reason and permission.
Integrity asks whether information and behavior have remained correct, complete, and free from improper change or destruction.
Example A student should not be able to cancel another student's registration or grant themselves organizer privileges.
Availability asks whether authorized users can obtain the service when they need it.
Example The registration service should remain usable during a popular event's sign-up period.
A single incident can affect more than one goal.
Example A compromised organizer account might expose attendee details, alter registrations, and delete events.
The following terms let a team discuss security precisely:
These terms describe different parts of one situation. A valuable database is not a vulnerability. A possible theft is not an attack until someone attempts it. A control can reduce risk without eliminating it.
Privacy is related to, but distinct from, security. Security asks whether information and capabilities are protected from unauthorized use. Privacy also asks whether collecting, using, retaining, and sharing personal information is appropriate in the first place.
Data that is never collected cannot later be leaked by that system. Strong access control cannot justify collecting data the system does not need.
Example The event system might need an attendee's name and contact address, but probably not their date of birth.
Perfect security is not achievable; secure engineering is risk management. A useful security claim must say what is being protected, from whom, under what assumptions, and to what degree.
Example Encrypting database backups, with keys held separately, can reduce the harm caused by someone stealing a backup. It cannot protect attendee data from an organizer who is legitimately permitted to view it. A rate limit can make an automated denial-of-service attack more expensive, but a sufficiently large attack may still overwhelm the system.
Security also has costs. A control can consume development time, reduce performance, make a system harder to use, or create new failure modes. The goal is not to add every possible control. It is to identify important risks and choose controls whose benefits justify their costs.
You do not need to memorize a catalog of vulnerabilities, but a few common labels are useful:
Each label has a matching engineering practice, and this textbook covers all four: enforcing authorization on every protected action, keeping untrusted data separate from commands, encoding output for the context it is placed into, and treating dependencies as part of the product.
The OWASP Top 10 is a widely used awareness list for web-application risks. It is a useful pointer for further study, but it is not a complete model of software security and should not replace thinking about the specific system in front of you.
Software can pass its normal tests and still be insecure. Functional tests usually ask whether expected users can perform expected actions. Security also asks what happens when someone deliberately uses unexpected inputs, identities, permissions, sequences, and volumes.
Example Consider a registration page that correctly displays registration 381 to its owner:
/registrations/381
If changing the address to /registrations/382 reveals another student's record, the feature works on its happy path but has broken access control. The server checked that the requester was signed in; it did not check that this record belonged to them.
Authentication and authorization answer different questions: authentication establishes who is making a request, and authorization decides whether that identity may perform this particular action on this particular resource.
A user can be correctly authenticated and still be unauthorized. In most systems, authorization must be checked for every protected action, not just when the user first signs in.
A security failure can harm people who never chose to accept the risk. Exposed personal data can lead to harassment or fraud. Altered records can cause financial or academic consequences. An unavailable service can exclude users from something time-sensitive. Compromised software can also be used to attack other systems.
The engineer who writes a small part of the program may not see these consequences directly. Nevertheless, a missing permission check, leaked credential, or unsafe library call can bypass otherwise sound requirements, design, and testing.
Security problems also become more expensive after release. Correcting one line of code may be the easy part. The response may also require investigating what happened, recovering data, revoking credentials, updating dependencies, deploying urgently, notifying affected people, and rebuilding trust.
An attacker needs one usable path to an asset, while defenders must protect every reachable path that matters.
Example A team may secure the main web page but forget an older API, an import feature, an administrator script, or a default account. The forgotten path can be enough.
That asymmetry is why security work concentrates on reducing the number of paths, covering each of them consistently, and putting more than one control around anything that matters.
A control that people cannot use correctly will often be bypassed.
Example If secure setup is much harder than insecure setup, developers will postpone it. If every harmless action triggers an alarming warning, users will learn to ignore warnings. If a password policy makes passwords impossible to remember, users may record them somewhere unsafe.
Software engineers do not control every human decision, and this textbook does not teach in depth. However, engineers influence interfaces, workflows, defaults, documentation, and operational procedures. Good security makes the safe action understandable and practical.
Secure software results from combining threat modeling, secure design, careful implementation, and verification. It cannot be obtained by applying one tool or adding one library.
Threat modeling is a structured way to ask what needs protection, what could go wrong, and what should be done about it. It can be performed with a whiteboard, a short document, or comments on a design diagram. The value comes from the questions, not the format.
Use the following six-step method:
Start with what matters, not with a list of attacks. Ask:
Example For the event system, assets include attendee details, registration records, organizer privileges, and availability during sign-up periods.
A trust boundary is a place where data or control passes between parts with different levels of trust. Examples include a browser calling a server, a server reading an uploaded file, an application querying a database, and a build process downloading a dependency.
Example A simple sketch of the event system is enough:
student's browser
|
| untrusted request
= = = = | = = = = = = = = = = = = = = = = = = trust boundary
v
registration server - - - - -> email service
| (boundary: another
| database query organization's service)
= = = = | = = = = = = = = = = = = = = = = = = trust boundary
v
registration database
Each marked line is a place where trust changes. Data or control crossing one, in either direction, has to be treated according to the trust level on the receiving side.
Do not label the entire "inside" of a system as trusted without thought. A database can contain malicious text entered earlier by a user. A partner service can be unavailable or compromised. A configuration file can be edited by someone with different privileges. Trust should be earned at each boundary, not inherited forever from where data was first seen.
Ask how someone without permission could read, change, or exhaust each important asset; impersonate an identity; or bypass a check that protects it. Consider both outsiders and legitimate users exceeding their authority.
Example For the attendee list:
STRIDE (Developed by Praerit Garg and Loren Kohnfelder at Microsoft) is one optional checklist for teams that want named categories. It can help prompt questions, but it is not necessary for a useful threat model.
| Category | The question it prompts | In the event system |
|---|---|---|
| Spoofing | Can someone claim to be another user or service? | Signing in as an organizer with a stolen password |
| Tampering | Can someone change data or code they should not? | Editing a registration record through an unprotected endpoint |
| Repudiation | Can someone deny having acted, with no record to contradict them? | Deleting an event with nothing recorded about who did it |
| Information disclosure | Can someone read what they should not? | Reading another event's attendee list |
| Denial of service | Can someone make the service unusable? | Flooding registration during a popular sign-up period |
| Elevation of privilege | Can someone gain rights they were never granted? | A student making themselves an organizer |
Spend effort where plausible threats could cause serious harm. Exact numerical estimates are rarely available, especially early in a project. A simple scale such as low, medium, and high can still support useful decisions if the team records its assumptions.
Estimate likelihood by considering whether the feature is reachable, whether exploitation requires special access, and how easily the attack can be repeated or automated. Estimate impact by considering the sensitivity and amount of affected data, the privileges at risk, the number of people affected, whether the damage is reversible, and how long a service could be unavailable.
Combine the two ratings using a rule the team agrees on in advance, so that two people analyzing the same threat reach the same answer:
| Likelihood ↓ / Impact → | Low | Medium | High |
|---|---|---|---|
| High | Medium | High | High |
| Medium | Low | Medium | High |
| Low | Low | Low | Medium |
The table is a starting point, not an answer. A team may override a cell, but it should record why.
Example Three event-system threats, rated on that scale with the assumption behind each rating recorded:
| Threat | Likelihood | Impact | Risk | Response |
|---|---|---|---|---|
| A student changes an identifier to read another event's attendee list | High: reachable by anyone signed in, and trivial to repeat | Medium: names and email addresses for one event | High | Check ownership on every request |
| A compromised organizer account grants organizer rights to others | Low: requires an account compromise first | High: privileges spread, and the change is hard to notice | Medium | Log privilege changes and allow them to be revoked |
| Automated sign-ups flood registration during a popular event | Medium: needs no account, and is easy to script | Medium: service lost during the window that matters, but recoverable | Medium | Limit how often one source can attempt registration |
The first threat is rated highest even though its impact is not the largest, because it needs no special access and can be repeated at will.
Do not spend most of the security budget on an exciting but implausible attack while ignoring ordinary permission checks and exposed credentials.
Prefer controls that prevent a problem, while preparing to detect, contain, and recover from failures that prevention misses. Also prefer a control people can use correctly: one that is routinely worked around protects nothing.
Example For an attendee list, controls might include:
One control can address several threats, and one threat may need several controls.
Not every risk gets a control. A risk can be reduced with a control, removed by dropping the feature or the data that creates it, or accepted as a cost of doing the useful thing. Accepting is a legitimate decision, but only when it is recorded with its reason, its owner, and the date when it will be reconsidered. An unrecorded acceptance is indistinguishable from an oversight.
A control is only a claim until it has been reviewed or tested. Verify the paths an attacker would try, not only the intended path.
Example For the attendee list, the correct organizer is the least informative case to test. The cases that decide whether the control works are the ones it is supposed to refuse.
Record the assumptions on which the controls depend. Re-rate the risk with the controls in place, using the same matrix, and record what remains and who accepts it. Keep the threat model near the design or feature it describes so that it can be revisited when the system or those assumptions change.
A concise statement can connect an asset, security goal, adversary, and assumption:
<Asset>should retain<security property>against<adversary or misuse>under<relevant assumptions>.
Example Attendee email addresses should remain confidential from other students, even if they change identifiers in a request.
This statement is more useful than "the attendee list must be secure" because it can guide both design and testing.
The last two steps carry most of the work, so the rest of this topic expands them. Sections B to F cover how to choose and implement controls; section G covers how to verify them.
Apply the following six design principles:
Least privilege means giving each user, component, and process only the permissions it needs for its current job.
Example An event organizer needs access to their own event's attendee list, not every event's list. A component that sends email does not need permission to change registrations. A build job that only reads source code does not need production database credentials.
Smaller privileges reduce both the chance of misuse and the damage caused by a compromised component.
Deny by default means access is refused unless a rule explicitly permits it. When a new action, role, or resource is added, it should begin inaccessible rather than accidentally available to everyone.
Example Prefer:
if user may view this event:
return attendee list
otherwise:
deny
over a growing collection of special cases that attempt to identify everyone who should be denied.
An allow rule is usually easier to review than an incomplete list of forbidden cases.
Deny by default is the access-control case of a wider principle, secure by default: a system should be safe in the configuration it ships with. The rest of that principle covers an installation's initial state: no default accounts or passwords, debug and diagnostic interfaces turned off, network exposure limited to what is needed, and data retained no longer than necessary.
Minimizing the attack surface means reducing the features, interfaces, data, permissions, and dependencies available to be attacked. Remove unused endpoints, default accounts, debug interfaces, old code paths, unnecessary data fields, and unneeded packages.
Every extra part has a maintenance cost even if it currently appears harmless. A feature that does not exist does not need to be configured, patched, monitored, or defended.
Defense in depth means protecting an important asset with several independent controls, so that one failure is not enough.
Example An attendee list can be protected by an application permission check, by database permissions that let the reporting account read only the rows it needs, and by access logging that lets a reviewer notice an unusual read.
A second control limits or reveals the damage when the first is missing, misconfigured, or bypassed.
When something goes wrong, the failure should deny rather than allow. An error should not grant access, skip validation, or reveal sensitive internals.
Example If the check that answers "is this user an organizer of this event?" fails or times out, the attendee list request should be refused rather than served.
Denying on failure trades availability for the other two goals, so a check that denies when it fails needs availability attention of its own.
Authentication, password storage, encryption, and session management are easy to get subtly wrong, so prefer mechanisms many people have already reviewed and attacked.
Example Use the authentication and session support your stack already maintains, such as Spring Security for Java, Django's authentication system for Python, or a maintained session middleware for Node.js. The common alternative is to invent a scheme that stores a user identifier in a cookie and trusts it on the next request. It fails because that cookie is under the client's control and carries nothing the server can verify.
Untrusted data includes more than text typed into a form. Requests, file uploads, command-line arguments, environment variables, configuration, database records, messages, dependency metadata, and responses from other services can all cross trust boundaries.
Untrusted data creates three separate responsibilities: validate it for its intended use when it enters, keep it separate from instructions wherever it reaches an interpreter, and encode it for each output context it is placed into.
Validation checks whether data is acceptable for its intended purpose. Check its type, form, length, range, and meaning as early as practical.
Example An event identifier might need to be a positive integer referring to an existing event. An event name might need a reasonable maximum length. A start time might need to precede an end time.
Prefer an allowlist, which describes acceptable data, over a denylist of known-bad strings. Attackers can often express the same harmful meaning in forms a denylist did not anticipate: a rule that rejects <script is bypassed by <ScRiPt, by <img onerror=...>, and by encoded forms the browser decodes only after the check has run.
Client-side validation can improve usability, but it is not a security boundary because a client can be modified or bypassed. Validation that protects server-side assets must also occur on the trusted side of the boundary.
Do not construct executable commands by concatenating untrusted strings. When an interpreter cannot distinguish data from instructions, an attacker may be able to change what gets executed.
Example This query construction is unsafe:
query = "SELECT student_name, email FROM registrations WHERE event_id = " + eventId
database.execute(query)
Use an API that sends the instruction and the data separately:
query = "SELECT student_name, email FROM registrations WHERE event_id = ?"
database.execute(query, [eventId])
The same principle applies to operating-system commands, templates, directory services, and other interpreters. Prefer APIs that accept structured arguments. Escaping a manually assembled command is usually harder to get right.
Before placing untrusted data into structured output, encode it for that specific destination context. Text that is harmless in one context may become executable in another.
Example Suppose an organizer enters this as an event name:
<script>performUnwantedAction()</script>
Rendering it as raw HTML could cause the browser to execute it:
page.setRawHtml(event.name) // unsafe for untrusted text
Rendering it through an ordinary text API preserves the visible characters without treating them as markup:
page.setText(event.name) // preferred for plain text
The attack this prevents is cross-site scripting (XSS): content supplied by one user is executed as a script in another user's browser.
HTML text, HTML attributes, URLs, scripts, styles, and other destinations have different encoding rules. Prefer framework features that encode safely by default, and avoid raw-output features unless the content comes from a source the system itself controls. An authenticated user is not such a source: the organizer in the example above was signed in. When a feature genuinely requires user-authored markup, neither plain encoding nor raw output will do: run the content through a maintained HTML sanitizer that keeps a known set of tags and attributes and discards the rest.
Input validation does not replace output encoding. A valid event name can legitimately contain characters with special meaning in HTML. Parameterized queries do not replace output encoding either; the two controls protect different boundaries.
Derive identity from an authentication mechanism the trusted part of the system can verify. Do not trust a user identifier, role, or isAdmin flag merely because the client sent it.
After identifying the requester, authorize the particular action on the particular resource. "This user is an organizer" may still be insufficient; the relevant rule might be "this user is an organizer of this event."
Example A safer attendee-list operation has this shape:
function getAttendees(request, session):
user = authenticatedUserFrom(session)
event = findEvent(request.eventId)
if event does not exist:
return notFound
if user is not an organizer of event:
return denied
return only the attendee fields this organizer needs
Distinguishing not found from not permitted is safe here, because any student may see that an event exists. When the existence of the resource is itself sensitive, such as one student's registration, return the same response for both and record the difference only in the server's own logs.
Authorization belongs on the trusted side. A disabled button, hidden menu item, or client-side route guard is a usability feature, not sufficient protection.
Apply the same authorization rule to every path to the action. If an operation is available through a web page, mobile endpoint, import feature, and administrator tool, a forgotten path can bypass a check applied only in the user interface.
A secret should be available only to the people and components that need it. API keys, private keys, access tokens, and database passwords should not be hard-coded, committed to version control, pasted into AI prompts, included in screenshots, or written to logs.
Keep a secret in a store built for the purpose, and let the deployment supply it to the process that needs it. Depending on the environment, that store is a CI/CD secret store, the platform's own protected configuration, or a dedicated secret manager. In each case, the deployment injects the value at run time, and only authorized identities and components can read it. Environment variables and untracked .env files are a common way to receive an injected secret, but they are not themselves protected storage: their contents can reach logs, crash dumps, and child processes, and keeping a file out of version control avoids only one leak path.
Give each credential the smallest useful permissions, use different credentials for different environments, and make replacement possible. If a secret is exposed, remove it from the code and revoke or rotate it; deleting the visible line does not make the old value secret again.
Do not collect or return sensitive data merely because it is convenient. Select only the attendee fields needed by the feature, restrict who can access them, and remove them when there is no longer a reason to retain them.
Passwords require specialized handling. Never store plaintext passwords or protect them using reversible encryption. Prefer an established authentication provider.
A password verifier is the value a system keeps in order to check a password without storing the password itself. If the system must store verifiers rather than use an established provider, use a vetted password-hashing facility designed for that purpose, together with its current recommended configuration. Such facilities are deliberately expensive to compute, and they salt each password separately, because the threat is an attacker who has already stolen the stored values and can then guess offline, in parallel, as fast as hardware allows. An ordinary fast hash such as SHA-256 is unsuitable for that reason. Argon2id, scrypt, bcrypt, and PBKDF2 are the usual choices, and the OWASP Password Storage Cheat Sheet tracks which to prefer and with what parameters. Do not invent a password-storage scheme.
Errors should help legitimate users without teaching an attacker about the system. Avoid returning stack traces, queries, filesystem paths, credentials, or private records. Keep detailed diagnostics in appropriately protected logs, and remember that logs themselves can contain sensitive data.
Reused code runs with the permissions of your software and therefore becomes part of its security. This includes libraries, frameworks, build plugins, container images, development tools, and transitive dependencies pulled in by other packages.
Before adding a dependency, ask:
Keep the dependency set small. Commit the ecosystem's lock file, so the exact resolved versions, including the transitive ones nobody chose deliberately, are recorded and reproducible. Check those resolved versions against published advisories using a maintained scanner, rather than against the version numbers written by hand in the build file. When an advisory matches, judge it by actual exposure: whether your code reaches the vulnerable path at all, and what the impact would be if it did. Test before updating, and when an update has to wait, record the decision, who owns it, and when it will be looked at again. Do not leave a known relevant vulnerability unassessed.
Security verification needs both tools and human reasoning. A tool can recognize certain patterns, but it does not fully understand the system's assets, stakeholders, permissions, and assumptions.
Focus review on security-sensitive boundaries such as:
Ask the author to explain the security rule and how the code enforces it. Code that no reviewer can explain is not ready merely because it looks polished.
Turn important threat statements into negative tests and misuse cases.
Example If a requirement says only an event's organizer can see its attendees, test the correct organizer, an unrelated organizer, an attendee, an anonymous user, and a revoked organizer.
Also test boundaries such as empty input, maximum lengths, unexpected encodings, repeated requests, missing resources, and operations performed in a surprising order. Maximum lengths and repeated requests are availability tests: they ask what one caller can consume. The aim is to challenge the assumptions on which the control depends, not merely to generate many random tests.
Compilers, linters, static analysis, secret scanning, dependency analysis, and dynamic testing can catch different kinds of weakness. A clean scan does not prove that software is secure. It means only that the tool did not report a problem within the checks it performed.
Run ordinary regression tests after a security change. A control that breaks required behavior is not a complete solution, and a functional fix can accidentally weaken a security property elsewhere.
Security work should accompany every stage of development because a late safeguard cannot reliably repair an insecure assumption made earlier.
| Stage | Minimum security action |
|---|---|
| Requirements | Identify sensitive assets and state important security requirements and misuse cases. |
| Design | Draw trust boundaries, perform lightweight threat modeling, minimize privileges and exposed interfaces, and choose established mechanisms. |
| Implementation | Validate untrusted data, encode output for its context, enforce authorization, protect secrets, use safe APIs, and review dependencies. |
| Code review and testing | Review security-sensitive paths and test misuse cases, trust boundaries, and permissions. |
| Integration and release | Check dependencies and secrets, use secure configuration and defaults, and remove development credentials and diagnostics. |
| Operation and maintenance | Monitor important failures, update relevant components, remove unused access, and prepare to contain and recover from incidents. |
The table is a timing guide. Apply the threat-modeling, implementation, and verification practices while the relevant decisions are still easy to change, and continue maintaining the controls after release.
State security needs as testable properties rather than vague wishes.
Example "The system must be secure" gives the team no direction. "Only an event's organizers can view its attendee list" names the asset (the attendee list), the property to preserve (confidentiality), and who must not be able to reach it (anyone who is not an organizer of that event). A corresponding misuse case is "a student changes an event identifier to download another event's list."
Revisit the security analysis whenever an important assumption changes. Common triggers include:
What to do first depends on whose system it is. In a system your team owns, contain the harm before treating the problem as an ordinary bug. In someone else's, report it first and contain only what you are authorized to touch. Beginning engineers are not expected to lead a major incident response alone; they are expected to recognize that a suspected security problem needs prompt, careful escalation rather than casual discussion in a public issue.
Depending on the situation, containment might mean disabling a vulnerable feature, revoking an exposed credential, restricting access, or taking an affected service temporarily offline.
Stop exploring once you believe you have found a real problem. Probing further can cause additional damage, and it makes the record harder to interpret afterwards.
Preserve what someone will need in order to understand the problem: the time, what you did, and the smallest reproduction that demonstrates it. Capture the reproduction before revoking or disabling anything, when capturing and containing are both possible, because containing the harm can destroy the record of it. Do not copy out the sensitive data you were able to reach, because that spreads the exposure you are reporting.
Report through a private channel rather than a public one. Use the project's security contact or reporting process where one exists, and course staff otherwise. If nobody acknowledges the report, follow this fallback order rather than wait indefinitely or raise it in public: the project's documented security policy, then its maintainers or the repository owner, then the hosting platform's own vulnerability reporting mechanism. For coursework, course staff are also the fallback.
Then correct the cause, check for related weaknesses, verify the fix, and recover normal service carefully. Withholding details until the people responsible have had a reasonable chance to act is called coordinated disclosure, and many projects publish a policy describing how they expect it to work.
Responsibility for AI-assisted code remains with the engineer and team that accept and deploy it. AI coding tools can explain code, suggest designs, generate implementations and tests, review changes, and operate development tools. Their names and capabilities will change; this rule does not.
AI-generated code must earn its place in the codebase through understanding, review, and verification. Fluent output can compile and pass happy-path tests while omitting authorization, mishandling errors, using an unsafe API, exposing a secret, inventing a dependency, or relying on an assumption that is false in the actual system.
The risk is not that all generated code is insecure. Human-written code can contain the same weaknesses. The risk is automation bias: polished and confident output can feel more trustworthy than the available evidence justifies. In one controlled study, students and professionals completing a set of security-related programming tasks produced less secure solutions when they had access to an AI assistant, and were at the same time more confident that their solutions were secure.
Do not use fluency, speed, compilation, or generated tests as evidence of security. Evidence comes from understanding the change, checking its assumptions, and independently exercising the relevant security properties.
Do not give a coding tool information it is not authorized to receive. Follow the project's rules and the tool's approved configuration. Do not paste secrets, private keys, personal data, confidential logs, or proprietary source into an unapproved service.
A tool may receive more context than the text explicitly pasted into its prompt. Depending on its capabilities and configuration, it may read nearby files, terminal output, version-control history, issue descriptions, connected services, and environment variables. Inspect and restrict that access when sensitive information is present.
The more an AI tool can act, the more important least privilege and explicit approval become. A tool that can edit files, run commands, install packages, access the network, or deploy software can cause harm even when its proposed code is never committed.
Give the tool only the access needed for the current task. Prefer a development environment isolated from valuable data and production systems. Require human approval for high-impact actions such as changing permissions, using credentials, installing an unexpected dependency, publishing artifacts, or modifying external services.
Treat instructions found in source files, web pages, dependency documentation, issue text, and generated output as untrusted data. This is an indirect prompt-injection risk: content the tool was asked to read is taken as an instruction about what to do. It is the same confusion between data and instructions that parameterized queries prevent. But there is no clean fix here: no interface separates the two for a model the way a query parameter does for a database. That is why the controls above are architectural. A tool should not gain authority merely because an instruction appeared in content it was asked to read.
Do not execute a generated command or install a generated dependency until you understand why it is needed and what it can affect. Confirm names and APIs against authoritative documentation. A plausible package name may be wrong, malicious, or unrelated to the intended project.
Inspect changes to configuration, build scripts, lock files, permissions, and automated workflows with the same suspicion as application code. A two-line source change can cause a package manager or deployment system to execute much more code elsewhere.
Apply the following process to AI-assisted changes:
If a generated patch is too large to understand, split it into smaller changes or regenerate it in stages. Code that is cheap to generate is not necessarily cheap to verify.
An AI assistant can help generate questions, but it cannot establish that its own answer is complete or correct. Useful tasks include:
Treat each suggestion as a hypothesis. Check it against the actual requirements, code, environment, and authoritative documentation. Asking the same tool to declare its own code secure is not an independent review.
Example Suppose an AI assistant produces an attendee-list endpoint for the event system and a test for the correct organizer. The generated endpoint accepts an organizerId from the request and treats that client-supplied value as proof of authority.
A security-minded review should:
organizerId claim with authenticated identity, then check that the user organizes the requested event.The same review would be required if a teammate wrote the code. AI changes the speed and source of the draft, not the security standard applied to the result.
Security protects assets and stakeholders from misuse and harm, whether deliberate or accidental. Analysis assumes the deliberate case because an attacker gets to choose the worst inputs, timing, and sequence, but accidental causes need attention of their own. A security mindset questions assumptions and considers what someone could do when trust is misplaced.
Secure software engineering is risk management, not a promise of perfection. Identify assets, stakeholders, trust boundaries, threats, and risks; then select controls whose benefits justify their costs.
Use a small set of durable principles. Grant least privilege, deny by default, minimize attack surface, handle untrusted data at every boundary, enforce authorization on the trusted side, protect secrets and sensitive data, keep one caller from consuming the resources others need, and treat dependencies as part of the product.
Verify security properties with reasoning, review, misuse cases, and several layers of automated checks. A passing happy path or clean scan does not prove security.
Consider security throughout the software lifecycle and whenever assumptions change. Requirements, design, implementation, release, operation, and maintenance each create security decisions.
Treat AI-generated code and actions as untrusted until they have been understood and verified. Limit what tools can see and do, independently check their work, and retain human accountability for accepted changes.
Before accepting a security-relevant change, ask: