how it happened
- whomalwareon an engineer's laptop
- the gapvalid sessionbypassed MFA
- thenproduction systemscustomer secrets stores
- the resultrotate every secretall customers told to
what everyone assumed
A build pipeline feels like a passthrough. You hand it secrets so it can deploy your code, and you trust that the vendor’s own compromise would stay on the vendor’s side of the line, hitting their infrastructure rather than yours. It is a reasonable place to draw the boundary, because storing tokens in a CI system is how CI works, and the whole model assumes the vault holds. The secrets you put there are the price of automation.
what happened
The stolen session did what a valid session does: it let the attacker in as a trusted employee. That employee had privileges to generate production access tokens, so the intrusion did not stop at one machine. It reached a subset of CircleCI’s production systems and the stores behind them, the places where customer secrets lived. Environment variables, tokens, and keys that customers had entrusted to the pipeline were now readable by someone who had walked in on a borrowed login, and for a stretch of days nobody outside knew it had happened.
step by step
- malware steals an engineer's sessionThe session is valid, so it bypasses login and MFA.
- the employee can generate production tokensThe attacker reaches a subset of production systems.the assumption breaks here
- customer secrets become readableEnvironment variables, tokens and keys stored in the pipeline.
- 29 Dec 2022 to 4 Jan 2023suspicious activity, then a blunt noticeCustomers are told to rotate every secret stored in CircleCI.
how it surfaced
CircleCI noticed suspicious GitHub OAuth activity on December 29, 2022, and traced it back to the compromised laptop. On January 4, 2023, it told customers to treat the incident as a breach, and the instruction was blunt: rotate all secrets stored in CircleCI. Not the exposed ones, not the suspicious ones, all of them, because the company could not draw a clean line around what the borrowed session had touched. The assumption retired itself in that sentence. A vendor’s stolen login had become every customer’s rotation weekend.
Every secret you store in a build pipeline is reachable by whoever can act as a privileged employee of that vendor, and a stolen session can make them one.
The tokens you hand a CI system are only as safe as the sessions of the people who run it.
picture it at a company like yours
The company here is made up. The other two paths are not: they happened, and each links to the story.
- in this storythe day it mattersThe vendor is breached through one of its own staff laptops.what it costs youYou are told to change every single secret you ever stored there, this weekend.
- or it goes like thisit happenedSalesloft / Drift, 2025howA chat tool's key into your CRM is stolen from the vendor.what leavessupport tickets, including passwords pasted into themwho ends up with itan extortion group working through hundreds of companies
- or it goes like thisit happenedGitHub OAuth tokens, 2022howIntegration keys to your code host are stolen from the integration vendor.what leavesyour private code and the secrets inside itwho ends up with itattackers looking for the next door
sources
your turn
what you can do about it
this week
Inventory the secrets you have given your build and deploy vendors, and confirm they are scoped and rotatable rather than long-lived and forgotten.
You have secrets sitting in a CI/CD system right now, and you probably could not list them all from memory. The quiet-months question is what would happen if that vendor told you to rotate everything by Friday: would you know what "everything" is, and could you do it without breaking production. The customers who moved fastest in January 2023 already had that list.
go deeper
if you're not sure where to start
Run the offboarding checklist on one leaver. It covers every account and token to close when someone leaves.
open the checklist →