mythsthe peopleCloudflare2023internet infrastructure

everyone assumed:

we patched the vendor breach, the old credentials can wait

what was actually true:

A credential you believe is unused is still a credential, and "probably inactive" is a guess until someone checks.

how it happened

  1. whoattackercredentials from the Okta breach
  2. the target4 unrotated credentials"assumed unused"
  3. thenAtlassianJira and wiki pages read
  4. how it stoppedcontainedno customer data impact
where it was aimed where it stopped

what everyone assumed

After a vendor breach you rotate what matters and move on, and “what matters” is the credentials in active use. A service token nobody remembers configuring, a service account for a tool that runs quietly in the background, these read as inactive, and an inactive credential feels like a low priority next to the live ones you just rotated. It is a reasonable way to triage a long list under time pressure.

what happened

One service token and three service accounts were meant to be rotated and were not, on the understanding that they were no longer in use. They were in use. Through October and into November the credentials stayed valid while the incident that exposed them was considered closed. On November 14 someone began testing them against Cloudflare’s systems. The token and one of the accounts still worked.

step by step

  1. October 2023Okta's breach exposes Cloudflare credentialsOne service token and three service accounts are among them.
  2. they are not rotatedThey are believed to be unused. They are in use.the assumption breaks here
  3. 14 November 2023someone tests themThe token and one account still work.
  4. the intruder reaches the Atlassian environment36 Jira tickets and 202 wiki pages. Segmentation keeps customer data and the network safe.

how it surfaced

The intruder reached Cloudflare’s Atlassian environment, opening 36 Jira tickets and 202 wiki pages, much of it the company’s own notes on responding to the Okta incident. Zero Trust segmentation contained the access and there was no impact to customer data or the global network. The post-incident account named the cause without softening it: credentials stolen in the vendor breach were not rotated because they were assumed to be unused.

A credential you believe is unused is still a credential, and “probably inactive” is a guess until someone checks.

The cleanup after a breach is only as complete as its least interesting entry.

picture it at a company like yours

The company here is made up. The other two paths are not: they happened, and each links to the story.

your company
a normal dayA vendor you use reports a breach and you rotate the passwords they could have seen.
the gap nobody seesA few keys are skipped because everyone believes they are no longer used.access that outlives the job
  1. in this story
    the day it mattersWeeks later someone tries those 'unused' keys, and they still work.
    what it costs youThe breach you closed in October reopens in November through the keys you skipped.
  2. or it goes like this
    howThe person logs back in from home, weeks after leaving.
    what leavescustomer lists and reportswho ends up with itthe former employee and whoever they work for next
    it happenedCash App / Block, 2022
  3. or it goes like this
    howThe forgotten account's password turns up in a breach dump.
    what leavesa working way into the networkwho ends up with ita ransomware crew
    it happenedColonial Pipeline, 2021

sources

  1. The Register, "Cloudflare publishes details of Thanksgiving security breach" (Feb 2, 2024)
  2. Infosecurity Magazine, "Cloudflare Suffers Breach After Failing to Rotate Stolen Credentials"
  3. CSO Online, "Nation-state actor used recent Okta compromises to hack into Cloudflare systems"

← all myths

your turn

what you can do about it

  1. this week

    Take the list of service accounts and tokens that touched a recently breached vendor and confirm each one is actually disabled, not just assumed to be.

    Your last credential rotation almost certainly had a few entries someone skipped because they looked dead, the service account from a tool you migrated off, the token from an integration nobody owns anymore. The Cloudflare gap was four entries on a list that was otherwise handled well.

  2. go deeper

  3. if you're not sure where to start

    Run the offboarding checklist on one leaver. It covers every account and token to close when someone leaves.

    open the checklist →

the letter · every two weeks

one access gap in your inbox, with the fix.

Every issue takes one way access creeps in, shows a real incident where it went wrong, and ends with one thing to check that week. Written for the people who run the tools, without a security team behind them.

  • 1one mechanism, explained plainly
  • 2one real incident, taken apart
  • 3one step you can take on Monday

get the next issue

One email every two weeks. Your address stays here and is never passed to vendors. One click to unsubscribe. Privacy.