mythsthe AISlack AI training default2024saas

everyone assumed:

our SaaS won't train on our messages by default

what was actually true:

A vendor's default is a decision made for you until you find it and change it.

how it happened

  1. whodefault settingmessages used for training
  2. the gapcustomers enrolledwithout choosing
  3. the resultpublic backlashpolicy clarified
the gap that let it happen the result

what everyone assumed

A trusted platform feels like a custodian, holding your data to serve you, not to learn from. The assumption is that using your content for the vendor’s models would be opt-in, announced, and obviously toggled, not the silent default.

what happened

There was no in-app switch. The opt-out existed only as an email to a particular address that few customers knew to send. The training was for non-generative features, but the posture was the same: your messages, enrolled by default, with the burden to leave placed on you.

step by step

  1. messages are enrolled in model training by defaultFor non-generative features.the assumption breaks here
  2. opting out takes an email to one addressNo in-app switch.
  3. May 2024the policy surfaces and a backlash follows

how it surfaced

In May 2024 researchers and customers surfaced the policy and a public backlash followed. The episode put a number on a quiet truth: the default disposition of a whole customer base’s data had been set without most of them choosing it.

A vendor’s default is a decision made for you until you find it and change it.

What a SaaS tool may do with your data lives in terms nobody reads, and silence in those terms is not the same as restraint.

picture it at a company like yours

The company here is made up. The other two paths are not: they happened, and each links to the story.

your company
a normal dayYour company chat tool has new AI features, switched on by default.
the gap nobody seesThe terms let the vendor use your messages for training unless you opt out by email.company data flowing into AI tools nobody vetted
  1. in this story
    the day it mattersSomeone reads the terms and the story spreads.
    what it costs youYour messages were part of a programme nobody at your company chose.
  2. or it goes like this
    howEngineers paste code and notes into a free chatbot to get help.
    what leavesconfidential code and meeting noteswho ends up with itan AI vendor on consumer terms
    it happenedSamsung, 2023
  3. or it goes like this
    howThe AI vendor leaves its own database open.
    what leaveseveryone's chats and the vendor's keyswho ends up with itanyone on the internet
    it happenedDeepSeek exposed database, 2025

sources

  1. TechCrunch, May 2024

← all myths

your turn

what you can do about it

  1. this week

    Read the data-use terms of your three most-used platforms.

    The tools your company already trusts have data-use terms you agreed to without reading, and some of those defaults favor the vendor. Slack is the reminder to actually read them. Knowing what each core platform may do with your content is a vetting question with a deadline you set.

  2. go deeper

  3. if you're not sure where to start

    Run the offboarding checklist on one leaver. It covers every account and token to close when someone leaves.

    open the checklist →

the letter · every two weeks

one access gap in your inbox, with the fix.

Every issue takes one way access creeps in, shows a real incident where it went wrong, and ends with one thing to check that week. Written for the people who run the tools, without a security team behind them.

  • 1one mechanism, explained plainly
  • 2one real incident, taken apart
  • 3one step you can take on Monday

get the next issue

One email every two weeks. Your address stays here and is never passed to vendors. One click to unsubscribe. Privacy.