mythsthe wider recordiTutorGroup2023education / hiring

everyone assumed:

an automated screener can't discriminate

what was actually true:

A screen does not become fair by being automatic.

how it happened

  1. whoscreening ruleage limits configured
  2. the gaphiring softwareapplied one unlawful rule
  3. the result$365,000 settlementfirst EEOC AI case
the gap that let it happen the result

what everyone assumed

A recruiting screen exists to handle volume so people do not have to, and routing the first pass through software feels like the opposite of prejudice, because software has no feelings to have. A screen applies the same rule to everyone, and uniform treatment sounds like the definition of fair. The assumption was that an automated filter, by being automatic, could not be the thing doing the discriminating.

what happened

The tutoring provider configured its application software to automatically reject women who applied at age 55 or older and men who applied at age 60 or older. The rule ran as designed. Over 2020 it rejected more than two hundred qualified applicants based in the United States on the basis of age, each rejection arriving as a routine outcome rather than a decision anyone had to defend. The screen was consistent, and consistency was exactly the problem, because it applied one unlawful rule without exception.

step by step

  1. software set to reject applicants by ageWomen 55 and over, men 60 and over.the assumption breaks here
  2. 2020more than two hundred qualified applicants rejectedEach looks like a routine outcome.
  3. a rejected applicant goes to the EEOCA lawsuit under the Age Discrimination in Employment Act follows.
  4. 9 August 2023365,000 dollar consent decreeThe EEOC's first AI hiring settlement.

how it surfaced

A rejected applicant, told no on what turned out to be a birthday, brought the matter to the U.S. Equal Employment Opportunity Commission, which sued under the Age Discrimination in Employment Act. The case resolved on 9 August 2023 with a consent decree: iTutorGroup agreed to pay 365,000 US dollars to the applicants the software had rejected, and to anti-discrimination training, a new policy, and injunctions against asking applicants for their birth dates. It was the EEOC’s first settlement of a hiring case built on artificial intelligence software.

A screen does not become fair by being automatic.

It applies whatever rule it was given, to everyone, every time, which is what makes a bad rule inside it more dangerous than a bad reviewer.

picture it at a company like yours

The company here is made up. The other two paths are not: they happened, and each links to the story.

your company
a normal dayYour hiring software filters applicants automatically using rules someone set up.
the gap nobody seesOne rule quietly screens people out by age, and nobody reviews the rules.an automated decision nobody reviews
  1. in this story
    the day it mattersA rejected applicant notices and complains to a regulator.
    what it costs youYou face a discrimination case for a decision no person in your company made.
  2. or it goes like this
    howA model learns past hiring patterns as signs of quality.
    what leavesa year of skewed decisionswho ends up with itcandidates who never knew why
    it happenedAmazon, 2018
  3. or it goes like this
    howRejections add up across thousands of applicants nobody compares.
    what leavesa collective lawsuitwho ends up with iteveryone screened out
    it happenedMobley v. Workday, 2025

sources

  1. "iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit," U.S. Equal Employment Opportunity Commission, press release, 11 September 2023.
  2. EEOC v. iTutorGroup, Inc., et al., Civil Action No. 1:22-cv-02565 (E.D.N.Y.), consent decree filed August 2023.

← all myths

your turn

what you can do about it

  1. this week

    Pull the auto-reject criteria for one role and check what they screen out and on what basis.

    Somewhere in your hiring flow there is probably a filter that says no before a person reads the file, on experience, on a knockout question, on a date. The question the quiet months ask is what that filter is actually keying on, and whether anyone has read its rules as a lawyer would. The iTutorGroup rule was visible in its own configuration, to anyone who looked at it as a rule about people.

  2. go deeper

  3. if you're not sure where to start

    Run the offboarding checklist on one leaver. It covers every account and token to close when someone leaves.

    open the checklist →

the letter · every two weeks

one access gap in your inbox, with the fix.

Every issue takes one way access creeps in, shows a real incident where it went wrong, and ends with one thing to check that week. Written for the people who run the tools, without a security team behind them.

  • 1one mechanism, explained plainly
  • 2one real incident, taken apart
  • 3one step you can take on Monday

get the next issue

One email every two weeks. Your address stays here and is never passed to vendors. One click to unsubscribe. Privacy.