The Appointment Firm ← All articles

Home · Blog · Screening

Screening

The confident, wrong shortlist

A ranked list is a statement about vocabulary. It looks like a statement about people, which is what makes it dangerous: the error arrives with a percentage attached.

The seductive thing about an automated shortlist is not that it is fast. It is that it is confident. A human recruiter says "I think these three are worth seeing". A system says 94%, 91%, 88%, and confidence is contagious.

What the number is actually measuring

Most matching works on the overlap between the language of a CV and the language of a brief. That produces a reliable measurement of something: how well a candidate's vocabulary matches a hiring manager's vocabulary.

Which rewards people who write CVs the way the industry writes job specs. It penalises the career changer, the person whose last employer used different internal titles, the candidate who does the work but describes it plainly, and anyone who has been out of the market long enough to have missed a fashion in terminology.

Amazon's four-year lesson

The best-documented case remains Amazon's. From 2014 the company built a tool to score applicants from one to five stars, trained on ten years of CVs it had received. By 2015 it was clear the system was not rating candidates for technical roles in a gender-neutral way: it penalised CVs containing the word "women's", and downgraded graduates of two women's colleges. Most CVs in the training data had come from men, so the model learned that being male was a feature of a successful applicant. Amazon could not reliably make it neutral, and scrapped it.1

The model was not broken. It learned exactly what it was shown, which is the problem.

The lesson is not that Amazon was careless. It is that a system trained on who you hired before will reproduce who you hired before, including the parts nobody intended. In South Africa, where a desk has employment equity obligations and a client base that asks about them, that is not only an ethical exposure. It is a commercial one.

The quieter failure

Bias is the famous risk. The ordinary one is more mundane and costs more placements: the system cannot see what is not written down.

The candidate is doing the job already but has not updated their title. They led the project but the CV credits the team. They are willing to relocate for this specific role and no other, which no document anywhere records. They are the right person and they rank eleventh, and if the list is trusted, nobody ever calls them.

How to use a score without being used by it

  • ✓Treat it as a reading order, not a verdict. It decides who you look at first, never who you exclude.
  • ✓Read past the cut-off. Whatever line you draw, read ten below it. That is where the mis-titled and the plainly written sit.
  • ✓Ask what it scored on. If nobody can tell you why someone got 62%, the number is decoration.
  • ✓Watch for suspicious agreement. If the top of the list looks like the last three people you placed, that is the training data talking.
  • ✓Record the overrides. Both for the law and because the pattern of your disagreements tells you what the scoring is missing.

The useful version

None of this is an argument for reading four hundred CVs by hand. Ordering is genuinely valuable, and a consultant who starts with the most likely forty rather than a random forty is doing better work.

The distinction is narrow and it is everything: a score that orders your attention is a tool. A score that ends someone's application is a decision, and decisions belong to people.

Sources

  1. Dastin, J. (2018), Amazon scraps secret AI recruiting tool that showed bias against women, Reuters — summary and citation; further analysis from the ACLU
AF

The Appointment Firm

We have been placing people for 22 years from Northcliff, Johannesburg, across finance, IT and engineering. These notes come from our own desk, and so does the software we built to run it.

Scoring you can argue with

Match scores against the brief, with the reasoning visible, so a consultant can disagree and record why.

Book a demo