Skip to content
Aplora
EdTech
Aplora Sales

We validated the system inside our own operations first

A sales candidate is assessed from a recorded conversation, not from the impression they leave

An online IT school in Poland with its own sales methodology. The admissions team grows in waves with each intake, so hiring runs continuously.

Published: 2026-08-24 · updated: 2026-08-24

Someone going through a printed form at a meeting table
Illustrative photo of the working context, not a screenshot of the client's system

In short

  • Problem: candidates were judged on impression, interviewers disagreed, and a hiring mistake surfaced a month into the job.
  • Solution: a role-play inside the interview and a review of the recording against the same methodology criteria used for working calls.
  • Result: the decision rests on the candidate's observable actions; there is no measured effect yet — hiring quality needs a long observation window.
  • Systems: video calls, transcription, the in-house sales methodology, CRM.

Context

  • An online IT school selling B2C through a consultation call.
  • Team: the head of sales, admissions advisors, and interviewers from the business side.
  • Process volume: continuous hiring against intake waves.
  • Systems: recorded video calls, transcription, a documented sales methodology, CRM.
  • The constraint grew with the team: the more advisors there are, the more each misfit costs — what diverges is not only the result but the way the conversation with an applicant is run.

Baseline

Baseline metrics with their sources. Without them, any later result has nothing to be compared against.
  • Before the work started we fixed: the share of new advisors reaching target in their first quarter, and the time from interview to decision.
  • Data source: the CRM and internal hiring records.
  • The values are not published: hiring quality shows itself over quarters rather than weeks, and the sample is still too small to separate effect from chance. Publishing “it got better” with no basis for comparison is exactly what §14.4 forbids.

Diagnosis

Which hypotheses were considered, why this one was chosen, what was assumed, and the condition under which we would have stopped.
  • Three hypotheses were on the table: too few candidates, weak onboarding, or an interview assessment that does not predict performance.
  • The third was chosen: review showed that candidates the interviewers had rated equally highly went on to differ twofold in results — so what was being assessed was not what determines the outcome.
  • The assumption: the school's sales methodology describes practice that works, and checkable adherence to it in a role-play says something about future performance. The assumption is weak, and we say so: it has not been tested over a long horizon.
  • Stop criterion: if the role-play assessment cannot separate advisors who reach target from those who don't, the method counts as unfit. That check needs an accumulated sample and is not finished.

What we implemented

  • Data sources: the recording of the interview and role-play, and the documented methodology criteria.
  • AI components: transcription of the recording and a criterion-by-criterion review, each pointing at the excerpt where the criterion was or wasn't met.
  • Business rules: the review shows observable actions and the excerpts supporting them. There is no score summarising a candidate in one number — it would create a precision that does not exist here.
  • Integrations: video calls, CRM.
  • Human checkpoints: the hiring decision is the manager's. The system neither recommends hiring nor filters candidates out — §11.4 leaves no choice here: this is a decision about a person.
  • What is not analysed: voice, speech rate, emotion, appearance. The review covers only the actions described in the methodology — everything else turns quickly into discrimination on grounds unrelated to the job.

How the process changed

Before

5 steps
  1. A free-form interview
  2. An assessment from the interviewer's impression
  3. Candidates compared in conversation
  4. A hiring decision
  5. The mistake surfaces a month into the job

After

6 steps
  1. An interview with a shared structure
  2. A short recorded role-play
  3. Transcription
  4. A criterion-by-criterion review with excerpts
  5. Candidates compared against the same criteria
  6. The manager makes the decision

What was stuck

Hiring decisions were made on the impression the interview left. Interviewers assessed candidates differently, a mistake surfaced a month into the job, and the head of sales spent hours on reviews and arguments about who was better.

  1. 1An interview with a shared structure
  2. 2A short recorded role-play
  3. 3Transcription
  4. 4A criterion-by-criterion review with excerpts
  5. 5Candidates compared against the same criteria
  6. 6The manager makes the decision
  7. Measured result

Results

Why there are no numbers here

Hiring quality shows itself over quarters rather than weeks. The sample is still too small to separate effect from chance, and publishing “it got better” with no basis for comparison is exactly what §14.4 forbids.

  • Share of new advisors reaching target in their first quarter
  • Time from interview to a decision on the candidate

Economic impact

  • The effect is losses prevented from a bad hire: the least reliable kind of calculation, because it compares against an event that did not happen.
  • The second effect is more reliable and smaller: freed manager time that used to go into reviews and arguments about candidates.
  • No money figure is published: the cost of a bad hire depends on how long the person stayed and what they did, and that differs every time.

Adoption

  • Candidate assessment became a standard step rather than an extra one “if there's time”.
  • New advisors ramp up faster: the role-play review immediately shows what onboarding needs to cover.
  • The solution owner is the head of sales.
«Hiring used to be a lottery: the candidate speaks well, and a month later you realise they can't do it. Now we can see from the recording how the person actually sells.»

Client under NDA — Head of sales, LearnIT

What's next

  • Accumulating a sample: without it there is no way to check whether the assessment predicts performance.
  • Next initiative: linking the role-play review to the onboarding plan — the criterion the candidate dipped on becomes the first session.
  • What we decided against: automatic screening by threshold. The threshold is fallible, and a rejection cannot be taken back.

Have a similar workflow? Let's check whether the hypothesis transfers

Another company's result is not a promise. It does show where to look.

All cases