The science behind the Atlas assessment
A summary of how the instrument was designed, the studies used to test it, and the results so far.
- Retest agreement
- 0.81Median correlation between a person's first and second results, retested after 33 to 42 days.
- Top ten held
- 7.4 of 10Of a person's top ten constructs, the average number still in the top ten at retest.
- Scale reliability
- 0.89Median omega across the scales.
- Statements written
- 1,227Written and tested against the construct definitions. Only high value statements used in final instrument.
Atlas reads a leadership team against the plan it has to deliver. Part of that reading comes from an assessment each person takes. It covers 34 constructs: patterns in how a person thinks, decides, stays motivated, and works with others. Each construct has a written definition and a set of facets, and each statement in the assessment was written to express one of them.
The result is a ranking of the 34 constructs within one person, from most to least prominent.
- 01Writing
- 02Rating
- 03Desirability
- 04Pairing
- 05Piloting
- 06Review
1,227 statements
1,227 statements were written against the 34 definitions, covering the facets of each construct.
79,813 ratings
Respondents rated each statement for how well it described them, giving 79,813 ratings. Each session was screened for careless answering before its ratings were used. The ratings showed which statements measure their construct well.
Split-half reliability 0.95
A separate group of raters judged how favourable each statement is to agree with. Their judgements agreed closely, with a split-half reliability of 0.95.
Each construct appears in 14 pairs
Statements were paired so that both sides of a pair sound equally desirable. Each construct appears in 14 pairs, including its reverse-worded statements, and each statement appears in one pair only.
Five rounds
Five pilot rounds tested the pairs and the scoring allowing calibrations of pairs. The criteria for finalising the instrument were set before the final round.
Retested and reviewed
The finished assessment was taken a second time by the same respondents and people's results held. Our methodology, data, and results were reviewed and approved by our scientific advisor, Dr Susanne Frick.
On a rating scale it is easy to agree with everything that sounds favourable. Here, each question shows two statements, and the person says which describes them better and by how much, on a six-point scale with no neutral option. Both statements were matched on how desirable they sound, so the choice depends on the person rather than on which answer looks better. In the pilot rounds, the two sides of the average pair were chosen almost equally often.
The assessment has 260 pairs and takes about 18 minutes. Repeated pairs and attention checks are built in, so we can confirm that a sitting was answered with care.
Most statements in the assessment describe a construct directly. Each construct also has a statement that describes its opposite, which a person who is high on the construct should decline.
When every statement describes something positive, a person can only show which tendencies they prefer over others. A statement worded in reverse asks them to turn a tendency down, which adds information about how strongly they hold it. Reading each construct from both directions gives a fuller picture of the person than reading it from one.
Each reverse-worded statement is written against the core of its construct and tested before it is used.
Each choice is a comparison between two constructs. The scoring uses a Bradley-Terry model, a standard statistical model for paired comparisons. It estimates a strength for each construct from all of a person's choices together, so preferring a construct over one that is strong for that person counts for more than preferring it over a weak one. Each construct receives a score from 0 to 100, where 50 is that person's average, and a rank from 1 to 34.
We retested respondents 33 to 42 days after their first sitting, with a median of 40 days. The median correlation between their first and second results was 0.81.
Of a person's top ten constructs, an average of 7.4 were still in the top ten at the second sitting, against 2.9 expected by chance. The bottom ten held in the same way, at 6.7. The construct ranked first was within the top ten at retest in every case analysed. The highest scores sit close together, so their order can shift between sittings. The group itself stays stable.
An earlier retest, with six days between sittings, gave 0.80. The result at six weeks matches the result at six days, so the stability does not come from people remembering their earlier answers.
Using the ratings from the rating study, we measured the internal reliability of each of the 34 scales. The median omega was 0.89, with a range of 0.71 to 0.95.
We also examined each of the 561 pairs of constructs to see whether the data would be described as well by treating the two as one. In the 355 pairs where that comparison could be estimated, keeping the constructs separate fitted better in all 355. The median correlation between constructs was 0.19.
Dr Susanne Frick is a psychometrician at TU Dortmund University, where her research covers forced-choice measurement. She has been scientific advisor to Atlas People since May 2026. She advised on the design of the assessment and reviewed its methodology and results, with access to the full documentation, the respondent-level data, and the analysis code.
Dr Susanne Frick on Google Scholar“The assessment was designed and developed following established practice in psychometric test construction. The reported test-retest reliability, a median profile correlation of 0.806 over a median interval of 40 days, shows that a person's profile is stable over time.”