Insights from our data team
Being confronted with subconscious biases can be a mixed experience. Sometimes it will make perfect sense why we recommend you to change a phrase, other times you might wonder how we got that particular phrase as biased.
But don’t worry, there are reasons behind it all. One of them is that subconscious bias is.. subconscious. In a way, it is not supposed to make sense from the beginning, but using Develop Diverse should make you more conscious along the way.
Another reason is that the phrases and concepts we highlight has been proved to negatively influence how people read your job ad.
So, if you ever wondered how we got from research papers to a multilingual bias detecting model, do read on.
When Develop Diverse set out to create an AI model for inclusive writing, there was no existing overview, library, or database of biases and stereotypes. We each brought our own personal experiences with stereotyping, and our own unconscious stereotypes about others. Naturally, these individual experiences couldn’t form the foundation of an AI model designed to work across multiple countries.
So, we took on the task to create that essential library of biases and stereotypes ourselves. This required extensive reading and research.
A team of linguistic researchers reviewed and summarized over 100 academic articles published by universities and independent research institutions. The studies investigate biases and stereotypes in the workplace approached the subject from a sociological, psychological, or linguistic perspective, ensuring our model is both holistic and nuanced.
Some of the most influential works we rely on include:
While some studies are older, they remain the most cited and thorough research on the topic to date.
With a multilingual model, covering English, German, French, Danish, and Swedish, we also need a multicultural approach. That’s why we investigate biases and stereotypes within that language community each time we develop a bias detection for a new language.
While reviewing these articles, we mapped out biases and stereotypes across five demographic groups: gender, age, ethnicity, neurodiversity, and physical disability.
We exclude any biases that wasn’t sufficiently backed up by research. For example, biases relating to socioeconomic class exist, but lacked enough evidence to be included.
We organized all identified biases and stereotypes into more than 50 overarching categories. These categories represent cultural concepts like “power,” “humility,” or “influence”. Emphasizing these concepts in your job ad has been proved to affect who applies (read more about this in Trapnell &Paulhus (2012)).
Using this mapping, we developed a framework and began preparing training data for the AI model. Our dataset includes over 1,000 job ads from around the world, written in English, Danish, Swedish, German, or French, covering various industries and sectors.
Biases are everywhere, so our framework and annotation guidelines are very strict and thoroughly documented to minimize individual interpretation. All trainers are trained linguists, and the annotated data undergoes multiple rounds of validation throughout the training process.
We fine-tune a multilingual BERT language model on the annotated data. This approach combines BERT’s advanced contextual understanding with our in-house expertise in bias detection, enabling highly accurate identification of biased language.
In 2021, we collaborated with consultancy Epinion to rigorously test our framework. We replicated parts of Gaucher et al.’s 2011 landmark study on inclusive language in job ads and confirmed the same findings:
Read more about agency and communion here.
In 2025, we evaluated the model again to ensure these effects still hold. We compared number of applicants to jobs where the job ad had an inclusivity below and above 90. Our results showed:
| Inclusive towards | Increase in applicants |
| All genders | 52% |
| All ehnicities | 49% |
| All ages | 47% |
All neurodiversities | 34% |
| All physical abilities | 31% |
Beyond validating our model and framework, we continuously monitor new research and update the model accordingly.
Now that we have an AI model capable of detecting biases and stereotypes, the next question is: what should be written instead?
As mentioned, certain cultural concepts can deter applicants if overly emphasized. Our goal is to minimize or better contextualize these concepts—not simply swap out words with synonyms, which risks reintroducing biases.
Research on high performing job ad shows that being specific, detailed and describing the context of the job can increase the number of qualified candidates.
To find the best replacement for a biased phrase we use the surrounding context to suggest phrases that share meaning with the original words but are more specific. For example:
“If you love working with data, collaborating with cross-functional teams, and making a tangible impact – keep reading.”
This phrase highlights the candidate’s influence, but it doesn’t clarify what that influence entails. Influence isn’t inherently negative, but how it’s described affects how candidates perceive the role and your expectations to the candidate.
The sentence that come just before the one above goes like this:
“At XXX, we’re looking for a Data Analyst – Product to bring clarity to complexity, empowering our teams with the insights they need to build world-class products.”
It provides us with the context we need to suggest what kind of impact the role will have.
Instead of suggesting synonymous alternatives like “be influential,” we recommend more precise phrases like “provide data-driven solutions”, or “support team decisions,” which directly reflect the role’s responsibilities shown in the context.
If you want to read more about high performing job ads, we recommend these literature reviews:
