Embedding inclusion in pharmaceutical design

In this article, Included’s Raafi-Karim Alidina explores biases in product design, outlining how they impact pharmaceuticals as well as technologies, and practical steps to enhance diversity and ensure inclusion.

Recently, I went to a barbecue restaurant with my partner – it had been a long time since we were last able to go out for dinner, so it felt like a special treat. When I went to the bathroom to wash my hands, everything was touchless: there were automatic faucets, hand dryers, soap dispensers… even an automatic paper towel dispenser if you prefer that to a hand dryer.  However, as soon as I tried to use any of them, they would not work. No matter what I did, I just could not get the sensor to detect my hands and dispense water or soap. Then, someone else came into the bathroom and tried to wash his hands and everything worked perfectly.  What was the difference? His skin was white, while mine was brown.

Bias by design

If you are a person of colour, this is probably an all-too-familiar experience – and it happens because of a completely preventable mistake. The reason the sensors in those automatic taps do not work as well for people of colour as they do for white people is because the pictures the machine uses to learn when to dispense the soap or water are made up mostly of light-skinned hands. One study found these training sets contain 3.5 times more pictures of light-skinned hands than dark-skinned ones.¹ As a result, the machine learns that it should only really dispense soap for light-skinned people.

This is annoying for people of colour, but the real problem occurs when we use those same object-detection systems in more dangerous contexts, such as self-driving cars. A study at Georgia Tech University in the US found that many of the autonomous vehicles being developed that used these systems were significantly worse at detecting and so stopping for dark-skinned pedestrians.² This could lead to more Black and brown people dying.

Why is this happening?

Two main issues underpin these mistakes in the way we develop products, including pharmaceuticals:

1. Our teams are not diverse enough. When a lab or development team is made of people who are all from a similar background, it is much more likely they will have a blind spot when it comes to people different from them, including racial, gender and other differences. Non-diverse teams are just less likely to notice when a clinical trial group just is not properly representative of the population.
2. Our teams are not inclusive enough. Even when the team is very diverse and representative, often those from marginalised or non-dominant groups do not feel psychologically safe. They do not feel comfortable speaking up, voicing dissent or expressing a different point of view.  This prevents them from being able to actually say something if they notice a bias, blindspot or mistake.

What can we do?

The problem is that our default methods are not equitable. But if we are conscious and deliberate about inclusion and diversity in the process we use when designing new products – medications, medical devices or any treatment method – we can reduce the likelihood that our default biases will be reflected in what we have created.  Here are three ideas you can implement into your process:

1. Add an inclusion review to your product development process, similar to a legal review. The inclusion review should be completed before a product goes to market and could simply be a check by someone who is not on the development team to ensure any inclusion blind spots are covered.
2. Leverage the diversity of our teams by adding “inclusion checks” at different points of the development process. In this scenario, teams would actively ask everyone to specifically comment on whether they can see any potential bias in the way the research is being conducted.
3. Add questions at the beginning and end of a product’s lifecycle to notice and reflect on one’s own identities and biases – and those of the team as a whole. By doing this at the beginning, researchers can put themselves in an inclusive mindset as they create their products, and asking such questions at the end acts as a retrospective learning exercise to improve future design.

These are just a few ideas teams could implement to make better and more inclusive products.  Whatever you choose, notice that none of these solutions are passive; to stop making these very preventable mistakes, we need to take active steps and build them into the way we do our work every day.

References

1. Research Reveals Possibly Fatal Consequences of Algorithmic Bias [Internet]. School of Computer Science. 2019 [cited August 2021]. Available from: https://www.scs.gatech.edu/news/620309/research-reveals-possibly-fatal-consequences-algorithmic-bias
2. Samuel S. Study finds a potential risk with self-driving cars: failure to detect dark-skinned pedestrians [Internet]. Vox. 2019 [cited August 2021]. Available from: https://www.vox.com/future-perfect/2019/3/5/18251924/self-driving-car-racial-bias…
3. Mohn T. Dummies Used In Motor Vehicle Crash Tests Favor Men And Put Women At Risk, New Report Says [Internet]. Forbes. 2019 [cited August 2021]. Available from: https://www.forbes.com/sites/tanyamohn/2019/10/28/…
4. Zucker I, Prendergast B. Sex differences in pharmacokinetics predict adverse drug reactions in women. Biology of Sex Differences [Internet]. 2020 [cited August 2021];11(1). Available from: https://bsd.biomedcentral.com/articles/10.1186/s13293-020-00308-5
5. Curno M, Rossi S, Hodges-Mameletzis I, Johnston R, Price M, Heidari S. A Systematic Review of the Inclusion (or Exclusion) of Women in HIV Research. JAIDS Journal of Acquired Immune Deficiency Syndromes [Internet]. 2016 [cited August 2021];71(2):181-188. Available from: https://journals.lww.com/jaids/Fulltext/2016/02010/…
6. Santos-Casado M, García-Avello A. Systematic Review of Gender Bias in the Clinical Trials of New Long-Acting Antipsychotic Drugs. Journal of Clinical Psychopharmacology [Internet]. 2019 [cited August 2021];39(3):264-272. Available from: https://journals.lww.com/psychopharmacology/…
7. Phillips S, Hamberg K. Doubly blind: a systematic review of gender in randomised controlled trials. Global Health Action [Internet]. 2016 [cited August 2021];9(1):29597. Available from: https://www.tandfonline.com/doi/full/10.3402/gha.v9.29597
8. Alonso-Moreno M, Ladrón-Guevara M, Ciudad-Gutiérrez P. Systematic review of gender bias in clinical trials of monoclonal antibodies for the treatment of multiple sclerosis. Neurología [Internet]. 2021 [cited August 2021];. Available from: https://www.sciencedirect.com/science/article/pii/S0213485321000086
9. Gottesman R, Hamilton R. Recruiting Diverse Populations in Clinical Trials. Neurology [Internet]. 2021 [cited August 2021];96(11):509-510. Available from: https://n.neurology.org/content/96/11/509
10. Raynaud M, Zhang H, Louis K, Goutaudier V, Wang J, Dubourg Q et al. COVID-19-related medical research: a meta-research and critical appraisal. BMC Medical Research Methodology [Internet]. 2021 [cited August 2021];21(1). Available from: https://link.springer.com/article/10.1186/s12874-020-01190-w

Source