Large language models (LLMs) can generate personas based on prompts that describe the target user group. To understand what kind of personas LLMs generate, in this research led by Joni Salminen, we investigate the diversity and bias in 450 LLM-generated personas with the help of internal evaluators (n=4) and subject-matter experts (SMEs) (n=5).
The research findings reveal biases in LLM-generated personas, particularly in age, occupation, and pain points, as well as a strong bias towards personas from the United States. Human evaluations demonstrate that LLM persona descriptions were informative, believable, positive, relatable, and not stereotyped. The SMEs rated the personas slightly more stereotypical, less positive, and less relatable than the internal evaluators.
The findings suggest that LLMs can generate consistent personas perceived as believable, relatable, and informative while containing relatively low amounts of stereotyping.
Read the full research article! You will be glad you did!
Salminen, J., Lui, C., Pian, W., Chi, J., Häyhänen, E., and Jansen, B. J. (2023) Deus Ex Machina and Personas from Large Language Models: Investigating the Composition of AI-Generated Persona Descriptions, In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24), 11-16 May 2024, Oʻahu, Hawaiʻi, USA. Article 510, 1–20. https://doi.org/10.1145/3613904.3642036.