A difficulty faced by scholars and practitioners working with numbers about people is that those who study people want numerical data about these people. Unfortunately, time and time again, this numerical data about people is wrong.
Addressing the potential causes of this wrongness, we present examples of analyzing people numbers, i.e., numbers derived from digital data by or about people, and discuss the comforting illusion of data validity. We first lay a foundation by highlighting potential inaccuracies in collecting people data, such as selection bias. Then, we discuss inaccuracies in analyzing people data, such as the flaw of averages, followed by a discussion of errors that are made when trying to make sense of people data through techniques such as posterior labeling.
Finally, we discuss a root cause of people data often being wrong – the conceptual conundrum of thinking the numbers are counts when they are actually measures. Practical solutions to address this illusion of data validity are proposed.
The implications for theories derived from people data are also highlighted, namely that these people theories are generally wrong as they are often derived from wrong people numbers.
See more about using analytics to make impactful data-driven decisions in our book.
Jansen, B. J., Aldous, K, Salminen, J., Almerekhi, H. and Jung, S.G. (2023). Understanding Audiences, Customers, and Users via Analytics – An Introduction to the Employment of Web, Social, and Other Types of Digital People Data. Springer Nature.
Read the research article: Jansen, B. J., Salminen, J., Jung, S.G., and Almerekhi, H. (2022) The Illusion of Data Validity: Why Numbers About People Are Likely Wrong. Data and Information Management. 6(4), 100020.