Meet HAL: Human-Centered Analytics in an Increasingly Digital World

NDDCEL: What is your specific area of research expertise? 

Dr. Ken Kelley: My area of expertise is quantitative methodology. It is a technical field concerned with how we design studies to draw valid inferences, measure what we are interested in, and test theoretical models using data from persons. At its core, my work is about validity: the truthfulness of what we measure, the rigor of how we obtain those measures, and the soundness of how we draw conclusions from them. Another way to put it: my research is about how we know what we know about psychologically related variables and phenomena.

My work sits at the intersection of psychology and statistics, but it is not a simple combination of the two. I bring psychological measurement into a formal statistical framework, so that the methods we use to study people are as rigorous as the questions we ask about them.

This is what we mean by human-centered analytics: research concerned with outcomes related to persons, whether that context is an interaction with technology, a managerial decision, or a fundamental question in psychology. The methods must be worthy of the questions.

Dr. Ken Kelley

One way to understand where I sit: mathematical statisticians tend to think of me as a psychologist, and applied psychologists tend to think of me as a statistician. In some ways I am both, and in some ways I am neither. But that position, between the disciplines and accountable to both, is precisely where the methodological work that matters most gets done.

If you could tell business leaders one thing about your discipline, what would it be? 

KK: Understanding and impacting employees is necessarily multidisciplinary and multivariate in nature, and cannot be reduced to only simple “if this, then that” rules in a complicated environment. Context matters. Individual differences matter. People are always changing, and they are affected by many things outside the workplace, such as family, health, culture, life stage, among other things. A single variable rarely tells the whole story.

This means that the tools we use to study and make decisions about people need to be sophisticated enough to reflect that complexity: not just whether an effect exists, but how large it is, how precisely we have estimated it, and whether our study was designed well enough to support the conclusions we want to draw. Decisions about people deserve that level of rigor.

How does HAL reflect the mission of the Mendoza College of Business? 

KK: HAL, the Human-centered Analytics Lab, is first and foremost about human-centered outcomes, that is, outcomes related to people. It starts with the person, then moves to technology and the interaction between the two, because that is how business and much of society actually functions. Most companies are users of technology, and so are their employees. The environment is always changing, which means there is always something new to study.

Mendoza’s mission includes a commitment to “improve the human condition in an ever-changing society,” and aspirationally, to “contribute to the formation of ethical business leaders who integrate the mind and the heart, and have the competence to see and the courage to act.” HAL is inextricably aligned with those ideals. The mind and the heart, followed by the action of having the courage to act, are drawn from the Holy Cross foundational documents and are central to the University’s identity.

My research brings something specific to HAL in this regard: it is explicitly concerned with the measurement of things like “the mind” and “the heart,” which are not directly observed. They exist as latent variables. Latent is Latin for “hidden,” and that is an apt description for affective, emotional, and other psychological variables involving states and traits that we cannot see and therefore cannot measure directly. For such variables, we can measure only their reflections, that is, outcomes we believe to be caused by these hidden states. The challenge, then, is to incorporate those inferred variables into models that can evaluate a range of meaningful outcomes. For example, one cannot observe "extraversion," a personality dimension, but only indications of it, such as one's preference for being in large groups, talking to strangers, meeting new people, seeking out exciting activities, etc.

Why is the integration of analytics and human impact so critical?

KK: The short answer is that most of what matters in business as it relates to the person, including motivation, trust, leadership, culture, and decision-making, cannot be directly observed. These are psychological constructs (i.e., latent variables), not physical objects you can point to or weigh or some monetary value of a product or performance. Personality, for example, does not exist as something you can see (or measure) directly. It manifests through thoughts, feelings, emotions, and actions, but only indirectly. Measuring such phenomena therefore requires a fundamentally different approach than measuring something physical, and that opens an entirely different set of measurement and modeling challenges.

You cannot study business without studying people. Across every business domain, including operations, strategy, information systems, and marketing, the person plays a central role. How to design studies that account for this, how to measure the constructs that matter, and how to test theories about them: that is the essence of my collaborative work. When analytics is disconnected from the human dimension, it risks producing insights that are technically clean but practically meaningless.

Dr. Ken Kelley

What’s next in the space of human-centered analytics? What issues should be front-of-mind for today’s business leaders?

KK: We should continue to draw together skill sets from different perspectives, including psychology, statistics, computer science, and information systems, to better model, explain, and predict behavior and psychological outcomes, particularly in the uniquely human context of work in organizations.

Solutions to problems involving people rarely have siloed answers. And it is often difficult to change an entire system by adjusting one variable at a time. When you turn one dial, other variables move too, sometimes in ways you did not anticipate. Multivariate thinking is therefore not optional when the person and the organization are both in the picture. The leaders who will navigate this well are the ones who resist the temptation to reduce complex human dynamics to a single metric.

Any other insights you'd like to share? 

KK: The adage “people make the place” is a useful starting point.

As business leaders, the question is: how do you create an environment where people genuinely feel they matter, and where decisions about change take the human side seriously? Everyone is on their own trajectory. What works for one person may not work for another. Empowering individuals means acknowledging that. A related caution: the instruments and tools available for measuring people, such as surveys, assessments, and evaluations, may be useful, but they may not be. Having “better” scores does not automatically translate to better outcomes. It is essential not to focus on the wrong metrics, and not to let the polish of an instrument dictate consequential decisions, such as who is hired or promoted, without formal, rigorous justification of what that instrument actually measures and whether those measurements are valid for the intended use.

Dr. Ken Kelley

HAL engages affiliate faculty members whose expertise aligns with its mission. Professor John Lalor offered a lens on his research, and its implications for business. Dr. Lalor's current work is in natural language processing (NLP), the area of artificial intelligence that focuses on enabling computers to understand, interpret, and generate human language. This includes work on large language models (LLMs) that are the basis for the current AI wave. Dr. Lalor's work ranges from developing novel techniques for training and evaluating these systems to studying the impacts of these systems in firms and online communities.

Working on technical problems with business implications in the area of information systems allows for interesting perspectives for tackling research problems. Technical solutions with designs inspired by research in the business field can lead to novel solutions with scholarly and managerial impacts. 

Dr. John Lalor

We asked Dr. Lalor to share his experiences in the classroom: As our next generation of business leaders, what are students curious about? Excited about?

JL: Everyone is very excited about AI at the moment. In my undergraduate class, Unstructured Data Analytics, students learn about how to make sense of large collections of text data using techniques from NLP. They are excited to learn about some of the foundations that are behind things like large language models. They are curious about learning more about what, on the surface, look like black boxes. 

Ph.D. students at the Mendoza College of Business support HAL's mission and are engaged in impactful, cutting-edge scholarly research that considers the ethical dimension of data and its usage. Analytics Ph.D. student Sofia Calderon Farfan provided a glimpse into her work and its implications. With a background in business economics, causal inference, and experimental methods, Sofia's current research examines human-computer interaction and the integration of technological innovation across various contexts while respecting user agency, promoting inclusivity, and preventing harm.

Businesses can be simultaneously ethical and profitable, especially given growing awareness and concerns about topics such as privacy and the ethical use of information. Establishing safeguards to prevent harm can drive the sustainable adoption essential for long-term success.

Sofia Calderon Farfan

Sofia notes, "By understanding how individuals use new technology and identifying the ways they are vulnerable when interacting with certain products and services, we can provide guidance on ethical practices that protect users while supporting business objectives."

HAL's robot helps researchers understand the way humans interact with technology. A current HAL project engages social robots to connect with students on behavioral interview training. This study is a pilot for research in partnership with a local organization providing services to adults with dementia in support of an approach called reminiscence therapy.

How can your company get involved?

HAL offers opportunities for organizations to collaborate with top researchers to develop innovative strategies for success. You can:

• Share data with HAL for analysis, insights, and solutions

• Support the work of emerging scholars by sponsoring a post-doctoral student

• Affiliate your company with some of the most innovative scholarship at the intersection of people with technology

For more information, contact us at HAL@nd.edu.

Further Reading:

Read on for highlights from scholarship at the intersection of people with technology

• This article highlights research how bots can affect human-to-human interactions in online communities.  

  Helper bots in online communities diminish human interaction

• This article develops and applies a measure of perceived corporate social responsibility and examines how it relates to employee attitudes. It is a good example of the full arc of human-centered research: conceptualizing a construct, building a measurement instrument, validating it, and then using it to test a substantive theory.

  The Effects of Perceived Corporate Social Responsibility on Employee Attitudes

• This recent paper on the critical challenges of using large-scale digital experiment platforms for scientific discovery. It is a special paper because it is co-authored by our now-deceased friend and colleague. It combines multiple skill sets, including information systems, psychology, methodology, and partnerships with industry, and addresses questions much broader than any single discipline could reach alone. It is also a paper that reflects the kind of collaborative, cross-boundary work that HAL is built around. 

  The Critical Challenge of Using Large-Scale Digital Experiment Platforms for Scientific Discovery

• This article describes a framework for thinking about how bias and fairness in AI can impact multiple stages of a data science pipeline, whether it is how data is represented or how downstream predictions affect end users:

  Should Fairness be a Metric or a Model? A Model-based Framework for Assessing Bias in Machine Learning Pipelines

• This research examines how to design replication studies. Replication, the ability to reproduce a finding, is a foundational principle of science. Yet there has been a widespread failure to replicate across many areas of social science in recent years. We argue that one of the important reasons is that replication studies are often not designed properly. The paper provides a framework for doing this correctly.

  Sample Size Planning for Replication Studies: The Devil is in the Design