Stanford Seminar - The State of Design Knowledge in Human-AI Interaction

()

Human-AI Interaction

Human-AI interaction focuses on building useful, predictable, and controllable systems using machine intelligence.
The field is relatively young and has produced useful knowledge, such as the design pattern of split user interfaces.
Many assumptions in the field have not been empirically tested and need to be verified.
Predictive text keyboards make people's writing more predictable and shorter by suggesting easily available words, leading to simpler captions.
Biased language models can impact the content of people's writing, affecting how positive or negative their reviews are perceived to be.
AI-assisted decision-making can lead to poorer decision-making when people over-rely on the AI's suggestions, especially when the AI is incorrect.
Explanations for AI decisions may not improve the situation and can even lead to over-reliance on the AI if people perceive the presence of explanations as an indication of competence.
Cognitive forcing techniques, such as asking people to make their own decisions before seeing the AI's predictions, can help reduce over-reliance on AI and encourage deeper engagement with the information provided.
Researchers are exploring alternative AI designs such as evaluative AI, question-focused AI, and shared decision-making support systems.

Clinicians in mental health value shared decision-making and want systems that support conversations between clinicians and patients rather than taking the clinician's attention away from the patient.
Interactive AI systems should reflect all relevant knowledge, including patient preferences and side effects, and allow fast exploration of "what-if" scenarios.
Providers want to evaluate AI systems ahead of time through randomized clinical trials but do not want to validate every decision recommendation.
They would like contrastive explanations for deviations from clinical guidelines but not for every decision.

AI can make consequential decisions about our lives, such as loan applications or unemployment benefits, and grievance redressal mechanisms are being demanded.
Counterfactual explanations tell people exactly what they need to do to succeed, but they assume the algorithm knows everything about what is possible and easy for a person.
Reason codes give a larger range of options and make it easier to make correct decisions, even when the algorithm's model is misaligned with reality.
People applying for public benefits often do not know or believe they are eligible and need help determining eligibility and prerequisites.
Collecting information for applications can be effortful and costly, and people may not provide all possible details.
The current approach of providing counterfactual explanations for AI-assisted decision-making is insufficient and does not meet the stated goals.
Giving people decision recommendations and explanations alone does not lead to more thoughtful, engaged, or better decisions.
A different paradigm for human-AI interaction is needed, one that focuses on providing personalized support and guidance throughout the decision-making process.
Counterfactual explanations should not only tell people what they should do next but also support their decision-making by helping them understand whether they should reapply or argue for an exception.

Knowing the accuracy of an AI model can influence how humans take its advice, but it requires constant reminders to be effective.
Ensembling independent human and AI judgments can produce good decisions, but it does not address the societal need for human accountability and understanding of the decision-making process.
Gathering human input in a way that makes it indistinguishable from AI input could help reduce bias, but it depends on the domain.
The emotional field or instincts can play a significant role in decision-making, and it is important to consider their impact in experiments and interventions.
Time pressure negatively impacts the quality of AI-human joint decision-making processes.
There is a need to accelerate production design in AI.
Some people who are highly involved in productizing AI may not be paying attention to the potential negative consequences.