) Organizations increasingly use Artificial Intelligence-Driven Decision-Support
Systems (AI-DSS) to augment employee decision-making and enhance
decision-making performance. However, optimal employee reliance on AI-DSS
remains a challenge, as employees often either over-rely on system
recommendations or reject them due to intuitive aversion. Existing research
focuses on AI explainability to improve employee reliance. However, the best
performing algorithms are based on complex, opaque models, and opting for
more transparent but less sophisticated models can reduce decision-making
accuracy. This makes AI explainability less appealing in high-stakes
organizational environments, and highlights the need for alternative
interventions to promote appropriate employee reliance.
This research focuses on human cognitive engagement in augmented decision
making. Building on dual-process theories, we propose that biases stem from
Type 1 (intuitive and automatic) thinking, leading to over-reliance or unjustified
rejection of AI-DSS advice. In contrast, Type 2 thinking (deliberate and
reflective) supports appropriate reliance. Type 2 thinking can be activated by:
(1) increasing information accessibility (as in AI explainability) and (2) prompting
deliberate reflection. We propose that human explainability, wherein employees
articulate their reasoning for adjusting AI-DSS outputs, fosters metacognitive
engagement. Thus, employees are stimulated to critically evaluate their
adjustments and calibrate their reliance, ultimately improving decision-making
performance.
We conducted two studies in the context of demand forecasting. In a field study,
we analysed thousands of decisions from 60 demand planners over three years,
examining within-person (whether employees perform better in months where
they provide more explanations) and between-person (whether employees who
explain more perform better) effects. The second study experimentally
manipulates explanation conditions, comparing AI explainability, human
explainability and their combination in fostering appropriate employee reliance.
Data is currently being collected and analysed, and results will be presented at
the symposium.