Introduction
Instructed response items (IRIs) are a common method for detecting inattentive or careless responding in psychological survey research. Yet, failures are often treated as equivalent, without considering that different types of IRIs may be missed for distinct reasons, such as brief lapses of attention, complete disengagement, or content-related resistance.
Purpose
To understand the functioning of different types of IRIs, their failing rations, why they are failed and the alignment between the reasons and the intended purpose of IRIs.
Method
Data was collected using a panel of respondents. In total 339 participants completed the survey. Up to nine IRIs were embedded in surveys of random varying length (50, 100, or 150 items). IRI position was also randomized to control for order effects. Participants who failed IRIs were asked to provide reasons for their responses. Explanations were analyzed to identify inattentiveness categories (e.g., not reading carefully, distraction, disengagement) and other motives such as denial, defensiveness, technical difficulties, or disagreement with item content.
Results
Failure rates varied from 1.7% to 12.8%, suggesting that item content influence compliance. Thematic IRIs (e.g., technology, science) showed more failures and often elicited reinterpretation, while simple directives mainly reflected inattentiveness. Self-referential IRIs, such as "I am capable of following an order, therefore mark 'totally agree'," tended to provoke denial or defensiveness.
Conclusions
Although the sample of participants that failed the items and provided reasons is quite small, results suggest IRI failures cannot be uniformly attributed to same levels of inattention. Failures reflect diverse processes depending on item type, ranging from minor lapses to resistance. Validating different IRI types is essential to understand attentional problems and improve the accuracy of data quality assessments in psychological research.