Introduction: Self-report measures are widely used in psychological research, often through online platforms that facilitate administration and access to large respondent panels (Kim & Oh, 2022). While advantageous, such settings increase careless and insufficient effort responding (C/IER), where participants fail to attend adequately to items, producing poor-quality data (Podsakoff et al., 2012). Among the methods to detect C/IER, Instructed Response Items (IRI) (e.g., Mark the option "disagree") are valued for its simplicity, robust metric properties, and ability to identify different C/IER patterns (Kam & Chan, 2018). Previous experimental studies manipulated C/IER to analyze IRI adequacy; however further research is needed on contextual factors that may influence C/IER and affect IRI performance.
Purpose: This study aimed to assess the effect of questionnaire length and IRI percentage on the occurrence of C/IER.
Method: We conducted an experimental study with three data collections, manipulating questionnaire length (50, 100, or 150 items) and IRI percentage (4% or 7%). Questionnaire length was counterbalanced across the data collections, and IRI percentage randomly assigned. The Time 1 sample included 548 participants (53.8% men, aged 20-65). We assessed the dimensionality of the C/IER scale and modeled the relationship between C/IER (wrong IRI answers) and both questionnaire length and IRI percentage.
Results: Results partially supported the unidimensional structure of C/IER responses. Poisson regressions for count outcomes showed that questionnaire length predicted C/IER behavior, whereas IRI percentage predicted C/IER only in the first data collection.
Conclusions: These findings confirm that questionnaire length impacts C/IER, while the lack of consistent effects of IRI percentage suggest that varying their proportion does not influence the emergence or detection of C/IER. Further analyses are needed to clarify the IRI scale's non-unidimensionality, potentially linked to item format differences. Overall, the study offers guidance for managing C/IER efficiently in psychological research.