The increasing use of web and mobile surveys challenges traditional
validation methods. The "Web probing" method employs open-ended
questions to gather validity evidence on the response processes of web
survey questions and items. Data cleaning and response coding pose
significant challenges, particularly for "web probes," given the self
administered nature of "web probes". The integration of generative AI,
especially models based on GPT architectures, offers new opportunities
to automate these processes efficiently and accurately. In addition, new
developments could enable a more tailored administration of "web
probes" based on previous responses. This proposal focuses on the
development of a data post-processing solution for automated
debugging and coding of responses to web probes. This solution could
be implemented by utilizing advanced prompting techniques, such as in
an application of the GPT store, a standardized data processing
procedure that leverages these AI tools, or by an application that utilizes
the OpenAI API to offer advanced features, depending on the results or
performance of each option.
The objective of the paper is twofold: a) to present the development and
validation of a generative AI-based data post-processing application and
procedure that allows; and b) to illustrate how such a procedure,
depending on how and in which model it is applied, deductively codes
themes and sub-themes in the substantive responses, and automatically
detects indicators of low involvement in the response process that in turn
can affect web probing data quality.
Textual data from questionnaires on "quality-of-life" will be categorized

into substantive (1) and non-substantive (0) by coders. Subsequently,
this coding will be compared with the coding generated by four different
AI models (4th, fourth custom, O1, and Deepseek) and by the state-of
the-art OpenAI model using the API. Future developments of AI
generative for improving validation of response processes "in vivo" will
also be discussed.