Standardized State Assessments, a method of assessing psychological states in hypothetical situations, have recently been shown to be highly reliable and to have convergent validity with other self-report measures. These assessments require respondents to indicate their behavior in hypothetical situations in an open-response format and to rate their own behavior on a set of construct-specific self-report items. This study examines (a) how self-report ratings in Standardized State Assessments converge with human subject matter expert ratings of the same behaviors and (b) whether natural language processing capabilities of GPT can provide an economically attractive alternative to human ratings. Results showed that GPT provided highly reliable ratings of open-ended responses with small to moderate correlations with self-report ratings of the same open-ended response and with self-report trait measures. These results are discussed in relation to subject matter expert ratings as well as their relevance for future assessments.