The nature of licensing and certification tests has changed in recent decades. Once, such tests were built primarily based on what was taught in professional programs. A group of well-known professors would meet to decide what should be assessed. Today, legal decisions, at least in the United States, require that they be built on the results of practice or job analyses as such tests are employment tests if licensure is required for employment. They can only reflect curricular influences to the extent that such training relates to entry-level work performance. This requirement makes it increasingly difficult to assess one's professional skills using paper-and-pencil and even computer-administered tests in some professions. If such tests are used, there are often important aspects of the job that are neglected or missed on the examination.
To achieve validity in licensing and certification examinations, the key step is often determining the test plan based on the results of a practice analysis. When one limits testing to multiple-choice paper-and-pencil measures, then components of the job are absent from the assessment. Most such tests are justified based on content evidence of validation; under the Uniform Guidelines on Employee Selection Procedures, a professional job/practice analysis is hence required.
Using experts to help identify critical job components may generate misleading components because the requirements for licensing and certification are that one has minimally met the requirements of an entry-level professional. In many instances, experts have not dealt with such individuals, for a prolonged period. Moreover, if the experts come from academe, the test may overly reflect curricular aspects rather than those related specifically to the work of the professional.
After these validation topics are discussed, the importance of standard-setting, mode of testing, reliability, fairness, and generalizability will also be discussed. Examples from psychology, engineering, and dentistry will be provided.