 |
Cristian Gugiu Associate Professor, Dept. of Educational Studies, College of Education and Human Ecology |
Thursday, July 20, 2017
|
Presentation Title: Graphical Methods for Performing Test and Survey Validation
Presentation description
Modern test and survey validation requires more than performing factor and reliability analyses or establishing concurrent and discriminant validity with other instruments. One must also examine the properties of the items in relation to the ability of respondents. While it is readily understood that testing the ability of sixth grade students in math provides little information when such tests are composed of calculus questions or simple addition and subtraction questions, so it is the case when surveys are composed of items that are either too easy or too difficult to be endorsed by respondents. This presentation will introduce attendees to graphical analyses that can be performed within a Rasch modeling framework for detecting floor and ceiling effects, gaps along the latent continuum, and redundant items; visualizing construct validity; contextualizing group differences; setting performance standards; investigating the performance of response scales; visualizing measurement error in relation to person ability; and detecting item bias.
Presentation outline
1. Why Rasch?
a. Observed score to Rasch score ICC
b. Item-free & person free measures
2. Wright map
a. Floor effects
b. Ceiling effects
c. Gaps along the continuum
d. Redundant items
e. Implications for reliability
3. Construct validity
4. Contextualizing group differences
a. ANOVA with Rasch modeling
5. Setting competency standards
6. Scaling
a. Disordered categories
b. Number of response categories
7. Test information and measurement error
8. Differential Item Functioning
a. Uniform DIF
b. Nonuniform DIF