CHECKLIST
EDUCATIONAL TESTS

                                     
A. Evidence of Validity                              
                                     
  1. When a test measures a construct, evidence should be presented that the test scores are highly related to that construct (include inter-item correlations, bases for weighting subscores, theoretical basis for the construct, and a description of the construct-related validity study).
           
                                     
  2. If a test is purported to be related to an outcome criterion, then evidence of the relationship between the test score and the outcome measures must be provided. Information should be available related to the rationale for choosing criteria, justification for selecting cut-scores, and a description of the criterion-related validity study.  
         
                                     
  3. When test content is especially important (as in achievement tests), content-related evidence of validity should be provided. Clear definition for the content universe, domain specifications, item specifications, and a description of the content-related validity study should be provided.    
                                     
B. Evidence of Reliability and Standard Errors of Measurement              
                                     
  1. Methods of calculating reliability coefficients and standard errors of measurement should be reported (include equations, and a description of the reliability study).  
                                     
  2. The psychometric procedures underlying the calculation of reliability indices should be specified (such as classical measurement theory, item response theory, etc.)            
                                     
  3. Limitations in the reliability indices should be specified. For example, how does the index vary with changes over populations, the effects of adjustments for attenuation (e.g., due to restriction of the measurement range) and the effects of speededness (due to testing time limits).      
                                     
C. Test Development Procedures                          
                                     
  1. Domain definitions and specifications should be provided. Similarly item and test specifications should be clearly reported.  
                                     
  2. Item selection procedures should be indicated (including statistical item performance data, item bias information, and psychometric criteria for selection.)        
                                     
D. Test Administration                                
                                     
  1. Administration procedures should be standardized to reduce unwanted variation in test scores. The same administration manuals, training and testing conditions should be used.          
                                     
  2. Steps should be taken to insure test security, proper distribution and collection of materials, and a minimum of respondent burden.
                   
E. Scoring and Reporting                
                   
  1. Scoring procedures should be standardized, documented and checked for accuracy.  
                   
  2. The analysis of item responses or test scores should be clearly described (such as scaling, norming, equating, etc.)    
                   
  3. Score reports should be promptly reported in a standard format that is easy to read.      
                   
  4. The generalizability and limitations of reported scores should be presented.  
                   
F. Special Testing Conditions            
                   
  1. Testing modifications should be provided for examinees with handicapping conditions or language differences.        
                   
  2. When feasible, the validity, reliability and other indices of test integrity should be investigated for special populations.
                   
G. Rights of Test-Takers                
                   
  1. Proper consent should be obtained prior to test administration.    
                   
  2. Rights of confidentiality should be guaranteed for all uses of the data.