Gathering Criterion-Related Evidence of Validity

Gathering criterion-related evidence of validity is an important task for all language testers. This task is particularly difficult for the CAEL test given some of the unique features of the CAEL Assessment. That is, the use of constructed-response test items in a topic-based fully integrated language test is essentially a unique approach to language testing at the present time. These aspects of the CAEL Assessment strengthen the claim made by test developers that the CAEL Assessment is a reasonable approximation of the language demands of English for academic purposes, particularly in Canadian university contexts. However, the essentially unique nature of the test means that gathering criterion-related evidence of validity is problematic.

CAEL test scores have been compared with the performance of test takers on the Test of English as a Foreign Language (TOEFL). However, the TOEFL is clearly measuring English language proficiency in a very different manner. Does this mean, then, that a correlation of the two test scores provides criterion-related evidence in support of the CAEL test? Clearly, the establishment of an appropriate criterion is always a challenge when gathering this type of validity evidence.

One procedure that has been adopted in an effort to gather more meaningful criterion-related evidence of validity is to conduct follow-up studies of CAEL test takers who score at various proficiency levels. One such follow-up study was conducted for this manual. The university course performance of 79 test takers who achieved an Overall Result at a band score of 70 or greater was collected. The basic design was to determine the grade point averages (GPA) of these students in their first full term of study after achieving an Overall Result of 70 or greater on the CAEL Assessment. The score of 70 was selected for this study because test takers who achieve this score are permitted to register for regular courses at the university without any further ESL/EAP (English for Academic Purposes ) training. Data was collected for each test taker for the term immediately following their CAEL Assessment in an effort to avoid measuring the impact of language learning which occurred after the test was completed.

Advertisements

, , , , , , , ,

  1. #1 by Linda Thema on November 21, 2011 - 6:47 pm

    Excellent one !!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: