Tracktest English Test face validity
The measurement of user perception of whether the test measures what it is supposed to measure
Project details
Participants: 228 individual Tracktest customers which took the Core or Complete test in 2022 and responded to the feedback survey.
Dates: January-December 2022
Introduction:
The purpose of this analysis is to evaluate the face validity of the certificated Tracktest English tests. Face validity refers to the extent to which a test appears to measure what it is supposed to measure, based on the subjective perceptions of test takers. Face validity is a form of content validity and is an important first step in the validity evaluation process (APA, 2014).
Method:
To evaluate the face validity of the Tracktest English test, we conducted a survey with individual customers who used the Tracktest Core or Complete test. The survey asked the customers three days after they completed their first English test whether they believed their test result reflected their actual English knowledge. This question was part of the longer survey where they answered other questions for the Net Promoter Score (NPS) study. The responses were collected via Google Forms during the year 2022 and analyzed using descriptive statistics.
Tracktest English test shows the positive indication of the face validity as most customers believed their test result accurately reflected their English knowledge.
Results:
Of the 228 customers who responded to the survey, 66.7% answered: “Yes, I think it was mostly accurate”, indicating that they believed their test result reflected their actual English knowledge. This is a positive indication of the face validity of the Tracktest English test.
On the other hand, 9.2% of customers answered: “I think my test result was BETTER than my actual knowledge”, and 17.1% answered: “I think my test result was WORSE than my actual knowledge”.
Conclusion:
Based on the results of this study, the Tracktest English test appears to have positive face validity as most customers believed their test result accurately reflected their English knowledge.
Face validity is an important first step in the validity evaluation process, but it should not be the only method used to assess validity. As noted by Messick (1995), face validity alone cannot establish validity because it is based on subjective perceptions and does not assess the underlying construct being measured. Therefore, further research is needed to establish the construct validity and reliability of the Tracktest English test which is addressed in other Tracktest validity and reliability studies.
References:
- American Psychological Association. (2014). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
- Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
- Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
- Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. McGraw-Hill.