This study sought to examine the alteration of the cut-off scores determined by cluster analysis according to 3 different sample sizes (250, 500 and 1000) and 3 different distribution types (Normal, Uniform and Beta) in data compatible with the 2-parameter Item Response Theory (IRT) model. To this end, 9 different simulation data consisting of 25 items were generated in WinGen3 program. The cut-off score was determined by dividing the individuals into two groups using the Two-Step Cluster Analysis method. The study revealed that the cut-off score determined for the actual score of individuals with normal distribution by cluster analysis method was 14.5 in the N=250 study group and 11.5 in the N=500 and N=1000 study groups. For the actual score of individuals with a uniform distribution, the cut-off scores determined in the N=250, N=500 and N=1000 study groups were 10.5, 12.5 and 11.5, respectively. For the actual score of individuals with beta distribution, the cut-off scores determined in N=250, N=500 and N=1000 study groups were 8.5, 10.5 and 9.5, respectively. The study concluded that the highest cut-off score determined for the actual score of the individuals with regard to sample size and distribution type was obtained in the N=250 study group with a normal distribution. The lowest cut-off score was obtained in the N=250 study group with a beta distribution. It was concluded that the lowest cut-off score for the entire study group size was found in the data set with beta distribution. The results suggest that the cut-off scores determined by cluster analysis may vary according to sample size and the type of distribution. Researchers are recommended to use cluster analysis method, which does not involve subjectivity, to determine cut-off scores in standard setting studies. Researchers may investigate how the cut-off score determined by cluster analysis will change for data sets generated based on different IRT models (e.g., 3-parameter logistic model).

Standard setting, Cluster analysis, Cut-off score, Wingen.