This study aimed to examine, revise, test, and evaluate the items produced in mathematical literacy in terms of language, structure, and content and create an item pool containing valid, reliable, and applicable items. The study method was based on the formative assessment of development research types. Thus, small group tests were carried out with 8th grade students, field tests with 9th and 10th graders. The small group tests consisted of 20-30 students, and 3053 and 2918 students participated in the first and second field tests. A total of 102 items were tested. Following the small group tests, student solutions were analyzed, and revision procedures were carried out accordingly. At the end of the first and second field tests, item analysis was completed through 27% upper and lower groups method. According to the first field test results, 33 items were revised, considering the item discrimination index and the accumulation rates in distracter items among the upper and lower groups. Those items were re-tested in the second field test. The structural and cognitive factors that affect the validity were included in the results related to the second field test analysis of the revised items. These factors would contribute to the revision methodology literature. At the end of the study, there were 94 items, of which the discrimination index was r_jx≥0,30. Besides, the mathematical literacy sub-dimensions measured by each of the given items were determined within this study's scope. The items, mathematical literacy sub-dimensions, and item statistics were in the appendixes. It is suggested that using the given items in mathematics education research, in-class activities, and measurement and evaluation activities will be beneficial for students to develop mathematical literacy.

Item analysis, item pool, mathematical literacy, PISA, revision methodology.