Development of a Critical Thinking Assessment Instruments for ‎Use with Sc-SEBION Learning Media

Abstract

Critical thinking is an essential competency in twenty-first-century education. However, Indonesian students’ critical thinking skills remain relatively low, highlighting the need for innovative learning media that support higher-order thinking. Sc-SEBION (Syzygium cumini–Explosion Box Interactive Online Version) was developed as an interactive digital learning medium to facilitate critical thinking through contextual and engaging learning experiences. To evaluate its effectiveness, a valid and reliable assessment instrument is required. This study aimed to determine the validity and reliability of a cognitive assessment instrument developed to measure Grade VII junior high school students’ critical thinking skills on environmental change topics. The instrument, consisting of pretest and posttest forms, was developed using the 4D model (Define, Design, Develop, and Disseminate). Both forms comprised 12 essay items representing six critical thinking indicators proposed by Facione: interpretation, analysis, evaluation, inference, explanation, and self-regulation. Specifically, items 1–2 measure interpretation, items 3–4 measure analysis, items 5–6 measure evaluation, items 7–8 measure inference, items 9–10 measure explanation, and items 11–12 measure self-regulation. The pretest and posttest were parallel forms with different items but identical constructs, blueprint, and weighting, indicating equivalence in content and structure. The instrument was then tested for validity and reliability. Content validity was evaluated by four validators, comprising three university lecturers and one science teacher, while empirical validity and reliability were analyzed using Pearson item–total correlation and Cronbach’s Alpha in SPSS version 29. Empirical validity and reliability were tested on 21 students outside the research sample. Content validity scores ranged from 91.7% to 96.5%. Empirical validity coefficients ranged from 0.606 to 0.927 for the pretest and 0.657 to 0.916 for the posttest, exceeding the critical r-table of 0.433. Cronbach’s Alpha values were 0.936 and 0.934 for the pretest and posttest, respectively. The findings provide preliminary evidence of the instrument’s validity and internal consistency for assessing critical thinking skills in Sc-SEBION-based learning. However, this study is limited to the examination of validity and reliability without extending to other psychometric evaluations or large-scale implementation studies. Given the limited pilot sample, further studies with larger and more diverse samples and additional construct validation analyses are needed to strengthen the psychometric evidence.