Development and Validation of a Two-Tier Multiple Choice Instrument to Assess Students’ Science Process Skills in Acid-Base Chemistry
Abstract
This study aims to develop and validate a two-tier multiple choice (TTMCQ) instrument to measure senior high school students’ science process skills (SPS) in acid-base chemistry. Following Borg and Gall’s modified research and development design (nine stages), the instrument was developed, validated, and tested in two public high schools in Bengkulu, Indonesia. Validation involved content experts, assessment specialists, and experienced chemistry teachers. Field trials were conducted with 119 students, and data were analyzed for validity, reliability, item difficulty, and discrimination indices. Results show that the TTMCQ instrument demonstrated strong validity, high reliability, appropriate difficulty distribution (53% easy, 42% medium, 5% difficult), and sufficient discriminating power. Teachers reported positive perceptions, noting that the instrument effectively challenges students’ reasoning and enhances their SPS. The study highlights the potential of TTMCQ instruments as practical tools for formative and summative assessment in science education.
Downloads
References
Adauyah, R., & Aznam, N. (2024). Guided Inquiry Learning Model in Chemistry Education: A Systematic Review. Jurnal Penelitian Pendidikan IPA, 10(3), 77-87. https://doi.org/10.29303/jppipa.v10i3.6373
Ahmad, N. F., & Iksan, Z. H. (2021). Edmodo-Based Science Module Development on Students' Mastery of Science Process Skills: Need Analysis. Creative Education, 12(11), 2609-2623. https://doi.org/10.4236/ce.2021.1211195
Alioon, Y., & Delialioğlu, Ö. (2019). The effect of authentic m‐learning activities on student engagement and motivation. British Journal of Educational Technology, 50(2), 655-668. https://doi.org/10.1111/bjet.12559
Almoslamani, Y. (2022). The impact of learning strategies on the academic achievement of university students in Saudi Arabia. Learning and Teaching in Higher Education: Gulf Perspectives, 18(1), 4-18. https://doi.org/10.1108/LTHE-08-2020-0025
Astalini, A., Darmaji, D., Kurniawan, D. A., Wirayuda, R. P., Putri, W. A., Rini, E. F. S., Ginting, A. A. B., & Ratnawati, T. (2023). Impact of Science Process Skills on Thinking Skills in Rural and Urban Schools. International Journal of Instruction, 16(2), 803-822. https://doi.org/10.29333/iji.2023.16242a
Boekaerts, M. (2016). Engagement as an inherent aspect of the learning process. Learning and Instruction, 43, 76-83. https://doi.org/10.1016/j.learninstruc.2016.02.001
Bond, M., & Bedenlier, S. (2019). Facilitating Student Engagement Through Educational Technology: Towards a Conceptual Framework. Journal of Interactive Media in Education, 2019(1). https://doi.org/10.5334/jime.528
Damanhuri, M. I. M., Treagust, D. F., Won, M., & Chandrasegaran, A. L. (2016). High School Students' Understanding of Acid-Base Concepts: An Ongoing Challenge for Teachers. The International Journal of Environmental and Science Education. https://doi.org/10.12973/ijese.2015.284a
Darling-Hammond, L., Flook, L., Cook-Harvey, C., Barron, B., & Osher, D. (2020). Implications for educational practice of the science of learning and development. Applied Developmental Science, 24(2), 97-140. https://doi.org/10.1080/10888691.2018.1537791
Dejene, K., & Belachew, W. (2023). Topic-Specific Pedagogical Content Knowledge-Based Instruction and Level of Conceptual Understanding of Chemical Kinetics and Equilibrium concepts on Grade 11 Students. Science Education International, 34(2), 96-108. https://doi.org/10.33828/sei.v34.i2.3
Ekici, M., & Erdem, M. (2020). Developing Science Process Skills through Mobile Scientific Inquiry. Thinking Skills and Creativity, 36, 100658. https://doi.org/10.1016/j.tsc.2020.100658
El Nagdi, M. A., & Roehrig, G. H. (2022). Reality vs. Expectations of Assessment in STEM Education: An Exploratory Case Study of STEM Schools in Egypt. Education Sciences, 12(11), 762. https://doi.org/10.3390/educsci12110762
Gizaw, G., & Sota, S. S. (2023). Improving Science Process Skills of Students: A Review of Literature. Science Education International, 34(3), 216-224. https://doi.org/10.33828/sei.v34.i3.5
Grogans, S. E., Bliss-Moreau, E., Buss, K. A., Clark, L. A., Fox, A. S., Keltner, D., Cowen, A. S., Kim, J. J., Kragel, P. A., MacLeod, C., Mobbs, D., Naragon-Gainey, K., Fullana, M. A., & Shackman, A. J. (2023). The nature and neurobiology of fear and anxiety: State of the science and opportunities for accelerating discovery. Neuroscience & Biobehavioral Reviews, 151, 105237. https://doi.org/10.1016/j.neubiorev.2023.105237
Haekal, M. (2022). Tantangan Distribusi Guru di Daerah Terpencil Indonesia: Antara Manajemen, Isu Personal, dan Faktor Geografi. TA'DIB: Jurnal Pemikiran Pendidikan, 12(1), 18-23. https://doi.org/10.54604/tdb.v12i1.100
Hill, H. C., & Chin, M. (2018). Connections Between Teachers' Knowledge of Students, Instruction, and Achievement Outcomes. American Educational Research Journal, 55(5), 1076-1112. https://doi.org/10.3102/0002831218769614
Holzinger, A., Lettner, S., Steiner-Hofbauer, V., & Capan Melser, M. (2020). How to assess? Perceptions and preferences of undergraduate medical students concerning traditional assessment methods. BMC Medical Education, 20(1), 312. https://doi.org/10.1186/s12909-020-02239-6
Ilma, S., Al-Muhdhar, M. H. I., Rohman, F., & Saptasari, M. (2020). The correlation between science process skills and biology cognitive learning outcome of senior high school students. JPBI (Jurnal Pendidikan Biologi Indonesia, 6(1), 55-64. https://doi.org/10.22219/jpbi.v6i1.10794
Jenkins, J. L., & Shoopman, B. T. (2019). Identifying misconceptions that limit student understanding of molecular orbital diagrams. Science Education International, 30(3), 152-157. https://doi.org/10.33828/sei.v30.i3.1
Jovanovska, J. (2018). Designing Effective Multiple-Choice Questions for Assessing Learning Outcomes. Infotheca, 18(1), 25-42. https://doi.org/10.18485/infotheca.2018.18.1.2
Karpudewan, M., Treagust, D. F., Mocerino, M., Won, M., & Chandrasegaran, A. L. (2016). Investigating High School Students' Understanding of Chemical Equilibrium Concepts. The International Journal of Environmental and Science Education. https://doi.org/10.12973/ijese.2015.280a
Kasule, O. H. (2013). Overview of medical student assessment: Why, what, who, and how. Journal of Taibah University Medical Sciences, 8(2), 72-79. https://doi.org/10.1016/j.jtumed.2013.07.001
Lukum, A., Karongkong, N., Pikoli, M., Tangio, J. S., Mohamad, E., & Rewini Kunusa, W. (2023). Analysis of Student's Conceptual Understanding Using Two-Tier Multiple Choice Diagnostic Test on Acid-Base Topic. E3S Web of Conferences, 400, 04003. https://doi.org/10.1051/e3sconf/202340004003
Maison, M., Hidayat, M., Kurniawan, D. A., Yolviansyah, F., Sandra, R. O., & Iqbal, M. (2022). How Critical Thinking Skills Influence Misconception in Electric Field. International Journal of Educational Methodology, 8(2), 377-390. https://doi.org/10.12973/ijem.8.2.377
Misbah, Z., Gulikers, J., Dharma, S., & Mulder, M. (2020). Evaluating competence-based vocational education in Indonesia. Journal of Vocational Education & Training, 72(4), 488-515. https://doi.org/10.1080/13636820.2019.1635634
Nugroho, I., Aufa, M., Chasanah, N. N., & Wardani, D. A. (2023). Analysis of Higher Order Thinking Skills and Science Process Skills in 21st Century Education (pp. 729-735). https://doi.org/10.2991/978-94-6463-284-2_78
Nur'ariyani, S., Jumyati, J., Yuliyanti, Y., Nulhakim, L., & Leksono, S. M. (2023). Scientific Approach to Learning Science in Elementary Schools. Jurnal Penelitian Pendidikan IPA, 9(8), 6659-6666. https://doi.org/10.29303/jppipa.v9i8.3680
Özer, F., & Sarıbaş, D. (2023). Exploring Pre-service Science Teachers' Understanding of Scientific Inquiry and Scientific Practices Through a Laboratory Course. Science & Education, 32(3), 787-820. https://doi.org/10.1007/s11191-022-00325-3
Permatasari, M. B., Rahayu, S., & Dasna, I. W. (2022). Chemistry Learning Using Multiple Representations: A Systematic Literature Review. Journal of Science Learning, 5(2), 334-341. https://doi.org/10.17509/jsl.v5i2.42656
Primasari, R., Miarsyah, M., & Rusdi, R. (2020). Science literacy, critical thinking skill, and motivation: A correlational study. JPBI (Jurnal Pendidikan Biologi Indonesia, 6(2), 273-282. https://doi.org/10.22219/jpbi.v6i2.11124
Purtadi, S., Suyanta, S., & Rohaeti, E. (2023). Science Process Skills as Learning Outcomes in Inquiry-Based Learning: A Literature Review. Jurnal Penelitian Pendidikan IPA, 9(12), 1403-1414. https://doi.org/10.29303/jppipa.v9i12.4774
Rahmatika, R., Amin, M., Al-Muhdhar, M. H. I., & Suwono, H. (2022). Scientific literacy refinement at Islamic junior high schools using socio-science spirituality learning model. JPBI (Jurnal Pendidikan Biologi Indonesia, 8(1), 40-50. https://doi.org/10.22219/jpbi.v8i1.18989
Rahmawati, Y., Zulhipri, Z., Hartanto, O., Falani, I., & Iriyadi, D. (2022). Students' conceptual understanding in chemistry learning using PhET interactive simulations. Journal of Technology and Science Education, 12(2), 303. https://doi.org/10.3926/jotse.1597
Rintayati, P., Lukitasari, H., & Syawaludin, A. (2021). Development of Two-Tier Multiple Choice Test to Assess Indonesian Elementary Students' Higher-Order Thinking Skills. International Journal of Instruction, 14(1), 555-566. https://doi.org/10.29333/iji.2021.14133a
Rusmini, R., Suyono, S., & Agustini, R. (2021). Analysis of science process skills of chemical education students through self project based learning (SjBL) in the pandemic COVID 19 era. Journal of Technology and Science Education, 11(2), 371. https://doi.org/10.3926/jotse.1288
Saher, A.-S., Ali, A. M. J., Amani, D., & Najwan, F. (2022). Traditional versus authentic assessments in higher education. Pegem Journal of Education and Instruction, 12(1). https://doi.org/10.47750/pegegog.12.01.29
Salame, I. I., Ramirez, L., Nikolic, D., & Krauss, D. (2022). Investigating Students' Difficulties and Approaches to Solving Buffer Related Problems. International Journal of Instruction, 15(1), 911-926. https://doi.org/10.29333/iji.2022.15152a
Schellekens, L. H., Bok, H. G. J., de Jong, L. H., van der Schaaf, M. F., Kremer, W. D. J., & van der Vleuten, C. P. M. (2021). A scoping review on the notions of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). Studies in Educational Evaluation, 71, 101094. https://doi.org/10.1016/j.stueduc.2021.101094
Schildkamp, K., van der Kleij, F. M., Heitink, M. C., Kippers, W. B., & Veldkamp, B. P. (2020). Formative assessment: A systematic review of critical teacher prerequisites for classroom practice. International Journal of Educational Research, 103, 101602. https://doi.org/10.1016/j.ijer.2020.101602
Sidek, R., Halim, L., Buang, N. A., & Mohamad Arsad, N. (2020). Fostering Scientific Creativity in Teaching and Learning Science in Schools: A Systematic Review. Jurnal Penelitian Dan Pembelajaran IPA, 6(1), 13. https://doi.org/10.30870/jppi.v6i1.7149
Simonson, S. R., Earl, B., & Frary, M. (2022). Establishing a Framework for Assessing Teaching Effectiveness. College Teaching, 70(2), 164-180. https://doi.org/10.1080/87567555.2021.1909528
Sıbıç, O., AKÇAY, B., & ARIK, M. (2022). Review of Two-tier Tests in the Studies: Creating a New Pathway for Development of Two-tier Tests. International Journal of Contemporary Educational Research, 7(2), 81-98. https://doi.org/10.33200/ijcer.747981
Soeharto, S., & Csapó, B. (2022). Exploring Indonesian student misconceptions in science concepts. Heliyon, 8(9), e10720. https://doi.org/10.1016/j.heliyon.2022.e10720
Sun, L., Kangas, M., Ruokamo, H., & Siklander, S. (2023). A systematic literature review of teacher scaffolding in game-based learning in primary education. Educational Research Review, 40, 100546. https://doi.org/10.1016/j.edurev.2023.100546
Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J. M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., El Hayek, S., Augereau-Landais, M., Mourey, F., Feybesse, C., Sundquist, D., & Lubart, T. (2023). Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education. Journal of Intelligence, 11(3). https://doi.org/10.3390/jintelligence11030054
Uyulgan, M. A., & Güven, N. A. (2022). Analysis of Prospective Primary School Teachers' Knowledge Regarding Chemical Representations on Crystallization Experiment. Journal of Science Learning, 5(1), 176-192. https://doi.org/10.17509/jsl.v5i1.34772
Veldhuis, M., & van den Heuvel-Panhuizen, M. (2020). Supporting primary school teachers' classroom assessment in mathematics education: effects on student achievement. Mathematics Education Research Journal, 32(3), 449-471. https://doi.org/10.1007/s13394-019-00270-5
Williams, A. (2024). Delivering Effective Student Feedback in Higher Education: An Evaluation of the Challenges and Best Practice. International Journal of Research in Education and Science, 10(2), 473-501. https://doi.org/10.46328/ijres.3404
Yan, Z., Li, Z., Panadero, E., Yang, M., Yang, L., & Lao, H. (2021). A systematic review on factors influencing teachers' intentions and implementations regarding formative assessment. Assessment in Education: Principles, Policy & Practice, 28(3), 228-260. https://doi.org/10.1080/0969594X.2021.1884042
Yeo, J.-H., Yang, H.-H., & Cho, I.-H. (2022). Using A Three-Tier Multiple-Choice Diagnostic Instrument Toward Alternative Conceptions Among Lower-Secondary School Students In Taiwan: Taking Ecosystems Unit as An Example. Journal of Baltic Science Education, 21(1), 69-83. https://doi.org/10.33225/jbse/22.21.69
Zhou, S., Han, J., Koenig, K., Raplinger, A., Pi, Y., Li, D., Xiao, H., Fu, Z., & Bao, L. (2016). Assessment of scientific reasoning: The effects of task context, data, and design on student reasoning in control of variables. Thinking Skills and Creativity, 19, 175-187. https://doi.org/10.1016/j.tsc.2015.11.004
Authors who publish with ISEJ: Indonesian Science Education Journal agree to the following terms:
- Authors retain copyright and grant the ISEJ: Indonesian Science Education Journal right of first publication with the work simultaneously licensed under Creative Commons Attribution License (CC BY 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors can enter into separate, additional contractual arrangements for the non-exclusive distribution of the published version of the work (e.g., post it to an institutional repository or edit it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) before and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
