Examining a Multisemiotic Approach to Measuring Challenging Content for English Learners and Others: Results from the ONPAR Elementary and Middle School Science Study

Abstract

This study contributes to the empirical research base on the effectiveness of ONPAR, a promising multisemiotic test item development process. ONPAR uses a variety of multisemiotic performance techniques to present and measure challenging concepts and skills of students, including low English proficient English Learners (ELs) and non-ELs. Experimental trials were used to investigate how 648 ELs at three English proficiency levels and native English speaking non-ELs performed on randomly assigned 4th and 8th grade traditional items and equivalent ONPAR items of challenging science content. General linear modeling using a covariate variable of classroom performance and bi- and multi-nomial regressions found differential boost across both grades. That is, findings showed that lower English proficient ELs perform better on ONPAR vs traditional forms in both grades, with p < .05 in favor of ONPAR in grade 8, whereas there were no significant differences between the two forms for non-ELs. The results also underscore the viability of the assessment methodology where students often demonstrate their response by showing their knowledge and skills. Item level results indicate that the ONPAR approach is useful at mitigating the effect of group.