Digging Deeper into Dialogic Feedback: Evaluating How Science Teachers Manage Uncertainty as a Predictor of Students’ Ability to Construct an Epistemically Sound Argument

In this study, we examined the relationship between the use of two teachers’ dialogue feedback as an educational practice to promote evidence-based argumentation in middle school science lessons and the students’ ability to create scientific arguments in a standardized critical thinking exam. The teachers had an equal amount of training on Argument-Based Inquiry (ABI) and taught in a federally-identified low-income school. When the patterns of talk were analyzed, divergent themes emerged and feedback that promoted critique correlated with student achievement on the critical

students provide an environment where the process of engaging in argumentation can take place, and the results of that endeavor are where they construct a scientific argument as a product of their work. McNeill and Knight (2013) discuss the importance of having both a structural and process definition for this type of teaching where they use the term "argument" to describe the artifacts students create to articulate and justify claims and the term "argumentation" to describe the process of generating these artifacts (p. 938). We draw on this work because our primary research is to attempt to measure if the teachers' argumentation process of creating an environment for learning through feedback patterns results in any difference in the quality of the students' product, measured by the argument they made on a critical thinking exam using a claim, evidence, and reasoning framework.
One process for promoting argumentation in the classroom is dialogic teaching, which is a pedagogical approach that involves students sharing control over crucial aspects of classroom discourse through collaborative construction of meaning (Alexander, 2017). However, changing classroom discourse practices from traditional modes of instruction towards more reform-based approaches, including argumentation, has been challenging for teachers (Bråten, Muis, & Reznitskaya, 2017). In some cases, teachers struggle with implementing argumentation because it involves giving students increased control over the classroom discourse (Windschitl & Stroupe, 2017). This introduces a new level of uncertainty at the dialogic level of argumentation, and learning how to manage uncertainty during argumentation lessons productively has become a necessary skill for teachers in reform-based classrooms.
According to the Framework for K-12 Science Standards (National Research Council [NRC], 2012): "Scientific knowledge is a particular kind of knowledge with its sources, justifications, ways of dealing with uncertainties, and agreed-on levels of certainty" (p. 251). Scientists identify uncertainty in data, sustain that uncertainty through argumentation, and eventually find solutions to seek a level of agreed upon certainty (Asterhan & Schwarz, 2016;Ford, 2012). The idea of observing how teachers manage uncertainty through talk moves is the focus of this study because many science classrooms only emphasize the product of argumentation and transmit that knowledge in the form of lecture or a teacher-led demonstration but provide little opportunity for students to experience how that knowledge was created using the epistemic rules of scientific thinking (NRC, 2012;Kapur & Bielaczyc, 2012).
In our research, we focused on two resources of measuring how the teachers in the study created an environment where the students engage in the process of argumentation. First, we calculated the www.scholink.org/ojs/index.php/wjer World Journal of Educational Research Vol. 7, No. 1, 2020 38 Published by SCHOLINK INC. percent of teacher talk and student talk during lessons where students were asked to make sense of data they collected and how it helped them answer the question they investigated. Second, we looked at specific patterns of dialogic feedback from the teachers during those lessons to determine if any specific talk move patterns affected the students' scores on a critical thinking essay assignment that focused on claims, evidence, and reasoning. The outcome of student achievement on a standardized critical thinking test was selected because of the ability to construct a claim that has evidence, and reasoning was determined to be a quality measure of an argument as a product. In the following sections, the theoretical framework for the study, methods, findings, and scholarly impact will be presented.

Argumentation Process: Managing Student Uncertainty
Reform-based science instruction asks teachers to adopt a more diverse range of instructional aims that include not just the traditional notions about conceptual learning in science but also that this learning should be guided by authentic science practices like modeling, engineering, and argumentation (NGSS, Lead States, 2003). Practitioners are asked to manage uncertainty so that students have opportunities for learning how actual scientific thinking results in evidence-based solutions through the process of arguing through uncertainty (Ford, 2012).
Research in the area of teacher talk (Michaels & O'Connor, 2015) supports the idea of teachers managing uncertainty by raising doubt (i.e., asking students to explore phenomena), maintaining doubt (i.e., asking students to construct claims and critique competing claims), and reducing doubt (i.e., checking with vetted resources to determine which claim to support).
Looking at how teachers manage uncertainty during moments of social negotiation is an area that needs more focus because it is a major shift from delivering lecture or other teacher-centric pedagogy. For example, (Kuhn, Rinehart, & Milford, 2019) found that traditionally oriented teachers used feedback to raise uncertainty by asking a question, but only maintained uncertainty long enough to identify student misconceptions, and then they reduced uncertainty by providing the correct answer. In that same study, reform-based oriented teachers maintained uncertainty by asking students to evaluate their understanding of the question, provided feedback that presented a critique of the idea, and asked students to defend their ideas with the backing of evidence and reasoning. The more refined feedback positioned students to listen to alternative ideas and then support or challenge the evidence that supported the claim in question. The Kuhn et al. (2019) study only looked at teacher feedback and did not consider the impact on students' ability to construct claims on their own. In this study, we were interested in the type of dialogic feedback that teachers used and evaluated if it would align with student achievement on a task that required critical thinking skills and the ability to defend choices based on evidence and reasoning. In the next section, we will describe dialogic feedback in greater detail.

Using Dialogic Feedback to Create An Epistemic Environment for Argumentation
Broadly defined, dialogic teaching is a pedagogical approach that involves students sharing control over crucial aspects of classroom discourse through collaborative construction of meaning (Alexander, 2017). In the context of science education, dialogic feedback encourages a fruitful discussion of science concepts that requires peer-to-peer meaning-making opportunities, which ask learners to make sense of information from various sources and engage in the process of constructing claims and critiquing the evidence of opposing ideas (Resnick & Schantz, 2015). Learning progressions for argumentation describe a transition from less sophisticated practices, like only making a claim, to more sophisticated practices like constructing one's claim with supporting warrants and data as well responding to the claims, warrants, and data provided by another's counterargument (Berland & Reiser, 2011;Osborne et al., 2016).
Teachers who use a dialogic approach ask students to consider a range of ideas and pose questions to students as they explore and discuss different points of view to manage uncertainty (Jordan & McDaniel Jr., 2014;Manz, 2015). Teacher and student feedback would rarely verify whether a student is "right" or "wrong"; instead, the teacher would ask students to clarify, generalize, or expand on claims presented to the class (Berland & Reiser, 2011). Teachers who use an interactive dialogic approach would likely build upon the student's understanding of the phenomena and help guide it toward a current scientific understanding (Scott, Mortimer, & Aguiar, 2006). However, as noted earlier, this type of instruction is rare, and a feedback protocol that models interactions aligned with reform-based teaching would help teachers adapt their practices to meet the expectations of the contemporary science standards.
Promoting argumentation as a social practice where ideas can safely be vetted in the public domain is essential for teachers attempting to promote the process of science (Berland, 2011). An initial step in the process of creating more autonomous learners is to signal to students that their meaning-making discussion with their peers is worth the time it takes to flesh out ideas and allow them to reduce uncertainty by process of arguing about the evidence (Osborne, Erduran, & Simon, 2004). Dialogic interactions enable students to become learners with agency, rather than passive receptacles of information (Polman, 2004). Furthermore, students build an understanding of science through the processing of information through multiple forms of activities and media, including teacher-peer and peer-peer discourse.
The way that teachers use dialogue is one of the most critical decisions that they make to communicate with their students that their ideas are important to the learning process. Through talk, for example, teachers can choose to highlight specific ideas over others, ask students to consider alternative ideas, present competing claims to the students, and ask them to provide evidence and reasoning about why individual claims deserve support and why others do not (Alexander, 2017). Conversely, teachers could simply tell students which idea is correct, which sends a message that teachers are not interested in their ideas and are more concerned with compliance and memorization.

Research Questions
The present study aims to explore the relationship between teachers' use of dialogic feedback practices, aspects of their classroom environment, and student achievement on a critical thinking assessment.
Specifically, we investigated the following questions: (1) What is the relationship between teachers' dialogic feedback and middle school student achievement on critical thinking assessments?
(2) Are there any patterns of teacher talk correlate with student achievement on critical thinking assessments?

Method
In the study, two 6th grade teachers and their students (n = 217) in a large metropolis school district in the southwest United States served as the participants. The teachers in the study had finished the first year of a multiyear professional development designed to help teachers develop foundational skills and dispositions to promote ABI in their classrooms. Specifically, teachers were given a general framework based on the Next Generation Science Standards (NGSS) in the United States (NGSS Lead States, 2013) and aligned to the Argument-Based Strategies for STEM-Infused Science Teaching (ASSIST;  where they were asked to (a) allow students to explore phenomena, (b) give students agency in developing questions to investigate, (c) ask students to analyze the results of the investigation and engage in argumentation to promote meaning-making, (d) ask students to compare their claims to the consensus of the scientific community, and (e) communicate understanding through multimodal writing. Participants self-selected to attend the five-day workshop and received a stipend for completing the interview and submitting videos of their instruction.
Each teacher was asked to record ten 45 minute videos (two per unit throughout the year; two in September, two in December, two in February, two in March, and two in May), in which students were asked to analyze the results of an investigation and engage in argumentation to promote meaning-making. Using lessons recorded after students collected data allowed the researchers to evaluate the types of dialogic feedback used by teachers to support evidence-based argumentation.
Specifically, teachers were asked to "Record a typical lesson after your students have collected data from an investigation".
Initially, each reviewer independently coded a random sample of four transcripts of videos using an a priori coding scheme from the teachers' videos as dialogic or non-dialogic using the framework presented by Scott et al. (2006, pp. 611-612). A Cohen's κ was conducted to determine if there was an agreement on whether or not the feedback was dialogic or not, and a high level of agreement was found between the two coders (κ = .801 .p < 0.001).
Next, the twenty transcripts of the teachers' videos were coded using a constant comparison method based on the type of dialogic feedback provided by the teacher. Using the list of dialogic feedback utterances collected from the original analysis, the coders then re-evaluated the feedback. They generated the following codes using a grounded theory approach (a) reframing the conversation, (b) elaboration, (c) reflection, (d) construction, and (e) critique (Table 1). Another Cohen's κ was conducted to determine if there was an agreement between two reviewers on the type of dialogic feedback based on the five categories (reframing, elaboration, reflection construction, and critique) and there was a high agreement between the two reviewers, (κ = .779 p< .001). This coding scheme was used on all twenty videos, and when a teacher made a talk move that aligned with the description of one of the codes it was noted, and the aggregate for each category was used in the analysis (see Table 2 for a total of each talk move).
Finally, each student in the teachers' classes was given two modified versions of the Illinois Critical Thinking Test (Finken, 1992) in the fall of 2017 (before any science instruction) and in May of 2018.
The Illinois Critical Thinking Test was chosen to measure the quality of the students' argument because the rubric for the assessment used a claim, evidence, reasoning framework similar to Toulmin's argument framework (Toulmin, 1958), which was the focus of the ASSIST science curriculum that the teachers enacted.
Before administering the assessment, teachers provided a prompt of "Please read the question at the top of the page, think about your answer, read the available evidence and then write the best scientifically-based answer you can that explains the reason why you support your idea over others." Next, students were provided with three documents of evidence supporting the argument and three documents of evidence that present a counter-argument. Students wrote an essay where they were asked to report if they support the claim or not, provide evidence, and validate their decision through reasoning (Note-the question in the fall was "Do you think technology should be added to cars that disable all cell phone use once the car is turned on? The spring question was, "Do you think violent video games cause students to behave violently?"). Finken's (1992) rubric was used to score the student essays from both fall and spring were analyzed by two reviewers, and a correlation analysis found a high level of inter-rater agreement (κ = .791, p< .001).

Result
Initially, we calculated the amount of student talk and teacher talk during the argumentation lessons.
These data were collected using the software from the recording device that had a teacher microphone and desk microphones for the students. These non-parametric data showed us that the two teachers in the study used a similar amount of time talking, and the students in their classes used nearly an identical amount of time talking in both classes (see Table 2). The fact that the two teachers let students talk through ideas shows that autonomous learning was promoted in both classes, and each teacher was focused on students attempting to work through their ideas through talk. Data in Table 2 was collected to demonstrate that the two teachers used a similar amount of talk in the ten lessons. If one of the teachers used significantly more time talking than the other or if one of the teachers' students spent significantly more time talking than the other, it may explain any disparity in the results. However, the data in Table 2 tells us that the teachers used very similar amounts of time talking in the ten lessons.
With this information we were able to eliminate that possible variable and focus on the amount and type of dialogic feedback each teacher used. However, there was a difference in the type of dialogic talk that the two teachers used. According to Table 3, the Teacher 1 used a relatively even amount of dialogic feedback for reframing the conversation, elaboration, reflection, and critique (17%-20%) and slightly elevated feedback that promoted construction (28%). Teacher 2 relied heavily on construction feedback (45%) and elaboration (27%) and had much less in the other categories, especially critique where we only observed 3% of the dialogic feedback in that category. The data from Tables 2 and 3 indicate that both teachers allowed the students to use talk as a tool to learn science, but Teacher 2 promoted feedback that attempted to help students construct their claims and explain their evidence. Teacher 1 also spent a lot of class time having students talk to develop their claims, but in that classroom, students were asked to reflect on their understanding, think about how their ideas aligned (or did not) align with the views of their peers, and critique the ideas of others more than Teacher 2's classroom. Note. Each teacher was asked to record ten forty-five minute videos of them teaching a science lesson where there would be a high level of student argumentation. The mean number of dialogic feedback occurrences is noted for each type of feedback.
Next, a t-test was conducted to determine if there was a significant difference in critical thinking scores between the two groups of students at the beginning of the year, and no significance was found (see Table 4). Another t-test of the May test was conducted, and Teacher 1 students scored significantly higher on the critical thinking test than Teacher 2's students (see Table 5).  An analysis of the teachers' dialogic feedback was conducted, and Teacher 1 used significantly more feedback coded as reframing, reflection, and critique. Teacher 2 used a significant amount more feedback coded as construction (see Table 6).

Discussion
There are multiple findings of this research. For example, one professional development implication of this work is that a route to helping novice teachers move toward more reformed approaches of teaching might involve helping these teachers diversify the type of talk moves that they use. The literature outlined in the theoretical framework points out that science teachers writ large are not meeting the expectations of reform-based standards, so simply asking them to engage students in argumentation will not provide enough context for instruction. For example, Teacher 2 understood that students should be involved in the discussion, but relied primarily on a few specific types of dialogic talk moves.
Asking students to provide evidence for their claims (a move used frequently by Teacher 2) is a worthwhile first discussion point, but it only asks students to reflect on the idea that they support. Even asking students to elaborate on why they support that idea (another move used frequently by Teacher 2) continues to focus on the construction of the students' claim. However, the focus of reform-based standards and the nature of science asks investigators to consider alternative ideas and make a decision based on the merits of the evidence and if the evidence connects to the claim through reasoning. If professional development providers want more productive discussion in the classroom it would be beneficial to include ways to have students compare ideas and focus on evidence and reasoning rather than simple construction of ideas.
The broader range of dialogic feedback suggests that Teacher 1 had a more diverse set of instructional techniques and had a better sense of how to navigate the complexity of teaching lessons where the focus is student-centered argumentation. Professional development providers should take note and provide examples of talk moves that consider more than asking students, "what do you think?" During these moments of dialogue, teachers can use their words to invoke knowledge advancement by making students use different resources of social negotiation (i.e., raise, maintain, and reduce doubt) to manage their uncertainty.
For example, both Teacher 1 and 2 used a similar strategy to raise uncertainty, by asking the students to make a claim about the question they were investigating, the data they collected, and observations they made during the investigation. It was clear that both teachers had aligned themselves toward a pedagogical approach influenced by ABI because a more traditional move would be to tell the students what the data they collected meant and how it answered the question, they were investigating. The significant differences in the two teachers' talk moves became more apparent when they attempted to maintain uncertainty. Teacher 1 attempted to extend dialogue by asking students to think about one idea at a time and dig deep into the reasons why students should support or reject the claim. In doing this, Teacher 1 invited critique into the discussion, which was a talk move that was lacking in Teacher 2's transcripts. According to Table 3, Teacher 1 used dialogue that invited critique 20% of the time compared to only 3% for Teacher 2. It is possible that the students in Teacher 1's class benefited from the way that uncertainty was maintained because students were asked to take an idea, consider if there was sufficient evidence to support it, and list potential reasons why the claim might be flawed. It is possible that using an instructional approach that asked students to consider multiple claims and not influencing their reasoning by guiding them to the answer improved their achievement on the critical thinking test. Essays that explained why they supported one idea and provided specific reasons why the alternative idea lacked evidence would receive a higher score than essays that only provided evidence for the claim they supported. Consistent practice with this type of learning may have helped students in Teacher 1's classroom improve their ability to reason.
For example, look at a transcript from a discussion that occurred between Teacher 1 and a group of students (note the class was discussing ideas about their project related to NGSS Performance Expectation-MS-PS2-1. Apply Newton's Third Law to design a solution to a problem involving the motion of two colliding objects).
T (move 1): So, who has an idea about what we could do to make street scooters safer for students? I know a lot of you have crashed into things, and some of you have been hurt, so this is a real issue. S1: What if we added a bunch of pads to the side of the scooter and, like the handlebars and stuff. S2: You could just wear lots of pads, like elbow pads, knee pads, and helmets.
T (move 5): Okay, class, we have a few different ideas. One is to add padding to the scooter, and one is to add padding to the ridder. Here is what we are going to do. We are going to have a table discussion, and I want you to follow these steps. First, write the question we are trying to figure out at the top of your paper. Next, write out the two ideas, protecting the scooter and protecting the rider, next ask your table partners if they have any other ideas if they do add their ideas to the list. Then, I want you to make a T chart like this (the teacher draws a T-Chart on the board) and write Pro on this side and Con on this side. I want you to do this with each idea. I am going to go around right now and assign a member of your group who will be the recorder and write all this down on paper at your table (teacher goes around the room and assigns one student as the recorder).
Next, focus on one of the ideas, and each person in the group will make their individual T-chart in their science journal similar to the T-chart on your group paper. Each member of the table will write out all the reasons why we should support the idea under "Pro" and all the reasons why you think we should not support the idea under "Con". Finally, everyone will share what they wrote for their pros and cons, and the recorder will write all the ideas on the big paper. Everyone got it? (The Majority of the class responds "yes"). Okay, just in case you forgot what to do, I wrote out the steps for you to follow, and In the vignette, Teacher 1 raised uncertainty by asking students to think of a solution to a real-world problem affecting their community (people getting hurt on rideshare scooters) and allowed student 1 to make a claim. Next, uncertainty was maintained at move 2 when the teacher decided to invite other students to comment instead of offering an evaluation, which would substantially reduce uncertainty and end the peer-to-peer nature of the conversation. Teacher 1 continued to maintain uncertainty by asking student 2 to offer a counter solution rather than just a critique of the initial claim in moves 3 and 4. Finally, at move 5, the students had to consider both ideas and write out the pros and cons of each.
This was another example of how Teacher 1 offered a peer-centric way to maintain uncertainty by asking the students to consider each argument and debate their merits. Later on, in this lesson, Teacher 1 asked the students to read multiple sources of evidence that helped them make an informed decision about which argument they should support and the physics behind how helmets and pads keep people safe. During those lessons, the teacher provided conflicting reports, and the students had to decide which claim was more accurate. The students evaluated the evidence and were responsible for deciding the accuracy of the documents, thus giving them a say in how uncertainty was reduced.
Teacher 1 never reduced uncertainty by telling the students which argument to support, but instead, put the ownership back on the students and asked them to make a claim backed with evidence. Mainly, Teacher 1 was in charge of raising uncertainty but then asked the students to take control of the maintain and reduction phases and only interjected by providing resources and asking students to consider multiple options.
Now, look at the contrast in the vignette below between how Teacher 2 handled moments where opportunities of uncertainty management presented themselves. T (move 6): Why a helmet, why would that help people who rent the scooters?
S3: It helps so that if you crash, it doesn't break your brain.
T (move 7): That sounds like a great idea! Let's read some more about this, everyone take out your computer and open the document I just sent to you, you will learn how a helmet works and why this idea is smart.
Teacher 2 raised uncertainty in a similar way that Teacher 1 did but then diverged in the way that they maintained and reduced uncertainty. At move 3, the teacher instantly rejected student 2's idea, thus providing no opportunity to maintain uncertainty and closing the potential learning opportunity available through peer-to-peer negotiation. Additionally, at moves 5 and 6, the teacher uses feedback that was coded as "elaboration", which was a common way that Teacher 2 maintained uncertainty.
However, the discussion was between one student and the teacher, and the element of critique was never introduced. Instead, Teacher 2 asked questions that were meant to encourage students to elaborate on their initial idea until they heard what they wanted, and then they provided a resource that confirmed the idea was correct.
This subtle difference in how the teachers in the study maintained and reduced uncertainty was highlighted in Table 3, where Teacher 1 used more feedback that reframed the question, asked students to reflect on their ideas, and encouraged them to critically examine if the idea had evidence backed with reasoning. It is unclear if the teachers had different aims or values in regards to their pedagogical beliefs about teaching science, but the fact that they allowed for similar amounts of student talk suggests that they both valued dialogic feedback. However, Teacher 1 was able to expand how they maintained uncertainty by asking questions that went beyond simply asking students to expand their thinking and providing evidence.
These data are an essential finding for the field of argumentation research because if the number of instructional aims that teachers value is greater, there are urgent needs for multiple pedagogical forms that would be used to address meeting each of these needs. One professional development implication of this work is that a route to helping teachers with traditional-oriented views of instruction move toward more reformed approaches might involve helping these teachers diversify the set of talk moves that they use. However, this might require professional development on the nature of science, why a dualistic view of science content and process is important, and asking teachers to promote discussion that invites critique. If the goal of science education is to improve scientific knowledge and an understanding of how that knowledge grows and is reliable, it is important that teachers put students in www.scholink.org/ojs/index.php/wjer situations where they are asked to evaluate claims, question evidence, and make epistemically sound decisions. The data from this study suggests that using instructional strategies that promote these ideals improves critical thinking in students.