Innovate to Mitigate: Science Learning in an Open-Innovation Challenge for High School Students

In this exploratory study, we report results from hosting two rounds of an open innovation competition challenging young people age 13-18 to develop a method for carbon mitigation. In both challenges, teams worked within the classroom and extensively on their own time out-of-school. The challenges were structured to engage participants to work collaboratively and independently in an open-ended, goal-oriented way, yet constrained their work by the parameters of the challenge, and supported it by a suite of tools, and resources. Evidence of learning science concepts and practices, student persistence, and the enthusiasm of participants, teachers and coaches, convince us that the Challenge structure and format is highly worthy of further development and investigation. Our findings indicate that Challenges such as this have the potential to enlarge the “ecosystem” of learning environments in the formal education system.

In particular, the emphasis on production of knowledge and of a designed product is crucial since it requires all participants to be "producers", which in turn leads to higher-order thinking skills (Gee, 2011). In addition, working in a free-choice collaboration allows learners to shed the constraints of an (institution-imposed) school identity (Gee, 2000(Gee, , 2005, which frees them to engage, learn, and participate as scientists would. Henry Jenkins, a key theorist about participatory culture, points out that interaction within a "knowledge community" builds critical social skills and cultural competencies for youth, e.g., collective intelligence (the ability to pool knowledge with others toward a common goal), judgment (the ability to evaluate the reliability and credibility of sources), and negotiation (the ability to "travel across diverse communities, discerning and respecting multiple perspectives, and grasping and following alternative norms") (Jenkins, 2009, p. 106). A competition that offers a rich real-world challenge and that can accommodate divergent thinkers-a key feature of crowdsourcing (King & Lakhani, 2013)-can offer participants a new, potentially transformative learning environment in a technology-rich learner-centered context (Luckin, 2010).

Using Social Media
Since the majority of youth now engage in regular active creation of online content (Lenhart & Madden, 2005), daily use of the internet (Lenhart et al., 2010), and social media (Lenhart, 2015), we expect them to feel comfortable working in a hybrid face-to-face and online community.
Social media can be seen to embody a social constructivist view of knowledge as decentralized, accessible, and co-constructed by and among a broad base of users (Greenhow et al., 2009).
Seventy-five percent of teens age 12-17 own cell phones, and the average teen sends 1,500 texts a month. Online social networking benefits youth by providing and exchanging information and feedback, help from peers with school-related tasks, reinforcement of identity, and forming connections within and across geographic boundaries (Greenhow & Burton, 2011). Findings reveal that online social networking can include issue-oriented, argumentative writing (Beach & Candace-Stevens, 2011), or online chats in lieu of book reports (Hughes, 2016). Furthermore, teamwork facilitated by social media has also increasingly been deployed successfully by practicing scientists (Henry, 2016).
In addition, college-age and graduate students have used multimedia projects and social media as the basis for collaborating with others, discussing science with a wider community, and explaining scientific concepts to peers and family. For example, graduate student participants in the NSF IGERT video competitions (igert2013.videohall.com) expressed pride in their accomplishments, increased their self-identity with regard to science, and increased their sense of belonging to the wider scientific community (Stroud & Falk, 2015).
In this paper, we describe a mixed methods research study of a competition that engaged youth in two rounds of a challenge to mitigate climate change. We report outcomes related to three overarching research questions: (1) What was the nature of the Challenge experience? (2) What did students learn?

Round 1 Participants
Eleven teams signed up for the Challenge, but five teams dropped out within 2-3 weeks of the start of the competition period. They citing reasons such as conflicting schedules, engagement in afterschool sports or other activities, and completing college applications. Five teams ultimately submitted final projects to the video forum (http://www.innovatepilot.videohall.com). One team was at an American school abroad, one was a parochial school team, one a private school team, and two were at public schools. Nine girls and 7 boys participated; 11 students (68%) were white. Five of the 16 students were at the middle school level, while the remainder were high school students.

Round 2 Participants
We received 104 abstracts in the qualifying round. Fifty-four of the entries qualified for the final round, and a total of 23 individuals or teams, totaling 74 participants in all, submitted a video and paper (archived at innovate2015.videohall.com). All but one of the participants heard about the competition through their teacher, and all of the teams conducted their work under the coaching of their science teacher. The majority of participants reported attending U.S. schools, while 2 teams attended International schools, whose students noted their home states in the U.S. Median age reported was 16, and the range was 13 years (2 students)-18 years (2 students). Median grade level reported was 10 th grade and the range was 7 th grade (1 student)-12 th grade (3 students). Twenty participants were female, and 29 were male; two students chose not to share this information. Participants self-identified as Asian American (30 students), or White (15), while other ethnic groups were underrepresented with respect to their prevalence in the population as a whole (African American=1, Hispanic=4, other=2, not reporting=4). We do not have data on population demographics of the school districts in which the teams were situated.

The Challenge Design
The Challenges incorporated many of the essential features of crowdsourcing:  A widely broadcast invitation to participate (Howe, 2008);  A rich real-world challenge that could accommodate divergent thinking (King & Lakhani, 2013);  A combination of intrinsic and extrinsic rewards (He et al., 2014;Brabham, 2008;NRC, 2007);  A subsidy for the investment cost of participation to optimize the number of contributors-a materials stipend for participants in the first challenge (King & Lakhani, 2013).
Features of both rounds included: i) A first call to enter the Challenge publicized widely through postings on Facebook, as well as emails, which included a problem statement about mitigation, and an invitation to solve the problem.
ii) Teams could access the project website, which featured breaking stories about inspiring research projects that are currently producing potential mitigation solutions-from news outlets, links to Youtube videos, and reports in popular science blogs-to inspire creativity and seed ideas. Also available were a brief introduction to the science of climate change, and mitigation and adaptation.
iii) Prospective teams of participants submitted an abstract outlining their mitigation idea. iv) Teams brainstormed and developed the solution they had outlined in their abstract over a period of several weeks. They were helped by a local coach with execution, problem-solving, and logistical challenges, and the science/engineering content if they had the necessary expertise.
v) Teams submitted their projects to the online video forum, using the TERC Videohall (Note 1) developed by another team of researchers at TERC. During this forum, each was judged by a panel of four scientists. Teams were encouraged to respond to questions from the judges. Submissions were open for a public comment period.
vi) Prizes were awarded for innovation, best video, best poster or paper presentation, and most engaged participant in community comment. In addition, a community choice award was made for the submission that received the most likes in the video forum.

Round 1: September 1 2014-February 29 2015
In addition to the features just described, this round included: i) A graduate student mentor, recruited by the project, for each team. Mentors received an orientation via webinar, about the project, the competition and website functions, and mentoring tips on features to look for in student work, how to stay in contact, and how to ask productive questions.
ii) The website featured a password-protected "team space" for each team where they could post progress reports, store resources, and discuss their work.
iii) Submitted abstracts were open for a short period for public comment and questioning.

Round 2: January 22-May 15 2015
In the second round, we issued a wide call for proposals via an email to a list of over 75 names, generated from a wide range of teacher, environmental education, and informal education organizations, and our own network of educators. It included a link to the Inspirations page on the project website, which led to research and development information about current innovations in mitigation. A url for entering the competition was also provided, as well as information about the structure and requirements of the competition. These included the submission of an abstract that briefly described the proposed innovation and a statement about why submitters considered it to be innovative. We also included a media toolkit for dissemination in recipient's own social media venues.
Abstract submissions in this qualifying round were open for crowdsource-like public comment for a period of three days, after which a panel of three project scientists reviewed the abstracts, using a rubric that evaluated innovativeness, feasibility, and potential for impact. Since we also wanted the review to be educative, two project staff gave written feedback. Qualifying participants were notified that they had proceeded to the final round.
For the final round, the qualifiers were invited to make a two-minute video pitch for how and why their approach would work, and to predict the possible mitigation impact that their idea would have if it were implemented. They were also required to write a 1200-word essay that justified the argument for their for a community choice award, and a "best critic" award for the person who posed the most meaningful queries and comments to their peers in the discussion forum. After the event, the site was placed in archive mode and the content and discussion remains available.

Research Questions
We used mixed methods research to address the following overarching questions and sub-questions: 2) What did students learn?
-What does an analysis of student artifacts (video and paper) reveal about student learning?
-What are students' perceptions of their science learning, and of the nature of science?
3) To what extent did the competition support student innovation? -To what extent did judges rate projects as innovative?
-What were student perceptions of their own level of innovation and creativity?

Data Sources
Data presented from Round 1 are participant self-report from a student survey. Data presented for Round 2 are participant self-report from a pre/post survey; abstracts, videos and papers submitted to the videohall; the judges' ratings of participants' submissions; and a semi-purposive interview of the teacher whose students comprised the largest proportion of participants. Questions were intended to learn about the context in which the teacher had invited her students to enter, what her perceptions of their experience had been, and what amount of class and out-of-school time they spent on the project.

Student Post-Survey
Because of the open nature of the Challenge competition, and the resulting need for participants to remain anonymous, they participated voluntarily in the survey. The survey drew on a range of established tools developed for other research projects. We selected items related to student motivation (e.g., "I have always been a motivated learner in science"), persistence (e.g., "I usually finish tasks even if they are difficult") and self-concept towards Science, Technology, Engineering and Math (STEM) (e.g., "I think of myself as capable in science"). Items for student motivation came from the Science Motivation Questionnaire (Glynn & Koballa, 2006), persistence from Student Persistence in Engineering survey, and self-concept towards STEM from College Biology Self-Efficacy Instrument (Baldwin, Ebert-May, & Burns, 1999). Nature of science items were drawn from Views of Nature of Science (Khishfe & Abd-El-Khalick, 2002).

Artifact Review
Competition submissions were coded using a rubric created by the project to gather data related to our conjectures about student learning of science practices, and level of creativity and innovation, as follows: Science practices: Codes related to the presence of a clear problem statement, a theoretical framework, a model underpinning the mitigation strategy, prediction about impact, development and testing of a prototype solution, and appropriate citation of literature.

Innovation:
The extent to which the mitigation strategy proposed built on others' prior work, and the degree to which it was innovative (that is, was it a new entirely inventive idea that broke rules and conventions, a new development of an existing idea that used common materials and/or ideas in new ways, or not innovative).

Judge's Rubric
Judges were asked to rate the overall quality of the science evident in each submission; the potential of the idea for future development as a feasible mitigation effort; the level of innovation of the submission, the extent to which the team broke rules and conventions or used common materials and/or ideas in new ways; the quality of the paper and video presentations; and, the quality of student responses to judge's queries in the discussion forum in the video hall (Table 1). Judges completed a survey after the competition to give their overall impressions of the level of innovation they encountered in the group of projects they rated. Questions addressed the extent to which students' ideas were innovative; the extent to which presentations were creative; whether any projects, ideas or teams sparked their interest, and why; and, overall the extent to which they think projects might have potential for future development.

Data Analysis
Fifty-four participants completed the survey; we discarded data for three students because they either did not complete Likert-scale items, or entered meaningless answers for open-ended questions, or both.
We analyzed data from both rounds qualitatively and quantitatively, and triangulated the data if possible.
Quantitative data gathered from online Surveys and Google Analytics were analyzed with Excel. Since we deconstructed previously validated instruments to create measures for surveys appropriate to our project, we needed to determine validity. Face validity of instruments was determined through careful expert review by our advisors. We created and used a directed coding scheme for open-ended response items in the survey related to our research questions, and for student submissions, based on pre-defined codes for level of innovation and creativity. All project submissions, and participant responses were coded by two researchers, reaching 85% inter-rater reliability. Where disagreements occurred, coders discussed the differences and established an agreed coding.

What Was the Challenge Experience
The results reported here relate to student motivation to enter the competition, the extent to which it sustained their participation, the nature of their experience in the video poster hall, and the use of social media.

Entering the Competition
In Round 1, students were not asked this question. Students entering Round 2 were asked on a beginning survey to check as many reasons as applied for what motivated them to enter the competition.
For the majority, the opportunity to feel like they were doing something about mitigation, or that they cared deeply about the topic, were among the primary reasons stated (Table 2). Just over half of the students (28 of 54) reported that they entered as a class, and were required to enter the competition by their teacher. Almost half said that they found the prize money motivating. In response to a final question on the survey, "Anything else you'd like us to know about you?" one participant wrote, "My friends and I love your process as it gives students a great opportunity to think and collaborate like in the real world while the cash incentive motivates those who only want money". Students were also asked to what extent addressing climate change had been something they had thought about before entering the competition. Three quarters of the participants reported that they had thought about it "a lot" or "somewhat" (Figure 1), the remainder responded "just a little" or "not at all".  To ascertain whether students' "environmental identities" were a factor that motivated entry, we asked a series of proxy questions that might indicate prior interest in environmental issues, such as whether they had studied environmental science at school, and what was their experience in environmental activities, and their relationship to nature. Just over half of the students (28) agreed with the statement, "I would describe myself as someone who loves nature". Likewise, just over half (29) had done an environmental science course prior to the competition. Students were asked about their participation in environmental activities at home, at school, and in the community. Recycling was the most common activity (at home=49 students; at school=46 students). A small number of students appear to have stronger environmental identities as evidenced by their membership in environmental clubs (9 students), or having started a school (7 students) or communitywide (5 students) environmental effort. Twenty reported that they were at schools that were identified as "green" or that supported green efforts.

Sustaining Scientific Inquiry
Teams that completed the challenge in Round 1 took six months overall to develop and test their prototype idea and submit a final product. However, in Round 2, the overall time spent on the project was three months from abstract submission at the beginning, to final project submission. We will return to this point in the discussion.
Given that all of the teams that entered the competition were school-based, we wondered when they would complete the majority of the work on their project. In response to the prompt, "I/we completed most of our work…", the greatest number of students (19 students) reported working only outside of school hours, either after school (10 students) or afterschool and on weekends (9 students) ( Figure 2).
The remaining students worked some combination of school and out-of-school hours.

Figure 2. Times When Students Reported Working on Their Project (n=37)
Pam Matthew, a teacher whom we were able to interview in Round 2, wrote in an email: Initially, I thought that my students didn't have time to compete and do all that we wanted them to […]  What helped to sustain student participation and persistence? The challenging nature of the competition, its direct connection to an important phenomenon, and the invitation to be creative and innovative, were clearly appealing to students. When asked to check all items that applied to complete the statement, "I loved that the competition was…" over three quarters of the Round 2 students checked "involved creative thinking", over two thirds that the "solution could include anything", and just under two thirds that it involved problem-solving and allowed them to explore big ideas. Finally, two fifths also loved that the competition was challenging (Table 3). Allowed me to explore big ideas 34 Was challenging 22 As a body, the Round 2 students were overwhelmingly self-motivated, thought of themselves as capable in science, and found science highly engaging, as indicated by their ratings of statements regarding these constructs (Table 4). This was a contradiction of one of our conjectures going into the project. We conjectured that a science competition such as this would allow students who were not considered successful in science by conventional means, or who were not motivated by classroom science, to find a "niche" in which to excel. Science is boring 0 0 1 10 40 In response to a final question on the survey, "Anything else you'd like us to know about you?" one participant wrote, "I had a really fun time working on this project and as long as a great and wonderful idea is picked I don't care what the outcome will look like. Thank you for giving me this opportunity because I think it has helped me understand the importance of finding a clean renewable source of energy in greater depth. So once again thank you!". Another wrote, "Science was not much [sic] strong suit, but I was really excited about this project. I would love to continue to formulate this idea after the competition". A similar sentiment was expressed by 50% of the students who rated the following statement, "I will continue to pursue ideas about what we addressed in our project in the future" as "very true" (13 students) or "somewhat true" (14 students).

Participation in the Virtual Poster Hall
Round 2 students were asked to rate their experience participating in the video hall. When asked how the online presentation compared with face-to-face poster sessions they have participated in previously-as in a science fair, for example, just under one-third (17 students) rated the experience better than face-to-face sessions they had participated in. Interestingly, a similar percentage thought that the experience was the same as a face-to-face poster session (18 students). Over a fifth (12 students) rated it a worse experience; we did not ask for students to explain their ratings, so we do not know the reason for this rating. Approximately four-fifths of the students responded "somewhat" or "very much" to a statement about learning how to make an effective video, and how to communicate their research (Table 5). In response to a final question on the survey, "Anything else you'd like us to know about you?" one participant wrote, "The online presentation was very helpful, especially for students with anxiety because there is no pressure to be perfect and meet face-to-face with strangers".
Each presentation received queries from four judges, and presenters replied to these queries. This sparked informed scientific dialogue, which was made public after the competition ended. In total, there were 195 judge's queries and student replies. The judges' ratings of the quality of student responses to their queries was included in the overall judging score. Students responding to the question, "How valuable were the judge's queries?" stated that the queries were "somewhat" (20) or "very" (30) valuable (4 students did not respond). In addition, a general discussion area invited comments from colleagues and visitors to the site. There were 433 posts in the general discussion area. Many of these posts were content rich and originated from other teams in the competition, for example asking substantive questions about the submission.
Others were more social, for example, expressing congratulations for fine work, or best wishes for a future in science. The majority of students also felt that the general discussion in the public forum in the video hall was "somewhat" (38) or "very" (8) valuable, while 5 students did not think it valuable.
Queries and responses from the discussion and judge's forums are presented in greater detail in the student learning section below.

Use of Social Media
As has been found for other competitions or showcases staged at videohall.com (Falk et al., 2012), the competition provided a valuable opportunity for students to share their submissions with members of the public. The Video hall supports social media use in connection with competitions, providing links to Facebook and Twitter, as well as a unique URL for each video. Over a third of the teams (37%) reported sharing with family, and with friends (43%), while just over a fifth shared with their teacher (22%) (Figure 3). This enabled students to bridge their nascent scientific identity to their personal identity and social network, an increasingly important aspect of authentic science practices (Eagleman, 2013), and an important way to share science with the public (Tachibana & Zelinski, 2014).
Self-report data can be triangulated with data from website analytics. During the phases of the competition prior to the Video hall event, only a small minority of students visited the project's Facebook page or followed the project's Twitter feed (16% and 9% respectively). However, during the event (June 08-15, 2015), analytics showed an overall total of 443 shares and 397 likes on Facebook, and 67 shares using Twitter. The discrepancy between Facebook and Twitter use is not surprising, given that teens are on Facebook much more regularly, and use Instagram or Snapchat instead of Twitter (Pew, 2015).

Figure 3. Number of Students Who Shared Their Video
Also during the poster hall, a total of 433 public discussion posts were made, and 1,898 "Public comments and/or questions in the public discussion forum, while the remaining submissions had between one and ten public comments. In the private judging forum, there were 195 judges' queries and presenter's replies. In both the private and public forums, participants engaged in rich scientific discussions with judges and with members of the public. These data are presented below, under student learning.

What Students Learned
In this section, we discuss findings from Round 2 on student learning from their projects. Results presented here focus on science content, science practices, and the nature of science, drawn from two data sources: student self-report on the survey, and coding and analysis of student submissions.
Over half of the final projects submitted (13 of 23) focused on some aspect of alternative energy generation. Five of these described some method for capturing kinetic energy, two deployed a system for artificial photosynthesis, five focused on photovoltaics, and two captured and converted thermal energy. Five projects described ways to farm sustainably through local production of food in cities, three explored ways to sequester carbon through tree planting or the use of algae, two described innovations that improved efficiency (of lithium batteries, PDA chargers), and one focused on a regulatory mechanism for supporting the use of hybrid vehicles. It was notable that 30% (7 projects) were interdisciplinary. For example, one project focused on the chemistry involved in carbon sequestration methods, and considered the economic aspects of these methods, another integrated battery technology and microbiology, and yet another integrated agriculture, economics and social dynamics.

Science Learning
An overwhelming majority of students reported that they understood a moderate amount or a lot about climate change and about the specific area of their project when asked to rate their learning at the end of the competition (Table 6). In a retrospective open response item about what they had learned (At the beginning of the project I knew…, Now I know…), the specificity of student responses varied. Less than a fifth (8 of 43 students) made vague statements that they knew a little about a topic at the beginning (e.g., "not much about climate change"), and more at the end of their project (e.g., "more about climate change").
Approximately a quarter (10 students) reported with a little more specificity, e.g., "that solar energy existed" at the beginning, and "the different methods and problems [for] solar energy producers in the  Two students wrote about the personal impact of the learning they had done ("Now I know about the greenhouse gas effect from different sources and am excited to continue searching for and creating solutions", "Now I know the basics and I can hold a conversation about them with others"). Or about the urgency of climate change (e.g., "Now I know that if we don't act soon we'll be under water by 2050"). Ten students did not complete this item.
Participant responses in the video hall discussion forums provide a more in-depth snapshot of student learning, that we can use to triangulate with the self-report results. Overall, students displayed a depth of knowledge across a wide range of domains, and individual participants across different topics within a domain. The first example is from the "Cogediv" team, a group of three 10 th graders that was awarded first place by the judges. This team had described an innovation in which artificial leaves are mounted in arrays of various sizes. The leaves consist of catalysts and fuel cells embedded in a resin. The catalysts replicate photosynthesis, and the fuel cells immediately convert the glucose generated into electrical energy. Small installations can provide energy at the family level, while large ones could power factories. The cost of manufacturing these leaves could be offset by the fuel cell electricity production […] they can be arranged into overlaid structures that enable maximum gas capture. The following excerpt shows an exchange between one of the judges, and a team member: In this exchange, we see Denali's deep understanding not only of the chemistry involved in her proposed mitigation strategy, but also of having done additional research required to demonstrate its feasibility, i.e., citing the EPA statistics, while acknowledging the frontier engineering concerns that currently obstruct underground sequestration.
In another example, the Photoelectrics team designed transparent solar cells as smartphone covers to continually charge the phone during use, thus obviating the need for a plug-in charger. One of the judges raised a concern about the efficiency of the solar cell indoors: In this response, we see Hana's additional information that Hana and her team had gathered to considered the feasibility of their solution, but not included as part of their submission.

Science Practices: Communicating Results
Students were asked, "To what extent do you think you constructed an evidence-based explanation to support your innovation idea?" On a scale of 1 ("not at all") to 5 ("fully and completely"), almost three-quarters of the students (74%) rated their arguments as 4 (24 students) or 5 (16 students), showing that they thought their argument was scientifically sound. Seven rated their explanation at 3, and one at 2, indicating that they were feeling equivocal about the quality of their explanation.
Qualitative analysis of the submissions confirmed these data, revealing that all of the teams made clear problem statements about the mitigation strategy, and the overwhelming majority (91%) provided a coherent supporting argument backed up with evidence. Team KR, for example, proposed the They addressed several considerations to support their proposed design, and create a coherent argument that supports their design. For example, they describe the costs and benefits in terms of carbon sequestration rates, maintenance costs, and aesthetics, as well as providing data on proposed efficiency, and estimates for daily capture of CO 2 .
Almost half of the teams explained the mitigation strategy using both qualitative and quantitative data, while 25% used only qualitative data and 25% provided only quantitative data. Not surprisingly, given the structure of the challenge in contrast to the design of the first round, only three teams developed and tested a prototype of their chosen mitigation strategy. Interestingly, these submissions all came from younger students.
When asked to explain their rating for the degree to which their argument was evidence-based, several students gave specific and detailed responses. The following two statements are typical of this group, "A lot of our project was based on evidence found based on extensive research. We included large amounts of data based on the prior findings of the artificial leaf and solar panels", "We did a good job using realistic numbers. However, some of the numbers were less reliable as they were averages", and "We used published statistics to create plausible values for our innovation's impact". Another identifiable group of respondents acknowledged that, although they cited data to back up their proposed innovation, they did not gather the data themselves, but used data from the literature, for example, "We have good evidence to support our idea, however we did not gather the evidence ourselves, [a] scientist at MIT gathered it through experimentation". One team demonstrated media literacy, recognizing that data from the internet may not always be trusted, "All experiments can always be proven more thoroughly, but I can name a few flaws in ours. Our car emission estimate came from an average car... maybe a Honda Accord.
But the thing is we didn't test it ourselves. My group and I got it from the internet which may not have been reliable".
Another group stated that their case was evidence-based because they had spent a lot of time working on it, "My group spent countless hours researching the mechanisms in our project. We spent many evenings working out calculations in our project and deciding on how to maximize efficiency".
Thirty five percent of the teams provided a prediction of the impact of their chosen strategy clearly. The Subway Solutions team prediction is representative of this group. They proposed the installation of piezoelectric energy generators in subways to harness the kinetic energy from passing trains. While this has been experimentally tested, the team proposed adding pressure-plate flywheel generators, resulting in higher energy conversion. Citing data from research on piezoelectric systems under roadways, they translate this to their proposed system and predict 25-35% more efficiency:

Trials run with a piezoelectric system laid under one kilometer of a single-lane highway yielded nearly
200KWh or 720MJ. Assuming that this experiment was a best-case scenario and that a consistent amount of pressure was applied throughout the length of highway tested, the maximum weight of the cars on the highway is approximately 347,753 kg. Further calculations show that a similar mechanism implemented in a subway system would produce 222KWh (799.2 MJ), an 11% increase on the data provided. This is a significantly greater amount of energy production based on just one aspect of our method. Taking into account the increased pressure in the subway system, as well as the increase in the uniformity of the subway system, our entire method could be anywhere from 25-35% more efficient.
With regard to reading the scientific literature, an essential scientific practice for this competition, almost three-quarters of students (39 students) thought that they had become "more proficient," while eight students reported that they were proficient ("Yes, but still found it difficult").
As already mentioned above, student-student discussion in the video hall forum revealed student engagement in scientific give-and-take characteristic of professional scientific meetings. For example, a solo 11 th grader, Callie W, proposed a vertical farm in a multi-level high-rise building, that included passive heating/cooling systems, as well as renewable energy (e.g., biomass, geo-thermal and solar).
Housing livestock and hydroponic growing systems, the farm would house humans in the top floors.
The following exchange between another participant and Callie also reveals a depth of knowledge and skill in debating the relative merits of a project design with regard to mitigation: Mat M (participant, PHSgreenbeans team): Controlling the climate indoors will use a lot of energy, which could counteract the benefits of such an idea. How can you modify your idea to accommodate this fact? Also, what is the cost of building and maintaining such a building? The energy generated from the solar panels will not generate enough for the building to be self-sustaining.
Callie W: Indoor climates do use higher energy but by use of the 5 different renewables plus the 6 other energies/reclaims the farm will produce more energy than it uses and use fewer resources to produce more food. This is explained in my paper above. Solar energy will only account for 20% of the energy load. other renewables already included in the design are; Wind, Biomass, Thermal, Waste and methane gas. All of these form the design and construction of this farm. The cost estimate for this project would be around 1 billion US dollars to construct each farm. However, compared to its size and production this works out at about the same cost per square meter as a normal commercial farm. However, the vertical farm would require a tenth of the space and a fifth of the energy consumption of a normal commercial farm.
In this exchange, we see Mat M raising a valid concern regarding the feasibility of Callie's mitigation idea; if maintaining passive systems and using renewable energy were counteracted by the quantity of energy needed, how could the solution be justified. This question provoked a thoughtful response in which Callie attempts to counter the critique with an argument she had already used, but also comes up with additional counterfactuals to buttress her original claims.

Nature of Science
In response to the statement, "I understand more about the nature of science as a result of doing this project", well over half of the students (59%) rated the statement "true" or "somewhat true". Nine students were neutral about the statement, and about 25% rated the statement "somewhat false" or "false".
In addition to this question, we asked students a retrospective question about how they thought that science advanced. Student responses fell into three main categories ( Table 8). The first reflected expanded and possibly more nuanced views on the nature of science. For example, students recognized the importance of careful processes, the need for taking time and making effort, and for trial and error "as well as" innovation. They also identified the need for particular dispositions as part of the nature of science, for example, naming "creativity" "curiosity" "hard work" and "perseverance". The second category consisted of responses that could be said reflect an expanded view of who does science, and possibly shifts in personal identity with regard to science. For example, students included "people like you and me" and "newcomers" or "anybody" with new and good ideas. The smallest category included a view of science as problem-driven. In this view, science advances as a result of applying research to address a problem. Three students did not respond, and five provided the same vague answer (e.g., "hard work" "experiments") to both questions.  We also asked students to rate the statement, "Science is useful for understanding everyday problems".
Every student but one agreed that the statement was true or mostly true.
Finally, we asked students to check items that applied from a provided list of general skills, all relevant to the practice of science or to knowledge about mitigating climate change, that they thought they had learned because of participating in the competition (Figure 4). Interestingly, to "think outside the box" was the most frequent response (66%), followed by "collaboration" (59%), and being able to "build on previous knowledge" (57%). Just under a half of students (46%) checked that they had learned that there were "no easy solutions" to the challenge of climate change. Two other attributes important to a student's capacity to do science, the ability to "organize time" (44%) and the attribute of "perseverance" (31%) were not rated quite as highly (44% and 31% respectively).

Judge's Ratings
Innovation was defined by the project as: • Is entirely inventive; • Breaks rules and conventions OR uses common materials and/or ideas in new ways (e.g., develop a small-scale, more efficient method to sequester carbon) (see Table 1).
This rubric was used by judges in both rounds of the competition. In Round One, all 4 judges reviewed all 5 projects, while in Round Two, groups of 4 judges reviewed groups of 5-6 projects.
Round 1. All five projects submitted in the first round of the competition were deemed innovative by the judges. Two projects focused on carbon sequestration-one by generating biochar and the other by implementing "green roofs" on large vehicles such as buses. Two projects focused on different aspects of algal growth-one explored the impact of light type and level on algal growth to optimize biofuel generation while the other explored a miniature method for iron fertilization of the ocean to boost algal sequestration of carbon. The fifth project designed and tested systems to generate renewable energy using crank generators.
Round 2. Just over half of the 23 projects (15 projects) focused on some aspect of alternative energy generation or improving energy efficiency. Six of these described a method for capturing kinetic energy, one deployed a system for artificial photosynthesis, six focused on photovoltaics, and three captured and converted thermal energy. Five projects described ways to farm sustainably through local production of food in cities, nine explored ways to sequester carbon through tree planting, artificial photosynthesis, and the use of algae. Two described innovations that improved efficiency (of lithium batteries, PDA chargers), and one focused on a regulatory mechanism for supporting the use of hybrid vehicles.
The twelve judges were asked on a 1-3 scale, 1 ("not at all"), 2 ("somewhat") or 3 ("very much"), their rating of the following two questions: "To what extent do you think the submissions you judged were inventive or innovative?" and "To what extent do you think the submissions you judged have potential for future development?" The judges rated the overall level of innovation in the group they judged as 2 (6 responses) or 3 (6 responses), indicating that they thought the group was somewhat or very innovative. An overwhelming majority of the twelve judges rated the potential of the group of projects for future development as 2 (6 responses) or 3 (5 responses), indicating that they thought the group had somewhat or very much potential to be developed in the future. In addition, judges were asked to explain their rating. One of the judges singled out the "Decentralized Servers" project, and wrote: The distributed servers idea hit closest to the mark for me. Coming from a work environment that uses a lot of energy on computation, I could easily see an idea like this being implemented (though perhaps not in residential regions).
Judges were also asked the question: "To what extent did you think the submissions broke rules and conventions, or used common materials and/or ideas in new ways?" Five judges thought that projects did both, while seven thought that they had used common materials/ideas in new ways. In explaining their rating for the group they had judged, judges singled out specific examples of using common materials or ideas in new ways.
One of the judges singled out the "Subway Solutions" and the "Photo Electrics" projects in explaining their rating, writing: The subway piezoelectric project did a very good job at using two problems, lack of steady pressure and high power use in urban areas, to create a solution. I was intrigued by their idea and would be excited to see it in action in a big city. Overall, the power of the competition as a learning environment was amply demonstrated by the caliber of the projects the students produced, the depth of learning their posters, videos and discussion responses revealed, and the students' own ratings of the competition experience. Furthermore, the potential for this type of competition to generate fresh and innovative ideas for carbon mitigation was made evident by the judges' ratings and comments. The judging was rigorous, since twelve of the 16 judges were science graduate students, and four of the judges were project staff, all of whom hold advanced science degrees.
Nevertheless, all were highly impressed at the caliber of projects submitted by the student teams.

Project Ratings
While the judge's rating was based on their overall impression of the group of projects they judged, we also coded the level of innovation of each individual project using the same rubric. We found that projects were somewhat less innovative when rated individually (Table 9). It is possible that the favorable impression that two or three projects in a group colored the overall impression of the group in the minds of judges. phenomenon result in deep learning (Scardamalia, 2003;Barron & Darling-Hammond, 2009).
Problem-based learning provides such opportunities (Ravitz, 2009;Krajcik, Blumenfeld, Edelson, & Reiser, 2006;Wirkala & Kuhn, 2011;Drake & Long, 2009). Opportunities to reason about and communicate scientific explanations findings has been found to support deeper understanding of complex phenomena (e.g., Passmore & Stewart, 2002;McNeill & Krajcik, 2012). All these features are characteristic of mature scientific practice, and, indeed, science competitions have become popular arenas for engaging students in authentic science.
Such features all contributed to the power of this learning environment, as our findings show. The competition attracted and motivated teens to enter, and resulted in sustained engagement in deep science learning. Taken together, our results show that teams crossed disciplinary boundaries as they chose concepts from chemistry, engineering, mathematics or biology to address the mitigation challenge. Free choice of the specific STEM content they addressed included a wide diversity of topics ranging from biomimicry for artificial photosynthesis, to decarbonization of fossil fuels, to social media campaigns for reducing energy use, or improving transportation efficiency.
In addition to deepening their science learning, almost two thirds of the participants reported that they had learned more about the nature of science. In addition, they reported learning about many of the other dispositions that are necessary for success in science, for example, thinking outside the box, collaboration, and building on previous knowledge. In addition, teams engaged in science practices such as modeling, experimentation, error-analysis, representation and communication. In the design of the competition, we appropriated features of crowdsourcing that matched these features of effective learning environments.
Addressing the compelling societal problem of climate change was a powerful motivator for many students, while others cited the opportunity for sustained engagement and collaboration as a feature of the competition that they liked. The challenging nature of the competition charge, the invitation to be creative and innovative, and the open-ended requirements for acceptable solutions, were clearly appealing to students. Although these data resulted from a survey item that included a leading question, "I loved that the competition was…" the data appear to have been borne out by the judges' assessments of student projects, and through triangulation with other survey items.
Our design included a combination of intrinsic and extrinsic rewards, recommended by many as essential features of successful competitions (He et al., 2014;Brabham, 2008;NRC, 2007). Frey (2012), in discussing a history of prize incentives in science, observes that there is an inherent excitement in competing, which can spur creativity and innovative ideas (Frey, 2012). Evidence from the survey showed that both intrinsic and extrinsic motivation moved students to varying degrees to enter the competition. Prizes motivated some students, while others reported being motivated by the opportunity to develop standing with their peers and/or their teachers. The opportunity to communicate findings was another feature that many students appreciated, and data from video forum discussions showed that interactions with peers and judges deepened the learning of many. In addition, public recognition and the social media aspect of the competition in the video forum both enhanced the opportunity for students to realize the recognition from peers and teachers that they sought.
As Falk and colleagues have observed, researchers cite receiving feedback and networking with others, promoting communication skills and collaboration between group members, and creative assessment opportunities as some of the benefits of participating in traditional poster sessions (Aust & Kinnick, 1996;Johnson & Green, 2007;Stegemann & Sutton-Brady, 2009;Sisak, 1997, cited in Stroud & Falk, 2015. These same benefits accrued to participants in the Innovate competition.
From the perspective of designers, the structure of the competition presented us with a tension between wanting to support sustained engagement versus attracting many participants. The first Challenge supported intensive and sustained engagement that motivated participants to engage with complex science and real science practices, and elicited innovation, but only a handful of teams completed their submissions. On the other hand, the structure of the second Challenge supported higher participation, generated innovative ideas, and supported some crowdsource-like interactions, but team engagement in more authentic science practices was limited. This finding will necessitate a careful reexamination of the goals for the competition before deciding on future structures. Do we want to value broad participation or the in-depth doing of science? Is there a design that could optimize both, and, if so, what would that be?

Limitations
Crowdsourcing. Our original design decision was to maintain a crowdsourcing approach by making all the ideas submitted during the qualifying round open for any team to appropriate and develop for the final round. However, extensive discussions about what level of student motivation one might expect from students for an idea that was not their own, and feelings of ownership of the ideas they had submitted, led us to the decision to limit participation in the final round to the originators of qualifying ideas only. The practical result of this decision, however, was that teams worked individually on their projects, and had limited opportunities for the kinds of collaboration that Howe (2008) describes in commercial or competitive crowdsourcing settings.
Constraints of school culture. Where schools are facing intense pressures related to standards and accountability, interactions between student, teachers, and content are constrained, and the intellectual and professional space for building independent work into classroom activities is limited. For a truly open innovation and crowdsourced competition to be successfully supported in schools, teachers would need to pay greater attention to social skills and cultural competencies (Jenkins & Halverson, 2009;Kafai & Peppler, 2011). In addition, teachers would need to move students through the various stages of work-defining a design, problem-solving and iterating, developing arguments in support of explanations (Reiser, 2013;Kolodner et al., 2004)-and deal with students' failure and frustration (Hmelo-Silver et al., 2000). The kinds of support teachers need is a topic for further study.

Conclusion
Taken together, the data indicate that the Challenge provided an engaging and authentic scientific experience for participants. We conclude that regular competitions such as Innovate to Mitigate have powerful potential if added to the repertoire of learning environments that are possible in school ecosystems.