Problem-Based Learning Increases STEM Interest for High School Students and Instructors

The goal of Project STEMulate, a National Science Foundation ITEST study (DRL 1657625), was to develop, implement, and evaluate a program that fosters success in STEM for underserved and underrepresented high school students. The project was implemented at three sites of the Department of Education Upward Bound Program in Hawai ˋ i. Project STEMulate delivered teacher training on Problem-Based Learning curriculum to ensure students were motivated and empowered, and to support STEM-related postsecondary educational success of Hawaiian and Pacific Islander students. A critical design goal of the program was to introduce teaching and learning strategies and processes that were more relevant to underrepresented youth populations than those offered in typical high schools to provide opportunities and to increase participation in the STEM study and career trajectory, something all too often out of mind and scope of these students. This study reports on three years of mixed methods summer academy data on both student and teacher learning outcomes. Teacher dispositions, evidenced through data from interviews, observations, and multi-point surveys improved in a majority of the dimensions, including teaching inquiry-based approaches, integrating technology, and STEM career knowledge and awareness. Student motivation, Science self-efficacy, and STEM career interest, evidenced from similar data sources, increased as well. Finally, we discuss the larger implications of extending this work to impact similar populations elsewhere of isolated, under-resourced and under-exposed youth with these proven strategies.


Introduction
The primary goals of Project STEMulate, a three-year National Science Foundation funded STEM teaching and learning project were to develop, implement, and evaluate a program that fosters STEM learning and career opportunities for underserved and underrepresented high school students. The project was implemented at three island sites of the Department of Education Upward Bound Program in Hawaiˋi. Project STEMulate delivers teacher training on Problem-Based Learning curriculum to ensure students are motivated and empowered, and to support STEM-related postsecondary educational success of Hawaiian and Pacific Islander students.

Need for Increasing Interest in STEM
With the ever-growing concern for the future of the United States economy and workforce, the attention of policy makers, educators and researchers is increasingly focused on enhancing STEM education in the United States (NSB, 2010). Despite the extended focus on stimulating students, particularly underrepresented minorities (Allen-Ramdial, & Campbell, 2014), only 9.6% of minority male and 3.0% minority females pursue engineering careers (Malcom-Piqueux, & Malcom, 2013). Worldwide, U.S. students are falling behind, as per Programme for International Student Assessment (PISA), a global benchmark for measuring STEM proficiency, U.S. 8 th graders ranked 36 th in math and 19 th in science, out of 79 in 2018 (OECD, 2019). Nationally, Hawai`i students tested among the lowest in the nation in math and science (National Center for Education Statistics, 2016). Despite Hawai`i students' modest gains in math on the 2015 NAEP (National Center for Education Statistics, 2016), high school students on Maui score lower than the state averages in math and science (Hawai`i Department of Education, 2015). Moreover, low-income students, both nationally and locally, score lower on achievement tests (Hawai`i Department of Education, 2011;Plucker, Burroughs, & Song, 2010) with the achievement gap widening between low-income and high-income students.
Research on the pipeline to STEM fields and careers indicates that early exposure to inquiry, reasoning, and problem-solving skills in STEM stimulates student learning and interest in pursuing an eventual STEM-related degree (Dejarnette, 2012). In search of an explanation for what ignites and retains students' interest in STEM, many programs have been envisioned and developed from K-12 to college and at the graduate level, and several have explored strategies for attracting students to STEM. Several programs (such as Project STEMulate) have been implemented through funding from federal agencies or corporate entities. Goals have been varied, covering a wide range of purposes such as assessing how to retain college students in their STEM field, how to motivate and encourage middle or high school students to enrol in STEM programs, or how to provide K-12 teachers with STEM education and professional development. Meanwhile, other researches (Mathers, Goktogen, Rankin, & Anderson, 2012) have emphasized the hands-on experiences that will engage and inspire students toward STEM careers. Although some researchers focus on an earlier start on the educational pathways toward STEM fields and have identified elementary school students as the best targets mainly because they have more time to build competence in STEM (Alumbaugh, 2015;Cantu, 2011;Isabelle & Valle, 2016), others have concentrated on middle and high school students (Tai, Liu, Maltese & Fan, 2006). High school is a critical time for providing positive experiences that engage students in STEM activities since it is the time when they are beginning to consider possible career pathways (Hansen, 2011).

Problem-Based Learning (PBL)
To spark student interest in STEM and to prepare them for the STEM workforce, Project STEMulate provided students with challenging PBL hands-on activities and guidance to solve real-world problems. Research shows that PBL can be used with students of any age and skill level (Lockhart & Le Doux, 2005). Significantly, results of several high school PBL studies suggest that PBL may be as or more effective than traditional instructional approaches (Mergendoller, Maxwell, & Bellisimo, 2006;Savery, 2006), especially with low-income students (Cuevas, Lee, Hart, & Deaktor, 2005;Gallagher & Gallagher, 2013). Meta-analyses findings point to the fact that PBL exceeds traditional learning methods for teaching critical thinking, communication, collaboration, and applying knowledge to real world situations (Darling-Hammond et al., 2009;Strobel, & van Barnevel, 2009;Walker & Leary, 2009). STEM-focused PBL summer programs have also been shown to increase STEM career aspirations (Lam, 2005;Zhe et al., 2010). The results of a five-year study (Lam et al., 2005) showed significant increases in GPA and STEM selfefficacy, decreases in anxiety towards math and sciences, and the high degree of student enrolment in STEM degree programs following high school graduation. Project STEMulate research is much needed (a) because PBL curriculum is only offered at few schools (Atkinson & Mayo, 2014), (b) much of the research on PBL at high school level lacks structure and identification of what works for whom (Atkinson & Mayo, 2014;Ravitz, 2009), and (c) there is a lack of appropriate teacher training (Asghar et al., 2012). PBL teachers focus on creating an active, integrated, selfdirected, and collaborative student-centred environment (Ertmer & Simons, 2006) and teachers who engage with the PBL approach enhance their pedagogical content knowledge (Walker & Leary, 2009).

Theoretical Framework
The social cognitive career theory (SCCT) as articulated by Lent, Brown, & Hackett (1994), and driven from Bandura (1986) guided this study. SCCT suggests that selfefficacy and interest play unique roles in career choice (Armstrong & Vogel, 2009;Betz & Borgen, 2010;Byars-Winston, Estrada, Howard, Davis, & Zalapa, 2010;Donnay & Borgen, 1999;Lent et al, 2010;Silvia, 2003;Tracey, 2010;Tracey & Hopkins, 2001), and individuals develop interest in activities in which they believe they can perform well. Furthermore, previous research has shown self-efficacy to be positively related to student academic performance, that self-efficacy in science impacts student selection of science-related activities (Britner & Pajares 2006;Parker et al. 2014;Richardson et al. 2012), and that self-efficacy and interest in STEM are strongly related (LaForce, Noble, & Blackwell, 2017;Maltese and Tai, 2011). As a result, individuals' personal, academic, and career goals are consistent with their interest, self-efficacy, and the outcomes they expect to achieve . In other words, the development of interests is primarily on the basis of beliefs about self-efficacy and expected outcomes. If people believe they can do something well, it encourages further participation in that activity. Thus, SCCT hypothesizes that career interests and personal goals involve a process that includes performance, self-efficacy, and outcome expectations.

Study Design
The two primary research questions guiding this study were: • How did the PBL approach impact teachers to move from teacher-centred to student-centred teaching during Project STEMulate? • Was STEM PBL a strategy as modeled in Project STEMulate an effective way to sustain and enhance students' interest in STEM careers?

Participants
The program was implemented for three consecutive years with a different cohort of teachers and students each year. Professional development occurred in the spring of each year followed by the summer academy for the youth. Each year a group of 25 teachers representing the Hawaiian Islands of Maui, Oahu, and Hawaiˋi participated in the professional development, which was conducted primarily online with an opening and closing weekend in-person meeting. Through an application process, nine teachers (three for each designated Project STEMulate site) were then selected to lead the summer academies (the other 16 teachers supplemented their skillsets with PBL methods and strategies, but not as part of the STEMulate program). A site instructional team consisted of a science, a mathematics, and an English or writing teacher.
Students were selected from a pool of Upward Bound (UB) program participants. Upward Bound, a U. S. Department of Education program, provides fundamental support to participants in their preparation for college entrance, and performance, as well as opportunities to succeed in pursuit of higher education. The program serves high school students from low-income families and those from families in which neither parent holds a bachelor's degree. While the typical UB summer programs offer a variety of academic activities, STEM-related courses are often weak; Project STEMulate sought to fill this niche.

Teacher Professional Development (PD)
Professional development was delivered online over six weeks each year with a new assignment provided, countered and completed each week. Hawaiˋi, like other remote locations (e.g. Alaska) has a long history using distance learning methods and technologies and educators are accustomed to, and even expect these modes, so there was little effort for uptake or acclimatization one might expect in more tightly connected urban communities. The PD course began and ended with an in-person workshop in Maui. Weekly content modules focused entirely on developing and cultivating strategies and techniques of problem-based learning, where rather than be told how and what to learn from textbooks or traditional resources, students explore a subject by working in groups to solve an open-ended problem.
In STEMulate, the problems were STEM-focused with an emphasis on actual environment challenges facing Hawaiˋi, including erosion, energy, clean water, conflicts with large telescope installations and ocean acidification. These topics, while of interest to scientists everywhere, were of particular interest and relevance to native Hawaiian students. When possible, teachers were able to pilot units or components of PBL instruction in their (non-STEMulate) spring semester science courses, which helped prepare them for the summer academies. Teachers also learned how to help students identify relevant problems, brainstorm conjectures, research approaches, and potential solutions. For many, this was a departure from business-as-usual, as one teacher commented, "At first I thought it would be very difficult, I mean we are required to cover this kind of material from the textbook and I didn't think my students would be able to come up with solutions, it was all new to me."

Student Knowledge Acquisition
The summer academy provided students at each site with the same material and strategies. Students were engaged in daily classes and sessions that helped them identify and research a problem, explore and develop solutions, collect data, and present findings to the entire Institute body in a formal scientific symposium. In some cases students built physical models of their proposed solutions to demonstrate efficacy. Students collaborated, typically in small groups (four or five-person), learned how to collect, test, and analyze data; how to develop a compelling argument through communication skills, and how to compose and present results in a public forum. In addition to the increased science and literacy skills (geological, atmospheric, oceanic, biological, data collection and validation, hypothesis testing, statistics) students also learned useful soft skills of communication, argumentation, public speaking, and collective problem-solving, all at the forefront of today's and tomorrow's STEM field demands.

Sites
The program was developed and managed by a team at University of Hawaiˋi, Maui College. Three concurrent summer residential Upward Bound (UB) programs for participating youth took place each year at affiliated university campuses in Maui (Maui College), Oahu (Windward Community College) and Hawaiˋi (Hilo Community College), all as part of the regular UB summer academy. Project STEMulate provided STEM learning experiences as part of student programs.
Sites, all being Hawaiian Islands, naturally shared common geological, environmental, and socioeconomic factors. However, each site's choice of problem topic was related to the island's community interests. For example, with Maui's concern for clean energy and as a leading voice for Hawaiˋi Clean Energy Initiative (HCEI) to become carbon neutral by 2045, students in that group researched energy alternatives. On Hilo, the Thirty Meter Telescope, to be built on Mauna Kea, a source of controversy and friction between the scientific community and those who place high cultural and spiritual value on the mountain and her role in their culture, provided students with a relevant problem.

Data Collection
Common to other research projects, and particularly those supported by NSF, Project STEMulate included separate educational research and program evaluation components. The research team focused on student outcomes, engaged in mixed methods of data collection including pre and post surveys, focus groups, inventory analysis, and site observations. The evaluation focused primarily on project fidelity and teacher outcomes. A total of 287 students (148 in project STEMulate and 139 in comparison group) participated in this program over the three years. Nine teachers completed surveys and interviews each year for a total of 27 over the three years. Researchers conducted site visits each spring and during the summers in years 1 & 2; in year 3, as a result of COVID-19 pandemic, all data collection and visits were conducted online.

Measures and Instruments
Student instruments included: 1) Science Self-Efficacy; 2) Science Motivation, and 3) STEM Career Interest. Instruments used with teachers included: 1) Science Self-Efficacy (STEBI), pre-post prompted interviews with teachers, and site observations. Selected measures were based on earlier reviews of the effects and impacts of these measures.

The STEBI-B
This instrument measured teachers' science teaching self-efficacy (STSE). The STEBI-B is used most frequently and has demonstrated reliability and construct validity (Riggs and Enochs, 1990), has 25 Likert scale items on two subscales: Personal Science Teaching Efficacy Belief (PSTE) and Science Teaching Outcome Expectancy (STOE), where the PSTE measures the degree that teachers believe they can impact student achievement in science, and the STOE exhibits teachers' beliefs on the factors affecting student science achievement.

STEBI
Teacher STEBI data showed improvements pre-post each year as well as increased growth from one year to the next, however none met the threshold for significance. The STEBI was administered at three points: Time 1 was prior to the PBL course; Time 2 was at the conclusion of the PD course; Time 3 was at the conclusion of the summer program. For an analysis of the 25 items, and because the instrument utilizes intentional redundancy throughout, we clustered the items into two categories: those items related to how the program improved science instruction, and how the program improved overall teaching. For example, an item about improving science instruction was I understand science concepts well enough to be effective in teaching elementary science. An item about improving overall instruction was Even when I try very hard, I don't teach science as well as I do most subjects. These two categories proved helpful in communicating findings to stakeholders and teachers in particular, rather than address the gains on a per-item basis.
The mean of the gains increased each year, suggesting the program design was robust and implementation became more efficient with each annual cycle. The first year of the program saw gains at very small increments (0.04 point) and did not include a midpoint reading, the second year at 0.19, and the third and final year at 0.29. The greatest gains each year were between Time 2 and Time 3 suggesting teachers learned most through application of their skills with students.  Gains for teachers improving science instruction advanced 53% from year 2 to year 3 while gains for teachers improving their overall teaching increased 22%. For midpoint to post readings, gains within each year were higher for item clusters on teachers improving science instruction, with the differences between year 2 and 3 showing a 43% increase but a 24% drop for gains in improving overall teaching. For the pre-to-post readings, every cluster showed an increase each year, with a greater gain each year than the previous year, pointing to program improvement over time. The pre-post gains of Cohort 3 in how STEMulate improved teachers' science instruction showed a (technically) 3900% increase over Cohort 1 (Cohort 1 showed a minimal 0.01 pre-post gain) and a 317% increase in improvement of overall teaching. Though the data might be skewed to reflect the commonly rocky first implementation year, the key takeaway is the improvement over time each year in all item clusters.

Science Self-Efficacy (SSE), and Science Motivation (SM)
The overall scale averages derived from a scale defining students' self-confidence in their science abilities and skills, and their motivation toward learning science. Science efficacy items were partially adapted from the STEM Career Interest Survey, Science Section ( The level of science self-efficacy and motivation was calculated each year at the beginning and end of the summer academy. SSE data showed significant improvements from pre-to-post each year for both STEMulate and the comparison group. As Figure 4 displays, STEMulate data exhibited a higher SSE score gain from pre-to-post survey (which is a testimony to the program design). This was consistent with the teachers' STEBI data demonstrating greatest teacher learning occurred through application of teachers' skills in a PBL environment where their role was emphasized as a facilitator and activator for learning.  For further examination, a paired-samples t-test was also conducted to evaluate the impact of the intervention on students' science self-efficacy (SSE) and science motivation (SM) scores. In Cohort 1, there was a statistically significant increase in SSE score from pre (M=2.88, SD= .50) to post survey (M=3.57, SD= .71), t (63) = 24.48, p<.001 (2-tailed). The mean increase was .69 with a 95% confidence interval ranging from .64 to .75. The eta squared statistic (.91) indicated a large effect size (see Table 2, Appendix A). The same statistically significant increase in SSE score from pre-to-post survey with large effect sizes were also observed for SSE cohort 2 and cohort 3.
The pattern for a statistically significant increase in science motivation score from pre-to-post survey, and for Cohort 1 and Cohort 3, was consistent with the SSE pattern. However, the data for Cohort 2 science motivation, even with a nominal gain (0.18) was barely significant (p= 0.044).

STEM Career Interest (SCI)
Career Interest was assessed using the 12-item STEM Career Interest Questionnaire (Cronbach's alpha =.78-.94) adopted from Tyler-Wood, Knezek, and Christensen (2010). The SCI has three subscales that measure student perception of a supportive environment for pursuing a career in science (Interest), their desire in pursuing educational opportunities that would lead to a career in science (Intent), and their perceived importance of a science career overall (Importance). A number of parallel analyses were conducted to determine the extent to which the SCI documents the effects of project STEMulate on students' career attitudes. First, the internal consistency of the SCI for the pre and post survey was calculated for each cohort and the range of Cronbach's alpha was very high at .85 to .96.
As Table 3 shows, the level of STEM career interest among high school students was low at the beginning of each summer academy and increased by the end of the summer. Although the mean increased for both groups after the program, the pairedsamples t-test was statistically significant for the career interest score of the STEMulate students. These results align with previous research (Christensen & Knezek, 2017) stating that engagement in hands-on PBL activities will increase interest in a STEM career.

Pre-Mean Post-Mean
Cohort 1 2.96 3.58 Cohort 2 2.73 3.48 Cohort 3 3.63 3.88 Table 3: Pre-and post-Means of STEM Career Interest for the STEMulate students Furthermore, Table 4 (Appendix B) demonstrated gains in all parts of the SCI for STEMulate students with most being statistically significant at p<.001 except for cohort 3 where p<.05 was applicable to totals, parts 1 and 2, and part 3 was not significant. A closer look at the comparison data in the same Table reveals a smaller gain for that group at a higher significant level.
To identify the impact of science self-efficacy and motivation on the STEM career interest, regression analyses were conducted that allowed to confidently determine the influence of these variables and to establish which variable mattered most. The initial regression analyses results indicated that students' SSE and SM scores significantly predicted their STEM career interest (SCI) (p < .001) for all three cohorts. With a multiple regression analysis, (a) the overall regression model for cohort 1 was significant, F (2, 55) = 110.29, p <.001, R2 = .786, and the overall saturated model explained 78.6% of the variance in students' STEM career interest; (b) the overall regression model for cohort 2 was significant, F (2, 60) = 148.98, p <.001, R2 = .844, and the overall saturated model explained 84.4% of the variance in students' STEM career interest, and (c) the overall regression model for cohort 3 was significant, F (2, 27) = 79.98, p <.001, R2 = .856, and the overall saturated model explained 85.6% of the variance in students' STEM career interest.

Discussion
Findings related to research questions suggested positive outcomes, supporting the primary goal of developing a program that fosters success in STEM for underserved and underrepresented high-school students through Problem-Based-Learning instructional strategies.
Research Question: How did the PBL approach impact teachers to move from teacher-centred to student-centred teaching during Project STEMulate?
Positive gains from data analyses suggest as teachers shifted toward a studentcentered STEM PBL approach, their science teaching, and overall teaching practices improved. These gains increased each year, pointing to both traction of the processes over time, but also a more efficient and effective program delivery as the program evolved.
Research Question: Is the use of STEM PBL strategy as used in Project STEMulate an effective way to sustain and enhance students' learning?
Positive gains in student scores in science self-efficacy and motivation suggested the PBL strategy was successful and effective in sustaining and enhancing student learning. Prior research supported PBL learning as impacting student's recognition and selection of STEM careers (Christensen & Knezek, 2017;LaForce, Noble, & Blackwell, 2017). This study, framed by social cognitive career theory (SCCT), advocated for self-efficacy to play unique role in career choices. Results indicated an increase in science self-efficacy and science motivation scores, a validation that with hands-on exploration and problem-solving and search for solutions to a real-world problem, students developed interest in activities they performed and believed they could perform them well. This study aligned with previous research that self-efficacy in science impacts student selection of science-related activities (Britner & Pajares 2006;Parker et al. 2014;Richardson et al. 2012), and showed strong association with their interest in STEM (LaForce, Noble, & Blackwell, 2017;Maltese and Tai, 2011). Also, it illustrated that students' career goals were consistent with their science selfefficacy, and motivation . These results implied that participation in Project STEMulate positively affected students' career interests and personal goals, consistent with the SCCT. Overall, though gains were observed in STEM career interest for all participating high school students, those participating in the STEMulate program showed a consistent significant gain over the three years. These results support previous research (Christensen & Knezek, 2017) stating that engagement in hands-on PBL activities will increase interest in a STEM career.
The move from a teacher-centred approach using established curriculum, textbooks, and problem sets, common to high school science classes in Hawaiˋi (and elsewhere) to student-based learning where participants explored carefully selected problems and developed solutions through scientific processes resulted in three significant outcomes: 1) Increase in student interest in STEM, 2) Introduction of STEM career opportunities, and 3) Improved teacher knowledge and skills in STEM instruction.
Even as several studies have identified teacher reluctance to embrace PBL for a variety of reasons, including curriculum standards (Liu et al, 2012;Pagander L, Read, 2014;Subramanian, 2014), lack of training (Asghar et al., 2012), and discomfort with the unstructured PBL method (Asghar, 2012;Nowak, 2017), our study documents that a student-centred environment benefits both teachers and students by engaging teachers with the PBL approach that enhances their pedagogical content knowledge (Walker & Leary, 2009), and by creating an active, integrated, self-directed, and collaborative learning (Ertmer & Simons, 2006) for the students.

Conclusion
Through focused and rigorous teacher professional development and a carefully designed summer program that tapped into student interest and motivation, the departure from traditional classroom learning methods exemplified in Project STEMulate helped meet the program goal of increasing participation in STEM learning and career trajectory, helping to bridge the gap for underrepresented students. The project has shown traction and efficacy as a replicable model, and would very likely succeed in many other contexts that share the socioeconomic and cultural factors that have proven barriers for access to STEM learning. This work will add to the knowledge base and calls for additional studies that will validate and refine research on improving STEM learning opportunities for underserved populations and communities everywhere.