The need for more experts and innovators in the fields of science, technology, engineering and mathematics (STEM) in the U.S. has become a paramount issue for the success of our nation (Bray, 2010; Couto, Mani, Lewin, & Peeters, 2007; National Science Board, 2010). While this need is increasing dramatically, the number of students pursuing and completing degrees in these fields is decreasing (Kendall, Pollack, Schwols, & Snyder, 2007; National Academies of Science, 2007; National Science Board, 2010).
Three factors contributing to this problem may be (a) students’ poor preparation and lack of interest and success with mathematics and science in high school (Arizona Department of Education, 2009; Mullis, Martin, & Foy, 2008; Organization for Economic Cooperation and Development, 2007); (b) teachers underprepared to engage students in the application of mathematics and science concepts and new technologies to the solution of problems that both mirror those faced by the workforce and demonstrate a usefulness for that knowledge (Boyd, Grossman, Lankford, Loeb, & Wyckoff, 2009; Hill, Rowan, & Ball, 2005; National Mathematics Panel, 2007; Stigler & Hiebert, 1999); and (c) student, teacher, and family lack of knowledge about new STEM careers and the academic preparation in high school and college necessary for those careers (Arrington, 2000; Hughes, Bailey, & Karp, 2002).
Putting Knowledge to Work: Goals and Objectives
The overarching goal of P3, a 3-year project funded by the National Science Foundation (ITEST) that began in 2008, is to design, implement, and evaluate a scientific village strategy to enhance the science, technology, engineering, mathematics, and business (STEM-B) pipeline from high school to college by increasing the number of students who enroll in and pursue college majors in preparation for STEM-B careers. The scientific village strategy is designed to
- Increase students’ interest in and success with the study of mathematics and science in high school;
- Integrate workplace technologies, communication, collaboration skills, and critical thinking and risk-taking behaviors into the P3 learning environment;
- Increase student awareness of STEM-B careers, university preparatory programs for these careers, and their own talents as related to these fields;
- Update secondary school teachers in content (concepts and skills) in their own and related fields, technology, pedagogy, and STEM-B career opportunities;
- Increase parents’ knowledge of STEM-B careers, the preparation needed for them, and their children’s talents as related to these careers.
The core element of the P3 strategy is engagement of students and teachers in scientific villages (communities) whose members work collaboratively on long-term projects/problems that are of high interest and require application of STEM-B concepts and skills. Village members include high school students (class of 2011), secondary-certified teachers of STEM-B, undergraduate STEM-B majors who apply for and are trained to assist village leaders and to mentor participants, and scientists from Arizona State University (ASU), other colleges, and industry, who design and lead the villages.
P3 just began the third year of its 3-year funding at the time of this writing, but progress toward meeting two goals of the project are encouraging and compelling and provide the focus for this paper: Does the P3 scientific village strategy increase students’ interest in and success with the study of mathematics and science in high school, and does it change teachers’ instructional strategies and expectations for student performance? The rationale for the Pipeline project follows, along with a description of the instructional approach and program components, requirements for participation in P3, and the evaluation study, including participants’ (intervention and control) demographics, assessment instruments, data collection and analyses methods, results, and conclusions.
Putting Knowledge to Work: Project Rationale
Rationale for the P3 approach comes from research on (a) integrated projects for enhancing the acquisition and application of mathematics and science concepts and skills and various technology tools (Corcoran & Silander, 2009; Darling-Hammond, et al., 2008; Duschl, 2008; Markham, Larmer, & Ravitz, 2003); (b) motivation to learn (Allen, Bonous-Hammarth, & Suh, 2005; Allen, Bonous-Mammarth, Yang, González, & DuCros, 2004; Sedlacek, 2004; Sedlacek & Sheu, 2004); and (c) skill requirements for the 21st-century workforce (Meeder, 2008; Secretary’s Commission on Achieving Necessary Skills, U.S. Department of Labor, 1991).
Poor performance in mathematics and science and the lack of interest and persistence in STEM subjects in high school and college may occur as a result of a siloized approach to education and minimal time for true exploration and learning. In that approach, academic subjects are taught separately with few opportunities for students to apply what they have learned to solving interesting and compelling problems (Stone, Alfed, & Pearson, 2008). Students are not presented with situations in which to put their knowledge to work (Geier et al., 2008; Jerald, 2009; Kazis, 2005; Smith, Sheppard, Johnson, & Johnson, 2005).
In most high schools, mathematics has been taught as a collection of separate topics (e.g., algebra, geometry, statistics), often in 45- to 50-minute time blocks, with the primary goal of having students master computational algorithms (Stigler & Hiebert, 1999; Stone et al., 2008). More elaborate problems are generally avoided, with claims that students cannot solve them because they do not understand the context domains (e.g., economics, physics) or there is insufficient class time for students to wrestle with challenging problems.
Despite the fact that project based learning (PBL) is gaining momentum, it rarely involves problems typical of those solved by workers in STEM-B fields (Bronson, 2007; Clark & Ernst, 2007; Meeder, 2008). This situation has been the case since the seminal paper from the U.S. Department of Labor (Secretary’s Commission on Achieving Necessary Skills, U.S. Department of Labor, 1991) that addressed the needs of the workplace and stressed the importance of better preparing students for work on project types of problems that require application of problem solving, collaboration, and communication skills.
Core to the P3 scientific village strategy are the integrated projects, developed by scientist leaders not only to engage villagers in STEM-B explorations, but also to promote collaboration, communication, multiple uses of technology, and the joy of problem solving. Based on literature on motivation cited later, projects are designed to
- Encourage investigation, application, and the development of more robust understanding of key concepts and skills typically developed in high school mathematics and the sciences;
- Require the use of workplace technology tools including databases, simulations, numerical techniques, search, sorting, and graphing strategies;
- Examine problems of importance to society (e.g., alternative energy);
- Require team collaboration to handle multidimensional projects;
- Develop habits of mind such that students appreciate challenging problems, persevere to reach solutions, monitor their thinking and actions, and improve their performance by seeking more elegant solutions;
- Promote communication through technical writing documentations of their work and oral and multimedia presentations.
The theory for getting high school students interested in STEM-B disciplines and careers evolves from several areas of research:
Teachers. Students are influenced by their teachers’ expertise, innovative teaching styles, enthusiasm, and high expectations for performance (Boyd et al., 2009; Haladyna, Olsen, & Shaughnessy, 1982; Jones, 2005).
Experts. Student access to a network of experts provides exposure to powerful transforming experiences that enhance understanding and enthusiasm for STEM-B careers (Allen et al., 2005; Sedlacek & Sheu, 2004).
Peers. Peer influence is a major determinant of college choice and academic preparation (Griffin, Allen, Kimura-Walsh, & Yamamura, 2007). Team approaches, in which students collaborate to study core subjects, are particularly effective for preparing students for college (Fernandez-Santander, 2008; Seymour, 1995) and the workplace (Kalman, 2007; Meeder, 2008). The goal of collaboration is to build a community that is supportive of academic achievement (Sedlacek & Sheu, 2004).
Parents. The major influence on student preparation for and attendance at college is the family (Griffin et al., 2007). Parents need to be well educated about career opportunities for their children and the preparation necessary for them.
Interests. Research shows that students’ choices of careers can be influenced by capitalizing on their STEM-B interests, such as technology (Sedlacek, 2004). Students today are immersed in technology. They use cell phones and computers, create podcasts, play video games, write blogs, download and upload to their social communication networks, and they know how to search for information online. The greatest use of the internet is by youth in the age range of 12 to 18 (Gewertz, 2007). Students exhibit excellent learning behaviors when using these various technologies. They take risks and experiment with new ideas and they persevere (Gewertz, 2007). P3 is designed to stimulate those same learning behaviors while students are working with new ideas in mathematics, science, and other content areas.
Project-Based Learning. With its “hands-on approach,” relevance, and ability to accommodate various learning styles, talents, and the diversity of social transactions, PBL has been shown to correlate positively with student motivation (ChanLin, 2008; Helle, Tynjala, Olkinuora, & Lonka, 2007).
Time-on-Task. Just as time-on-task is an important factor in learning, sustained time with the same project/problem has been shown to increase achievement and interest (Aronson, Zimmerman, & Carlos, 1998).
Critical Workplace Skills
The economic success of the United States depends on students who are well educated to think critically, solve problems, demonstrate expertise with computers and other technology, communicate well, and assume greater responsibility for their futures (Partnership for 21st Century Skills, 2007; Public Works, 2006; State Educational Technology Directors Association, 2008; Secretary’s Commission on Achieving Necessary Skills, U.S. Department of Labor, 1991). Students also need to improve their abilities to apply mathematics and technology to the solution of problems, locate information, read for information, observe, write, listen, judge, and make informed decisions. These are among the skills nurtured in the P3 project components and activities.
Prime the Pipeline: Approach, Components, and Activities
Scientific Instructional Approach
The approach used during project engagement reverses the lecture-and-then-apply method of instruction. Rather, villagers are urged to bring to bear what they already know to begin the solution process and get information or direction at point of need. Village leaders design projects that prompt learning of new concepts and ways to use technology by causing villagers to bump into “obstacles.” For example, these obstacles may include epistemological conflicts in which results of actions contradict expectations or prior knowledge, and villagers must wrestle with ideas to resolve the conflicts; insufficient knowledge to continue that requires a search for relevant information; and ambiguities that require making assumptions and considering the use of alternative heuristics, data, or data analyses procedures. Rarely do our village leaders lecture for more than 15 minutes and not usually at the start of an exploration.
Educating Students and Teachers Together
Typical teacher preparation and in-service courses are taught with a single content focus, that is, mathematics teachers receive training in mathematics content, pedagogy, and assessment, and science teachers are trained in their specialty. Rarely are teachers of two or more subjects updated together in their respective and sister fields. Even rarer are programs in which teachers and students are educated side by side.
This approach is predicated on the following beliefs:
- Mathematics teachers need to be well trained in other content areas (e.g., the sciences, economics) in order to help students recognize the usefulness of mathematics for modeling and analyzing problems in those areas (National Governors Association for Best Practices and the Council of Chief State School Officers, 2010).
- Science teachers need to understand fully the value of mathematics as a tool to understand the sciences (American Association for the Advancement of Science, 2009), how formulas model phenomena, and how, for example, a change in one parameter of a formula affects a change in the model and vice versa.
- Both mathematics and science teachers need to be updated in the use of workplace technologies.
- Both mathematics and science teachers need to experience PBL themselves in order to better understand the elements of the problem solving/scientific inquiry approach.
- Both mathematics and science teachers profit from collaborating with students to solve complex challenging problems in terms of gaining greater insight into students’ learning and problem-solving talents.
- Students’ talents with technology (they have grown up with it) and their risk-taking adventurous spirit while exploring new technologies will place them in the mode of assisting teachers, thereby providing greater insight into their own abilities.
Because each population has separate talents, they can learn new ideas together, provide support for one another in terms of their area(s) of expertise, and come to appreciate the knowledge each offers.
P3 has three major components: academic year scientific villages, summer institutes, and summer connections courses for teachers. All village meetings take place in the computer and research laboratories on the Polytechnic campus of Arizona State University located in Mesa, Arizona. The end-of-semester and end-of-summer Showcase Open Houses provide venues for villagers to present their completed projects to their peers, families, and the community. The villages and the connections courses are supported by Blackboard, which enables communication among villagers during project explorations and provides village-specific information. The P3 website provides information about village topics, schedules of P3 events, student and teacher application forms, and links to information about STEM-B careers and college preparation for them.
P3 began in spring 2009. Since that time, four villages have been offered every semester and in the summer. Because lab facilities limit the number of participants to 24 and all villagers interested in some of the topics could not be accommodated in them, those villages were offered in a subsequent summer or academic year. Villagers continuing with P3 have first choice of village. Each village has a minimum of one scientist and one mentor.
Academic Year Villages
During the academic year, for 9 weeks each semester, villagers meet once a week on Tuesdays after school from 3:45 to 6 p.m. (20¼ contact hours per semester) to participate in scientific villages. Week 9 is the Showcase Open House for families and the community.
During the summer session, which begins shortly after the close of the previous academic year, villagers meet daily from 8 a.m. to 12 p.m. for 2 weeks (40 contact hours per summer). Each day begins with an assembly at 8 a.m. for announcements and village debriefings of work completed to date. Debriefings are presented by villagers—students and teachers alternate—to fellow participants and scientists. Day 10 is the Showcase Open House.
Connections Courses for Teachers
Held daily in tandem with the Summer Institute and led by project staff with guest presentations by experts in a variety of fields, teachers meet from noon to 2 p.m., Monday through Thursday, to gain greater insight into the big ideas in their content areas of expertise, in the sister subjects, and with various types of technology. They explore techniques for assessing students’ depths of understanding through problem solving talk-alouds, flexible/clinical interviews, and observation protocols. They explore better ways to counsel/guide their students toward STEM careers. They develop proposals to fund various activities, supplies, and experts to assist with implementation of integrated content, and project learning in their classrooms.
They also connect/network with each other to examine common difficulties and to learn about instructional programs offered by their peers. For example, two of our teachers described the math instruction they provide at a virtual online high school. Among the other special topics were Algebraic Reasoning: Development and Assessment, Assessing Teacher Success with the Integration of Mathematics and Science, Career Counseling and the Role of the Teacher, The Nature of College Preparation for STEM Careers, and Using White Boards to Enhance Group Collaboration and Documentation of Project Work.
P3 Website and Blackboard
The project website (http://primevillages.asu.edu) is designed to enhance communication among project staff, mentors, teachers, and students. All villagers have ASU identification and email addresses that enable them to gain access to all project software, whether at home, school, or working at one of the open-access computing labs on ASU campuses. Each village has a Blackboard site where village leaders post homework assignments, resources for participants, and village updates. Villagers can chat with their own working groups on the Blackboard site.
Village Leader Orientation
Prior to the beginning of each semester and summer program, P3 staff meet with village leaders and mentors to orient them to the goals of the program, review their project goals, and distribute information about attendance and assessment requirements. Village topics are generally the pet projects of the village leaders and are most often tied to their research or teaching activities at their respective institutions.
Scientific Villages: Spring 2009 through Spring 2010
To gain greater insight into the nature of the Scientific Villages and the numbers of student and teacher villagers, see Table 1. The links in Table 1 provide connections to village specific information, including descriptions of project activities, assessments, photos of villagers at work, and in some cases, videos showing the villages in action. Some videos were created by participants in the film villages. Brief descriptions of the villages are found in Appendix A.
P3 Scientific Villages Spring 2009 through Spring 2010
Number of Participants
Link to Additional Information[b]
|1. Spring 2009|
|2. Summer 2009|
|3. Fall 2009|
|[a] See Appendix A for descriptions of sessions.|
[b] See Appendix B for website URLs.
For the first P3 session in spring 2009, students and teachers were assigned to villages by the project staff. Subsequently, villagers indicated their village preference using a rank-ordering procedure. Efforts were made to honor their first or second choice. Initially, attempts were made to separate students and teachers from the same school. After one semester, students indicated that participating with their teachers was “not a problem” and that they thought it was great that they could “help their teachers with the technology!” Teachers agreed.
Participants and Their Communities
To participate in P3, students must have passing grades in Algebra I, membership in the graduating class of 2011, and interest in technology. Teachers are required to be state certified to teach secondary level (grades 7-12) STEM-B courses. Primary recruitment mechanisms include direct presentations to students and teachers, distribution of informational flyers to schools, teachers, and students, and word of mouth.
P3 students hail primarily from six school districts in Arizona: Chandler, Gilbert, Higley, Mesa, Payson, and Superior. With the exception of Payson and Superior, all high schools are within a 15-mile radius of the ASU Polytechnic campus, the location for all village work. Payson High School is 88 miles from the Polytechnic campus; Superior High School is approximately 50 miles away. The communities in which the schools are located vary by size, ethnic diversity, and median household income. Demographic data, including population, median income, and racial and ethnic diversity for each of our primary school districts are presented in Table 2. Note that Higley High School is located in the city of Gilbert. Teachers come from 15 different Arizona school districts.
Demographic Data for P3 Primary School Districts (2009)
|Median income (state average: $69,205)|
|Note: Ethnic diversity percentages and median household incomes are rounded up.|
Data reported in this study are based on 17 months of information collected from P3 project participants during this 3-year project. The ongoing evaluation plan for P3 includes both quantitative and qualitative measures to assess the degree of increase in students’ interest in and success with the study of science and mathematics in high school, and changes in teachers’ instructional practices and expectations for student accomplishments.
A quasi-experimental design was used to analyze student data and included multivariate analyses on several dependent measures over time, including high school GPA, number of STEM-B courses completed, GPA in STEM-B courses, and number of advanced courses completed (i.e., honors, advanced placement, or dual enrollment) . Students from the class of 2011 who participated in the P3 project compose the intervention group. Students who applied, were accepted into P3, and chose not to participate, were used as statistical controls. To reduce error associated with self-selection and other threats to validity, each participant was matched at baseline to a control subject with similar characteristics with respect to indicators known to influence student performance. These indicators, obtained during the first semester of the sophomore year, included (a) school district, (b) gender, (c) self-identified ethnicity, and (d) performance in Algebra I. If more than one student in the pool of nonparticipants was eligible to serve as a control, the match was conducted randomly, a situation which occurred in only two cases.
In addition to assessing outcome measures related to the goal, there was interest in whether participants had gained village-specific content knowledge. For this question, intervention students served as their own controls on village leader-designed pre- and postassessments. Analyses were conducted using a multivariate repeated-measures design. In addition, qualitative data from interviews and informal satisfaction surveys were conducted to gather information regarding students’ perceptions of the village experiences and their impact on STEM-B coursework and plans for future postsecondary education.
Cohorts are defined by the semester or summer in which they started P3. In the first semester of P3, student applications exceeded the available slots, so applicants were randomly selected for Cohort 1. In the subsequent summer session, students not selected originally for Cohort 1 were invited to participate as replacements for the dropouts. Thereafter, recruitment continued to achieve a minimum cohort size of 40. Cohorts are defined by the semester or summer in which they started P3. Attempts were made to match all intervention students with controls (those that applied but chose not to attend) within their cohort to reduce variability due to length of participation; 84.8% of matches were made within cohort. Attempts were made to create matches within schools and were successful for all but two students. In these two cases, matchers were made within school district.
Table 3 shows the participation and retention of intervention students who were recruited to and who completed P3 villages over the first 2 years of the project. As can be seen in the table, of the 52 students who completed the pilot (first semester, Cohort I), 50% were still participating at the end of Year 2. For all cohorts combined, over 17 months of activities, the retention rate was 45.2%. Factors affecting retention will be considered in the conclusion section.
Participation and Retention Rates for Years 1 and 2
|Note: Includes 5 students who participated in spring 2009, did not participate in summer 2009, and returned for fall 2009 P3 sessions. E = Enrolled, C = Completed.|
Although 42 students completed the fourth session in spring 2010, 9 of those students moved, and their year-end outcome data were unavailable. Thus, data analyses were conducted only for the remaining 33 students. No significant differences were noted for those who dropped out versus those who remained in the sample dataset (p > 0.05) with regard to grades attained in Algebra I or Biology I as covariates of concern. Table 4 shows participant type by gender and ethnicity.
Participant Type by Gender and Ethnicity
|Asian/ Pacific Islander|
Data Collection Methods
All P3 student applicants to the program must gain parental approval to participate; complete the student application, including demographic descriptors and a short survey about their use of and interest in technologies; and agree to allow project staff to contact their school to access their transcripts and cumulative records. Application forms can be found on the project website (http://primevillages.asu.edu). Upon completion of the application process, students were notified of their placement in one of the four concurrent villages being conducted.
Additional data, both quantitative and qualitative, have been gathered throughout the P3 experience. The types of data collected include the following:
Pre and post Village-Specific Knowledge Assessments focusing on village content were created by the village leaders and contained from 7 to 25 questions. They assessed basic content knowledge developed through village activities. All participants in P3 completed pre- and postassessments in their villages. Villagers were given the same assessment at the beginning and at the completion of their village exploration to measure knowledge gained during village participation and, thus, function as their own controls in the repeated measures design. The control group did not complete the village assessments.
Updated Academic Performance Measures were collected annually from the school districts by project staff for both intervention students and their matched controls following the spring semester. The measures obtained from transcript review included current GPA, GPA specific to STEM-B courses, and results from state standardized tests (Arizona’s Instrument to Measure Standards, or AIMS) in reading, mathematics, writing, and science. Grades are based on a 5-point scale with an A grade in an Advanced Placement or honors classes awarded 5.0 points, a B awarded 4.0 points and so on. Grades in regular courses are based on the conventional system of A awarded 4.0 points, B awarded 3.0 points, and so on. Plans for collection of SAT and ACT college admission exam scores, along with final class rank will be conducted at the conclusion of students’ senior year (spring 2011) and the conclusion of P3.
Progress in completing advanced coursework data were collected annually for intervention students and their matched controls and included the number and types of STEM-B honors, advanced placement, and dual enrollment college courses students completed.
The Student Plans for Postsecondary Options Survey requests information about students’ plans for after high school graduation, their intended college major if they plan to go to college, the job they want to do when entering the work force, and talents they believe will make them successful in their chosen careers.
One-to-one interviews were conducted in summer 2010 by the P3 evaluator with 8 students randomly selected from among students who began P3 in spring 2009 (Cohort I). Eight questions were posed, including the following:
- “How has the P3 experience affected your high school educational experience?”
- “How has the P3 experience affected your relationship with and/or expectations of teachers since they have been participants just like you in P3?”
- “How have your goals for education and work after high school changed as a result of P3?”
The interviews were used to validate and further explore other quantitative findings.
Data Analysis Methods
To address the research question, several analyses were completed. For all multivariate analyses, Wilks Lambda along with its associated degrees of freedom (Wd1,d2), F, and the significance level (p) were calculated. Univariate F-tests and interpretation of means were conducted if multivariate significance was found, thus controlling the alpha error and the potential to find statistically significant results that were, in fact, not meaningful. In addition, t-tests were completed for each set of village pre- and posttest assessments of intervention effects.
First, an assessment of potential covariates was conducted to determine if any external indicators could account for group differences that would not be represented by differences due to the intervention. A multivariate analysis of variance (MANOVA), using the 33 sets of matched subjects in the P3 intervention and control groups, was conducted with an additional between-subjects effect of cohort (I to IV) added.
Because intervention and control students had been matched on Algebra I performance at the onset, it was not included among the potential covariates. Dependent measures thought to be significant included grades in Biology I and Geometry I. Results, however, indicated no significant differences in these potential covariates between groups, W(2,57), F = 0.259, p = 0.773), cohorts, W(6,114), F = 1.028, p = 0.411, or cohorts within groups, W(6,114), F = 1.023, p = 0.414, at the onset of the project. These two potential covariates, therefore, were not included in any subsequent models. Their descriptive data are presented in Table 5.
Potential Covariates: Geometry I and Biology I
Potential Covariate Factors
|1. Geometry I|
p = 0.511
|2. Biology I|
p = 0.525
A second analysis of potentially confounding factors due to academic achievement was conducted using similar methodology. Because the sample size had decreased from 42 to 33 completers with matching controls, a separate assessment of differences in scores on standardized tests, taken in fall or spring of the sophomore year, was conducted. Only 29 students had all assessments completed at the time of analysis. Again, results indicated no significant differences between the intervention and control groups on measures of reading, writing, mathematics, or science achievement (p > 0.05). Descriptive statistics for the groups are displayed in Table 7. Results, however, indicated no significant differences in these academic measures as covariates between groups, W(4,54) = 0.935, F = 0.942, p = 0 .744, at the onset of P3.
Covariate Factors from the AIMS Test
|1. AIMS Reading (N = 29)|
p = 0.071
|2. AIMS Writing (N = 29)|
p = 0.504
|3. AIMS Math (N = 29)|
p = 0.053
|4. AIMS Science (N = 13)|
p = 0.235
|Cumulative GPA (Freshman GPA)|
(N = 33)
p = 0.803
As can be seen in Table 7, for the 13 students who took the AIMS Science test during their sophomore year (it was not state mandated in 2009), significant difference was found between the intervention and control groups (p = 0.14). Based on these results, the match between the 33 students enrolled in P3 and those used as controls was considered satisfactory.
Descriptive Results of the Pre- and Postassessments for Fall 2009 Villages
|Documentary Film Design and Post-Production|
(N = 4)
p = 0.180
|Wind Energy (N = 15)|
p < 0.001
|Visual Programming and Gaming with Scratch®|
(N = 8)
p = 0.626
|Exploration of Scientific Puzzlers|
(N = 18)
p < 0.001
To address the impact of the P3 project, several different analyses were conducted. Each analytic method was determined during the planning of the evaluation methodologies, and several directional hypotheses were proposed at that time based on the potential for an effective intervention through P3.
Increasing Knowledge as a Result of P3 Participation
A repeated-measures MANOVA was conducted on the pre- and posttests from the eight villages during the 2010 academic year. Results indicated significant difference in pre-and posttest performance over time, W(1,78) = 0.427, F = 104.53, p < .001, as well as differences between villages over time, W(7,78) = 0.557, F = 8.86, p < .001. Descriptive data regarding changes in knowledge are presented in Tables 7 and 8, along with univariate probability results for each village. A one-tailed alpha level of 0.05 was set to assess the significance of the improvement from the beginning to the end of the Scientific Village sessions for both fall and spring. This level was set a priori based on expectations of the impact of the P3 project.
Descriptive Results of the Pre- and Postassessments for Spring 2010 Villages
|Aviation: Flight Training (N = 13)|
p = 0.094
|Engineering Design (N = 12)|
p < 0.001
|See C: Advanced Computer Programming (N = 3)|
p = 0.013
|Trauma Simulation (N = 13)|
p < 0.001
As can be seen in Table 7, two of the four Scientific Villages in fall 2009 (i.e., Wind Energy and Scientific Puzzlers) showed significant gains in knowledge throughout the semester, despite small sample sizes in two of the groups. Each village had its own assessment, based on educational objectives, and the number of items varied on the assessments from five questions and two rating scales in Visual Programming to 10 items for both Documentary Film and Scientific Puzzlers.
As can be seen in Table 8, students in three of the four Scientific Villages during spring 2010 (i.e., Engineering Design, Advanced Programming, and Trauma Simulation) showed significant gains in knowledge throughout the semester, despite small sample sizes in the groups. Again, each village had its own assessment, based on educational objectives and core STEM concepts, and the number of items varied from four multipart questions in the Trauma Simulation Village to 25 items on the Aviation Village pre- and posttests. Students in the Advanced Programming language already had experienced another programming village in the prior session (it was a prerequisite), and therefore, their baseline scores were expected to be and were quite high (ceiling effect). Thus, the lack of significant growth in terms of pre-post change noted in that village was not surprising.
Increasing Interest in STEM-B Courses as a Result of P3 Participation
For quantitative data a MANOVA was utilized to determine differences between the matched intervention and control groups on a number of outcome measures assessing increased interest in STEM-B coursework. Data on two specific dependent measures representing interest in STEM-B included the number of STEM-B courses taken during the sophomore and junior years in high school as well as the cumulative GPA achieved in those courses.
Results indicated significant differences between the intervention and control groups with respect to courses taken and GPAs achieved in those courses, W(2,31) = 0.702, F = 6.58, p = .004. The descriptive statistics and probabilities associated with the univariate F-statistic for both dependent measures are presented in Table 9.
Increasing Interest in STEM-B Courses
|Number of STEM-B courses taken|
p = 0.001
|GPA in STEM-B courses|
p = 0.030
Results indicate a significantly higher level of interest by students attending P3 to engage in STEM-B coursework in comparison with the matched control students, F(1,32) = 12.68, p = .001. Intervention students were most likely to take additional courses including Chemistry (48%), Precalculus or Calculus (42%), Algebra II (27%), Physics (24%), and Advanced Biology (18%). Other STEM-B courses included elements of video, photography and graphic design (12%), Engineering Introduction and Design (12%) and Anatomy and Physiology (12%). More than 50% of these STEM-B courses were taken by students at the advanced placement or honors level. In addition, intervention students outperformed those in the control, F(1,32) = 5.16, p = .030. They were attaining higher grades in these courses with higher overall GPAs.
These findings are strongly supported by students’ self-reporting of interest in STEM-B careers and postsecondary options on the Students’ Postsecondary Options survey. Intervention students at the end of their junior year in high school were asked to identify postsecondary and career goals. It is noteworthy that all students had postsecondary educational goals, including community college or technical school (15%), community college matriculating to a 4-year institution of higher education (4%), 4-year institution of higher education (65%), or postbaccalaureate programs including graduate school, medical college, or advanced degrees (16%).
When asked about the nature of their intended college major, 58% described one or more STEM-B fields (e.g., mathematics, marine biology, chemistry, agribusiness, computer science, medicine, nursing, engineering), 23% identified a major in liberal arts (e.g., music, political science, psychology, criminal studies, journalism, film), and 19% stated that they were undecided at the end of their junior year in high school.
Impact on Teachers
The overall evaluation plan for the project directed two inquiries into changes in teacher behaviors as a result of participation in P3. Of great interest was how the P3 experience was affecting their teaching methods and their expectations for performance of the students in their classrooms. Although the summer connections course focused on making connections for integrating problem-based learning strategies into STEM subject areas, the project also offered direct instruction and support in grant writing.
Mechanisms for collecting data on P3 teacher behaviors included (a) brief surveys to better understand the ripple effects of engagement in P3 project to their students, (b) one-on-one interviews of teachers, (c) comments gathered from the anonymous Participant Satisfaction Surveys completed by teachers during every semester and summer session, and (d) artifacts from students of P3 teachers including students’ papers and lab documentations of project work, photos of student projects and final products, and some of the lesson plans that led to those products. Some teachers and their students made videos or YouTube films , for example, to facilitate student success with key concepts of AP Chemistry and difficult-to-learn mathematical algorithms.
Of great interest has been the education of teachers and students together. Not only has P3 offered teachers of specific subject-content (e.g., math, science, or technology) cross-disciplinary opportunities for learning (almost all teachers noted that they felt better prepared to delve deeper into content in their own and related fields), but also the opportunity to learn and work side by side with students who represent their classroom populations. Many teachers commented on how their instructional approaches have changed as a result of observing how much students know and can do.
About one third of teachers said that they were using long-term projects or investigations to enhance learning and exploration of new concepts. Twenty-eight percent described their increased use of activities to develop students’ critical thinking and problem solving talents. Twenty-one percent cited several methods they employed to foster collaborations among students. Most teachers stressed that, first and foremost, they wanted to offer more opportunities for their students to work together in groups for “creative idea generation” and “brainstorming and problem solving.”
Almost 30% of teachers increased expectations for their students’ engagement and performance based on their collaborative work with P3 students in scientific villages. Twenty-eight percent identified increased awareness of what interests and motivates students. Many teachers commented on the comfort level of students with technology, and the ability of students to assist teachers when challenges with technology occurred. One teacher summed up the experience: “We are digital immigrants while they [the students] are digital natives.”
P3 offered teachers the chance to experience problem-based learning for themselves, and they better understood the elements of the problem solving/scientific inquiry approach and saw its applicability in their own classrooms. They experienced their own higher levels of engagement and learning and witnessed the same in the students in their villages. They saw more clearly the positive outcomes engaging and encouraging students to bring to bear what they know—their knowledge and problem solving abilities—to solve problems and to offer fewer and shorter lectures only when students’ need for information is high and curiosity is piqued.
Following are some of the more poignant statements that captured the essence of the P3 impact on teachers.
- “I have tried to develop lessons/projects that show my students the relevance of math in the real world.”
- “I look for more ways for students to display their creativity in problem solving. Generally I provide a few specifications for a project and allow students to explore and develop projects along their interests.”
- “P3 has caused me to be more of a facilitator to my students by helping them answer their own questions.”
- “Interestingly enough, this experience has not only increased my expectations for my students, it has also increased my expectations for myself.”
- “I was able to observe how this type of instruction and the methods used are applied to the project involving high school students.”
- “The students in this program are engaged and motivated. That is the goal of me [as a] classroom teacher. If I can do things that engage students like P3, I think I would be a better teacher for it.”
- “I learned how to engage students who are normally reticent to participate” [through problem solving.]
- “My expectations are higher, yet the students seem to achieve what I am looking for with less effort on my part. I’m not pulling teeth anymore.”
- “I now look at the big picture and look for real world applications instead of being satisfied with assigning the questions at the end of the chapter. If the questions/problems don’t have real world applications, and I can’t find an application for a problem, I don’t assign it. It is critically important to me that there are connections between what is happening in the classroom and what is happening in the world. Those connections need to be evident to me and to my students.”
Among the accomplishments of P3 teachers is their success in securing external funding for projects in their classrooms. To date, more than 30 applications have been developed by P3 teachers, and 20, representing the work of 9 teachers, were funded in the range of $300 to $30,000. This funding is in addition to awards of instructional materials, including software, technology devices, and text.
Based on data collected over the 17 months of this 3-year project, we have evidence of the power of the P3 Scientific Village culture to increase students’ interest and success with the study of mathematics and science in high school. Results of analyses of baseline performance measures for intervention and control students showed no significant differences in achievement on the AIMS state tests of reading, writing, mathematics, and science and performance in Algebra I and Biology I. However numbers and difficulty levels of high school courses in mathematics and science completed by intervention and control students over the period of the project showed significant differences. Intervention students completed more honors, advanced placement, and dual college-high school enrollment courses than did their matched controls, an indication of their greater interest in these subjects.
This great interest was confirmed through one-on-one interviews with intervention students and their responses to the Postsecondary Options survey in which they indicated their plans for after high school graduation. Of course, the ultimate goal of P3 is to enhance the pipeline from high school to college STEM-B majors. By the end of this 3-year project, that information, as well as their performance on college entrance examinations will be available.
Also of interest are the gains in knowledge made by intervention students through participation in the scientific villages. As may be expected from any pre-post assessment of learning over a relatively short period of time, all students showed increased understanding of the village-specific concepts, skills, and content. Interestingly, while all gains were significant, the least gain was in the villages focusing on computer gaming and applications designs. This may be the result of students’ great interest in computer technology, their prior experience with technology, and their risk-taking behaviors, eagerness, and excitement about experimenting with new software.
In surveys and interviews, all P3 teachers were effusive in their support of the Scientific Village strategy for increasing their and the P3 students’ knowledge of STEM concepts and skills; for connecting them with experts (scientists, university faculty, and other teachers) for future collaboration; for introducing them to new methods of assessment and ways to use information collected; and for updating their academic-counseling skills. They highlighted the collaboration with students as the hallmark of the program and stressed the value of the scientific village strategy as a component in the preparation of preservice teachers and the updating of in-service teachers in mathematics, the sciences, and technology.
Allen, W. R., Bonous-Hammarth, M., & Suh, S. A. (2005). Who goes to college? High school context, academic preparation, the college choice process, and college attendance. In E. St. John (Ed.), Readings on equal education (Vol. 20). Improving access and college success for diverse students: Studies of the Gates Millennium Scholars Program (pp. 71-113). Brooklyn, NY: AMS Press.
Allen, W.R., Bonous-Mammarth, M., Yang, J., González, G., & DuCros, F. (2004, April). I know I can! I know I can! Comparing high school context, academic preparation and the college choice process among Gates Millennium Scholarship recipients and non-recipients. Paper presented at American Educational Research Association Convention, San Diego, CA.
American Association for the Advancement of Science. (2009). Benchmarks on-line. Retrieved from http://www.project2061.org/publications/bsl/online/index.php
Arizona Department of Education. (2009). Arizona’s instrument for measuring standards: AIMS report wizard. Retrieved from http://www.ade.state.az.us/profile/publicview
Arrington, K. (2000). Middle grades career planning programs. Journal of Career Development, 27(2), 103-109.
Aronson, J., Zimmerman, J., & Carlos, L. (1998). Improving student achievement by extending school: Is it just a matter of time? Retrieved from http://www.wested.org/cs/we/view/rs/95
Boyd, D. J., Grossman, P. L., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Teacher preparation and student achievement. Educational Evaluation and Policy Analysis, 31(4), 416-440.
Bray, H. (2010, June 22). Raytheon CEO calls for more tech education. Business Updates from the Boston Globe. Retrieved from http://www.boston.com/business/ticker/2010/06/raytheon_ceo_is.html
Bronson, E. (2007). Helping CTE students learn to their potential. Techniques: Connecting Education and Careers, 82(7).
ChanLin, L. J. (2008). Technology integration applied to project-based learning in science. Innovations in Education and Teaching International, 45(1), 55-65.
Clark, A. C., & Ernst, J. V. (2007). A model for the integration of science, technology, engineering, and mathematics. Technology Teacher, 66(4), 24-26.
Corcoran, T., & Silander, M. (2009). Instruction in high schools: The evidence and the challenge. Future of Children, 19(1), 157-183.
Couto, V., Mani, M., Lewin, A. Y., & Peeters, C. (2007). The globalization of white-collar work: The facts and fallout of next-generation offshoring. Retrieved from the Duke University Fuqua School of Business Offshoring Research Network website: https://offshoring.fuqua.duke.edu/pdfs/gowc_v4.pdf
Darling-Hammond, L., Barron, B., Pearson, P., Schoenfeld, A. H., Stage, E.K., Zimmerman, T.D., Cervetti, G.N., & Tilson, J. (2008). Powerful learning: What we know about teaching for understanding. San Francisco, CA: John Wiley & Sons Inc.
Duschl, R. (2008). Science education in three-part harmony: Balancing conceptual, epistemic, and social learning goals. Review of Research in Education, 32(1), 268-291.
Fernandez-Santander, A. (2008). Cooperative learning combined with short periods of lecturing: A good alternative in teaching biochemistry. Biochemistry and Molecular Biology Education, 36(1), 34-38.
Geier, R., Blumenfeld, P. C., Marx, R. W., Krajcik, J. S., Fishman, B., Soloway, E., & Clay-Chambers, J. (2008) Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. Journal of Research in Science Teaching, 45(8), 922-939.
Gewertz, C. (2007, March 29). Outside interests. Education Week. Retrieved from http://www.edweek.org/ew/articles/2007/03/29/30tcstudent.h26.html
Griffin, K. A., Allen, W. R., Kimura-Walsh, E., & Yamamura, E. K. (2007). Those who left, those who stayed: Exploring the educational opportunities of high-achieving black and latina/o students at magnet and non-magnet Los Angeles high schools (2001-2002). Educational Studies: Journal of the American Educational Studies Association, 42(3), 229-247.
Haladyna, T., Olsen, R., & Shaughnessy, J. (1982). Relations of student, teacher and learning environment variables to attitudes toward science. Science Education. 66, 671-687.
Helle, L., Tynjala, P., Olkinuora, E., & Lonka, K. (2007). Ain’t nothin’ like the real thing: Motivation and study processes on a work-based project course in information systems design. British Journal of Educational Psychology, 77(2), 397-411.
Hill, H., Rowan, B., & Ball, D.L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371- 406.
Hughes, K. L., Bailey, T. R., & Karp, M. M. (2002). School-to-work: Making a difference in education. The Phi Delta Kappa International, 84(4), 272-279.
Jerald, C. D., (2009). Defining a 21st century education. Retrieved from the Center for Public Education website: http://www.centerforpubliceducation.org/Learn-About/21st-Century/default.aspx
Jones, M. S. (2005, April). Science for all: Promoting interest and participation in a diverse middle school. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
Kalman, C. S. (2007). Successful science and engineering teaching in colleges and universities. Boston, MA: Anker Publishing Company, Inc.
Kazis, R. (2005). Remaking career and technical education for the 21st century: What role for high school programs? Retrieved from the Jobs for the Future website: http://www.jff.org/sites/default/files/RemakingCTE.pdf
Kendall, J. S., Pollack, C., Schwols, A., & Snyder, C. (2007). High school standards and expectations for college and the work-place (Issues & Answers Report, REL 2007-No. 001). Retrieved from the U.S. Department of Education, Institute of Education Sciences, website: http://ies.ed.gov/ncee/edlabs/regions/central/pdf/REL_2007001_sum.pdf
Markham, T., Larmer, J., & Ravitz, J. (2003). PBL handbook: A guide to standards-focused project based learning for middle and high school teachers. Novato, CA: Buck Institute for Education.
Meeder, H. (2008). The Perkins Act of 2006: Connecting career and technical education with the college and career readiness agenda (2008 Policy Brief). Retrieved from the Achieve, Inc., website: http://www.achieve.org/files/Achieve-CTEPolicyBrief-02-07-08.pdf
Mullis, V.S., Martin, M.O., & Foy, P. (2008). TIMSS 2007 international mathematics report: Findings from the IEA’s trends in international mathematics and science study at the fourth and eighth grades. Boston, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
National Academies of Science. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: The National Academies Press.
National Governors Association for Best Practices and the Council of Chief State School Officers. (2010). K-12 common core state standards for mathematics. Washington, DC: Author.
National Mathematics Panel. (2007). Final report on the national survey of algebra teachers for the National Math Panel. Washington DC: U.S. Department of Education.
National Science Board. (2010). Science and engineering indicators 2010. Arlington, VA: National Science Foundation.
Organization for Economic Cooperation and Development. (2007). Education at a glance 2007. Paris, France: Author.
The Partnership for 21st Century Skills. (2007, September). Beyond the three Rs: Voter attitudes toward 21st century skills. Retrieved from http://www.p21.org/documents/P21_pollreport_singlepg.pdf
Public Works. (2006). From education to work: Is Arizona prepared? (The Alignment Project Report). Phoenix, AZ: Arizona P-20 Council.
Secretary’s Commission on Achieving Necessary Skills, U.S. Department of Labor. (1991). What work requires of schools: A SCANS report for America 2000 (Report No. 20919). Retrieved from http://wdr.doleta.gov/SCANS/whatwork/whatwork.pdf
Sedlacek, W. E. (2004). Beyond the big test: Noncognitive assessment in higher education. Hoboken, NJ: Jossey-Bass.
Sedacek, W.E., & Sheu, H.B. (2004, January). Predicting the academic success of Gates Millennium Scholars. Paper presented at the American Education Research Association Convention, San Diego, CA.
Seymour, E. (1995). The loss of women from science, mathematics and engineering undergraduate majors: An explanatory account. Science Education, 79(4), 437-473.
Smith, K. A., Sheppard, S. D., Johnson, D. W., & Johnson, R. T. (2005). Pedagogies of engagement: Classroom-based practices. Journal of Engineering Education, 94(1), 87-101.
State Educational Technology Directors Association. (2008). Science, technology, engineering. Retrieved from http://www.learning.com/press/pdf/Science-Technology-Engineering-Mathematics-STEM-Report.pdf
Stigler, J. W., & Hiebert, J. (2009). Closing the teacher gap. Phi Delta Kappa, 92, 32-37.
Stigler, J. W., & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York, NY: The Free Press.
Stone, J. R. III, Alfeld, C., & Pearson, D. (2008). Rigor and relevance: Enhancing high school students’ math skills through career and technical education. American Educational Research Journal, 45(3), 767-795.
The authors wish to acknowledge the support of the National Science Foundation (#0833760, 2008 – 2011) for P3, as well as the contributions of a great many outstanding STEM-B practitioners. With the exception of those with identified affiliations, all others are faculty or staff at Arizona State University. Jim Anderson (Aviation), John Black (Perceptive and Cognitive Computer Development), Sean Dengler (Engineering Lab Supervisor), Mark Henderson (Engineering), Jon Howell (Health Informatics, East Valley Institute of Technology), Chris Lamont (Film Production), Tim Lindquist (Computer Science), Colleen Megowan (Physics Education), Al Mittelstaedt (Aviation), Lakshmi Munukutla (Electrical Engineering), Marc O’Brien (Aviation), Robert Pahle ( 3-D Modeling Technologies), Eric Perez (Computer Science, East Valley Institute of Technology), and Brad Rogers (Engineering).
We thank Ida Malian (Special Education) and Eugene Judson (Science Education) for leading Connections Course discussions; Carol Findell (Boston University) for collaborating on the evaluation instrument design; Deborah Toolson for her project director work during the first 1½ years of P3; Michael Piburn, for his assistance with the evaluation at the beginning of P3; and Tim Gunty (ASU undergraduate STEM major) for serving as senior mentor responsible for the training of all other village mentors. We are indebted to Intel and Motorola for supporting P3 by sending scientists to work with village leaders and villagers.
Arizona State University
email: [email protected]
Wattle and Daub Research
Rio Salado College
Arizona State University
Arizona State University
Villagers explored cell phones, cell towers, cellular networks, how they work and how they are designed, they constructed models of cellular networks for areas serving 50,000 cell phone users, and they prepared business plans taking into consideration the cost of construction, sustainability, aesthetics, and evolving patterns of usage.
Film and Media Production: Lights, Camera, Action!
Villagers explored technologies and techniques used by professional film and video producers, wrote original scripts, directed, used appropriate lighting and audio techniques, and filmed short movies.
Film and Media Post-Production
Villagers applied nonlinear digital editing using Final Cut Express software and added credits, graphics, music, and sound effects to their films.
Documentary Film Design and Postproduction
Villagers created short documentaries and learned techniques of interviewing, lighting, audio recording, directing, digital filming, and film editing.
Wind Energy: Harness the Wind
Villagers studied wind energy and its uses, conducted home energy audits, recorded and analyzed instantaneous wind data, and constructed and tested model wind turbines.
Engineering Design: Designing Rockets and Robots
Villagers designed, programmed, and tested model rockets. They researched, designed, programmed, and constructed Sumo Robots using light and motion sensors, and conducted a Sumo Robot competition.
Using principles of mathematics and physics, villagers constructed model wind turbines and tested them to determine wind turbine efficiency in a wind tunnel custom built for this project.
Visual Programming and Gaming with Scratch®
Villagers designed and programmed computer-based interactive stories, animations, and games, and created and played music using Scratch® software developed at MIT.
See C: Advanced Computer Programming
Villagers explored Objective-C and programmed applications for Apple® devices, including the iPhone®, iPod®, iPad® and Macintosh® computers.
Scratch®, C, and iPod® Development
Villagers created stories, animations, games, and applications programmed in Scratch® or created applications for the iPhone® using Objective-C.
Exploration of Scientific Puzzlers and What We Can Learn from Them
Villagers explored the physics of light and color, peripheral vision, depth and motion perception, color and brightness constancies, optical illusions, photography, image processing, and human vision and used MatLab® and Adobe Photoshop®.
3-D Virtual Modeling for Emergency Services
Villagers explored Geographic Information Systems, databases, 3-D modeling and building design, and web-based content management systems to construct virtual models of local buildings, to assist emergency responders and decision makers.
Villagers investigated normal structures and functions of major systems and key organs of the human body, researched the impact of trauma and various disorders on these systems and organs, and created two-dimensional virtual tours of the major organ systems of the human body using Adobe® Creative Suite® (CS4) software.
In a forensic microscope laboratory, villagers performed fingerprinting analysis, isolated and studied their own DNA through hair and saliva samples, and analyzed common foods to determine if they contain genetically modified ingredients. Using simulated crime scenes, villagers practiced laboratory techniques to identify criminals.
Aviation: Flight Training
Villagers learned instrument flying in ASU’s flight simulators, explored the biomechanics of flight, calculated lift and angle of attack, velocity and lift, lift and air density, the rate of descent required for the Instrument Landing System (ILS) Approach, and determined a visual descent point.
Aviation: Flight Training, The Sequel
Villagers studied GPS technology for navigating planes and used ASU’s radar control simulator, as well as engaging in flight simulator and instrument landing exploration.
Villagers studied the cleanroom environment and its role in the manufacture of various nanostructures, microelectronic devices, pharmaceuticals, and hospital surgical rooms, and designed and constructed a functioning model cleanroom to industry-standard specifications by using CAD.
Cellular Communications Network Design – http://primevillages.asu.edu/projects/1
Cleanroom Science – http://primevillages.asu.edu/projects/2
Film and Media Production – http://primevillages.asu.edu/projects/3
Film and Media Post-Production – http://primevillages.asu.edu/node/65
3-D Virtual Modeling for Emergency Services – http://primevillages.asu.edu/node/64
Visual Programming and Gaming with Scratch – http://primevillages.asu.edu/node/63
Wind Energy – http://primevillages.asu.edu/node/62