STEM Literacy in the Classroom to Enable Societal Change

Society and work environments are changing rapidly due to the innovations of the Fourth Industrial Revolution (4IR), which is characterized by the use of emerging technologies such as artificial intelligence, biotechnology, the internet of things, and autonomous vehicles, together with the ways humans interact with these technologies (Schwab & Davis, 2018). A number of reports describe how the automation of tasks performed in the workplace with advances in information technology and robotics will displace but not replace workers by creating new jobs that require unique abilities, knowledge, and high-level skills (Frey & Osborne, 2017; Gorle & Clive, 2013). Rather than eliminating labor, automation is likely to reshape the distribution of jobs, creating a greater demand for critical thinking, judgment, and higher levels of human interaction (Frey & Osborne, 2017).

Marr (2019) suggested that schools have several challenges to prepare students for the 4IR, including improving science, technology, engineering, and mathematics (STEM) education, developing the human potential to partner with machines rather than compete with them, adapting to lifelong learning models, facilitating student inquiry, and encouraging collaboration and creativity. Embedding computational thinking (CT) practices within mathematics and science courses provides opportunities to prepare students better as creative and critical thinkers to meet the future needs of the job market (Grover & Pea, 2013; National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010b; National Research Council [NRC], 2012).

Yadav et al. (2016) stated that many constraints exist to teaching CT within the context of a standalone computer science class. Therefore, providing professional development (PD) opportunities for teachers is important to prepare them better to integrate CT within specific disciplines. For students to prepare for successful careers in STEM fields that align with the needs of the 4IR, they need to move beyond mathematics and science curriculum that focus purely on the facts of each field and participate in a multidisciplinary approach. This multidisciplinary approach also includes the importance of the development of technical reading and writing skills as part of authentic, meaningful tasks that allow students to communicate what they have learned based upon evidence from reputable sources and direct observations of empirical investigations.

Within this context, we developed a summer institute aimed at addressing the need for high quality PD in disciplinary CT strategies. The STEM Literacy in the Classroom to Enable Societal Change project, herein referred to as STEM Lit, focused on the use of engineering challenges as a means to develop middle and high school science, mathematics, and STEM teachers’ content and pedagogical skills through the use of coding, robotics, 3D printing and modeling, and technical reading and writing. This paper describes our investigation of teachers’ experiences as participants in the institute, in which we examined the following questions.

1. What conceptual and attitudinal changes did teachers experience in terms of mathematics and science-specific technological pedagogical content knowledge?
2. How did participation in this institute impact teacher’ use of disciplinary CT strategies as applied to middle and high school mathematics and science?
3. How did participation in this institute impact teacher self-efficacy regarding using and teaching technical reading and writing strategies?

Literature Review

Disciplinary Computational Thinking

The Computer Science Teachers Association (CSTA) and International Society for Technology Education (ISTE) have developed an operational definition for CT that includes the following characteristics:

• Formulating problems in a way that enables us to use a computer and other tools to help solve them;
• Logically organizing and analyzing data;
• Representing data through abstractions such as models and simulations;
• Automating solutions through algorithmic thinking (a series of ordered steps);
• Identifying, analyzing, and implementing possible solutions with the goal of achieving the most efficient and effective combination of steps and resources; and
• Generalizing and transferring this problem-solving process to a wide variety of problems (ISTE & CSTA, 2011).

Weintrop et al. (2016) developed a CT in mathematics and science practices taxonomy that included four major categories: data practices, modeling and simulation practices, computational problem-solving practices, and systems thinking practices. The vision for CT in the classroom includes opportunities for students to find multiple ways to solve problems with and without computers and to learn how to work as a collaborative team and use key concepts through discourse to achieve a common goal (Barr & Stephenson, 2011; Yadav et al., 2016).

The K-12 Computer Science Framework (CSTA, 2016) outlined clear relationships between computer science, science and engineering, and mathematical practices embedded within Next Generation Science Standards (NRC, 2013) and Common Core State Math Standards (CCSMS; National Governors Association [NGA] Center for Best Practices & Council of Chief State School Officers [CCSSO], 2010). These direct connections include develop and use models, abstractions, or artifacts; make sense of problems and persevere in solving them and define computational problems; engage in argument from evidence and communicate about computing; reason abstractly; use tools strategically; analyze and interpret data; and plan and carry out investigations.

Kale et al. (2018) highlighted three strategies for helping teachers make connections between CT and their teaching: provide specific content examples that use CT tools; recognize the similarities between problem solving and CT; and operationalize the methods for teaching problem solving in the context of teaching CT. Furthermore, Asunda (2018) emphasized that integrating CT through design challenges is similar to the engineering design process and exposes students to essential skills in problem solving, teamwork, time management, communication, and leadership strategies that are important for STEM-related careers.

Integrated STEM PD

The Need for STEM PD

While the United States has been a world leader in technological innovation driven by knowledge and proficiency in STEM, the majority of the technical elite workers in the United States were born and educated elsewhere. The American Physical Society’s report, Recruiting Teachers in High-Needs STEM Fields (Marder et al., 2017), detailed that the lack of qualified teachers at the high school level, particularly in physics, chemistry, computer science (CS), engineering, and earth and space sciences, limits what courses are offered, particularly in comparison with other countries. This shortage is exacerbated in high-need, lower socioeconomic communities. Of specific interest to this study, 70% of future STEM jobs will be in the field of CS, and only 25% of high schools offer CS courses. 

Integrated STEM education is an approach in which students participate in design challenges centered around real-world problems through the integration and application of science, mathematics, and engineering (Thibault et al., 2019). Preparing teachers to use this approach requires the consideration of several challenges that differ from single-discipline PD. Real-world design challenges are often ill-defined, and teachers are asked to integrate multiple content areas that require distinct pedagogical content knowledge in which they have not received significant preparation (Bush et al., 2020).

Research studies provide evidence that only a small number of teachers feel prepared or comfortable teaching integrated STEM content (Du et al., 2018). Effective STEM PD should provide opportunities to develop content and pedagogical knowledge, collaboration with STEM area teachers or community/industry partners, practice identifying the key mathematics and science content to be taught, and a connection between how content taught within STEM disciplines is used within STEM careers (Bush et al., 2020).

Building on the need for integrated STEM PD is the need to prepare in-service teachers to use CT practices regardless of their respective academic discipline (Sands et al., 2018). PD for noncomputing in-service teachers should be sustained and should focus on supporting integrated CT through the use of problem-solving activities within disciplines. Use of communities of practice is more effective than presenting activities as instructional add-ons to the curriculum. Mouza et al. (2017) observed that most PD efforts focused on embedding CT in K-12 education have occurred primarily at the high school level within dedicated CS classes using CS curricula, rather than with teachers of science and mathematics classes.

Disciplinary Literacy Skills

While secondary mathematics and science teachers may have confidence in teaching their primary content areas, they may perceive numerous barriers to teaching reading and writing skills: secondary teachers feel a responsibility to solely focus on their specific content and a pressure to cover a wide breadth of content, subjects are often taught in isolation, teachers assume that students have mastered literacy comprehension skills, and these teachers lack expertise and support for integrating reading and writing instruction (Chambers Cantrell et al., 2008; Fine et al., 2011; Spencer & Bouwma-Gearhart, 2014). Teacher efficacy and beliefs about integrating content literacy often act as barriers to implementation; therefore, measuring these beliefs and explicitly addressing these as part of PD methodology are important (Chambers Cantrell et al., 2008).

Science and mathematics teacher preparation programs have traditionally equipped future teachers to teach content and strategies specific to their respective fields; however, the NGSS (NRC, 2013), CCSMS (NGA Center for Best Practices & CCSSO, 2010b), and the CCSS for Science and Technical Subjects for grades 6-12 teachers have presented a need for PD for in-service teachers (NGA Center for Best Practices & CCSSO, 2010a). These standards call for the integration of reading complex, discipline-specific texts and technical writing.

TPACK Framework and SAMR Model

The Technology, Pedagogy, and Content Knowledge (TPACK) framework and Substitution Augmentation Modification Redefinition (SAMR) models served as guiding technological frameworks for this project. As described by Mishra and Koehler (2006), the TPACK framework describes how technology is integrated with teaching through the following seven constructs of knowledge: technology (TK), content (CK), pedagogy (PK), pedagogical content (PCK), technological pedagogical (TPK), technological content (TCK), and technological pedagogical content (TPACK). Niess et al. (2009) described a five-stage Mathematics Teacher TPACK Developmental Model that teachers follow when they are learning to integrate technology in teaching and learning mathematics. These levels are Recognizing, Accepting, Adapting, Exploring, and Evaluating (p. 9).

The SAMR model, designed by Dr. Ruben Puentedura (2010), is a framework used to assess and evaluate digital technology use in the classroom. Technology allows learners to think differently and perform new tasks, and this model affirms that the specific technological tool is not as important as how the tool is used to improve student outcomes. The model includes four levels divided into two sections as a means to promote teacher reflection and technology integration. First, the Enhancement section consists of the Substitution and Augmentation levels. Next, the Transformation section consists of the Modification and Redefinition levels. The challenge is for teachers to develop tasks within the Transformation section that lead to greater student engagement, involvement, and ultimately increased student achievement and learning.

Professional Development Design

STEM Lit provided PD for 24 teachers, including 13 middle school (three mathematics, three STEM, and seven science) and 11 high school teachers (one mathematics, one STEM, and nine science). Each participating middle and high school identified at least two teachers to recruit to form a professional learning community (PLC). The project included a 2-week summer institute and one follow-up Saturday during the fall semester, for a total of 54 contact hours. Training focused on the use of engineering challenges that address current societal issues as a means to develop teachers’ knowledge and use of coding, robotics, 3D printing and modeling, technical reading and writing, and statistical analysis skills within their respective content areas. Teacher participants received a $75 daily stipend, a Lego^® Mindstorms^® EV3 Core Set with sensors, Lego^® Renewable Energy Add-on Set, Lego^® Temperature probe, 3D printer, and laptop, as well as a subscription to the National Science Teachers Association (NSTA) Learning Center.   

Professional Development Framework

STEM Lit was conducted jointly by education methods, engineering, mathematics, and CS faculty members to model pedagogy effectively and focus on building content knowledge within the context of embedding CT. Teachers worked in groups of four to explore how mathematics and science are used along with CT practices and technical reading and writing. CT practices were linked directly to CCSMS for content and practices; NGSS for disciplinary core ideas, crosscutting concepts, and science and engineering practices; and CCSS for Science and Technical subjects for grades 6-12.

This approach is in alignment with principles of effective PD for mathematics and science education that suggest the importance of incorporating curriculum connections among state standards and modeling teaching strategies and curriculum materials that are consistent with desired shifts in teaching and learning (Council of State Science Supervisors [CSSS], 2018; Loucks-Horsley et al., 1996). Project staff modeled problem-based learning, flipped learning, and instructional technology during the PD sessions and explicitly addressed teachers’ TPACK within the context of CT practices and technical reading and writing within mathematics and science. 

Institute Schedule and Design

Day 1 of the summer institute allowed for teachers to complete pre-assessments and an escape room icebreaker. Additionally, they sorted the bricks in their Lego Mindstorms EV3 Core Set and began constructing their first build. Subsequent days featured blocks of instruction including modules for mathematics content, introductions to 3D printing and Lego software, engineering grand challenges, and specific pedagogical skills for flipping classroom instruction and the use of technical reading and writing. 

Math Modules

 In planning for the institute, we identified the mathematical tools and concepts that would be necessary for teachers to be successful in designing and programming robots as well as designing 3-D printed artifacts to meet the real-world challenges. Since the teachers came from several backgrounds, we could not assume a common set of mathematical tools. To help address this concern, we designed a series of three mathematics modules using Nearpod ( n.d.) to increase the interactive nature of the presentations. We believed that some direct instruction was necessary to increase the teacher’s fluency with these topics.

The first mathematics module addressed the general concept of modeling: creating a construct, either physical (e.g., a Lego robot) or mathematical (e.g., creating a graph or a function) to make predictions about the behavior of some real-world situations. The module then described how to design a robot to gather data about its position over time, to graph this data using the Lego Mindstorms software, and to export the data to other software like Google Sheets. For this first experiment, the robot was attempting to move at a constant velocity. These data were then used to calculate the rate of change of position (velocity) of the robot, which was then used to create a model of the position of the robot as a function of time. The predictions of this model were then compared with the measured data.

A similar experiment was then conducted with an accelerating robot. The rate of change of position was calculated and then data were used to approximate the acceleration. Finally, models were constructed to predict both the changing velocity and position of the robot as functions of time.

The second mathematics module addressed the use of formulas in constructing models. In particular, the distance formula and the associated Pythagorean theorem were introduced to assist in constructing models to aid in the navigation of a robot by dead reckoning. We also addressed the formula for a circle and the transformation of formulas using horizontal and vertical shifts. The results of these transformations were graphed using Desmos. Finally, we introduced piecewise functions to address situations like a robot turning and changing direction.

The third mathematics module reviewed different methods for describing data, including time series, frequency data, means and medians, and histograms. The calculation of these descriptions using Google Sheets was emphasized in addition to using hand calculation. The identification of outliers and their effects on the data summaries were also presented.

Basic Introduction to Lego Mindstorms EV3 Software 

Based upon a preliminary needs assessment, 11 of the 24 participants had never used the Lego Mindstorms EV3 robots, eight had used them but felt that they were still operating at a novice level, four had some experience and were beginning to feel more confident, and one teacher used them on a regular basis in her after-school robotics club and in her middle school STEM classroom. This information was helpful in designing several opportunities for the participants to explore the software to prepare for extended challenges.

The participants were introduced to the EV3 platform through two activities: the color challenge and the hand piano challenge. In both activities an EV3 sensor was used to determine sounds to generate. In the color challenge it was the optical sensor, while in the hand piano challenge it was the ultrasonic sensor. The participants were given basic build instructions and EV3 programs. They were encouraged to modify these instructions by adding features such as new sounds and interaction with EV3 buttons.

Basic Introduction to 3D Printing

Seventeen of the 24 teachers had never used 3D printers, four had tried them out but still believed they functioned at the novice level, and three teachers were gaining confidence. Since the majority of teachers had no experience and none of the teachers felt that they were experts in this area, we started at an introductory level by introducing how to use the 3D printers and built up to teaching them how to create their own 3D files to print.

We purchased a daVinci Jr. 1.0 3D printer from XYZprinting (2018) for each teacher. These low-cost machines are easy to set up, troubleshoot, and operate. We had the teachers unpack their printers on Day 2 of the workshop, load filament, and print a simple object of their choosing from Thingiverse (Makerbot Industries, 2019). 3D printing comes with specialized vocabulary and skills, so we spent time teaching them the basics, including the file type supported by the printer (STL), 3D printer hardware basics (X, Y, and Z axes, extruder, print bed, how to load and unload filament, etc.), and when they should choose to add supports or a raft to their print. Our challenge for them was to keep their printers in use as much as possible throughout each day, so they could walk away from the 2-week workshop ready to use and troubleshoot the printer in the classroom. 

As participant teams began working on challenges, they were asked to design devices to 3D print and use with their Lego EV3 brick or robot. We introduced them to a free online program called Tinkercad that allows the user to create designs for objects that can be downloaded as STL files and printed on a 3D printer (Autodesk Inc., 2019). Tinkercad allows the user to design objects precisely at the millimeter level that was required for successful completion of the challenges provided. The teachers also spent time troubleshooting and reprinting their objects to find the best fit for their design.

Field Trip to Oak Ridge National Laboratory

We scheduled a 1-day field trip to the Oak Ridge National Laboratory (ORNL) Manufacturing Demonstration Facility (MDF), which is the “nation’s only large-scale open-access facility for rapidly demonstrating early-stage R&D manufacturing technologies and optimizing critical processes” (ORNL, n.d.). The tour enabled participants to see real-world applications of robotics and additive manufacturing situated within their local community. In addition to talking with engineers at the MDF, participants could view and touch a 3D-printed house, jeep, furniture, tools and more.

Real-World Challenges

Teachers worked in cooperative groups to optimize opportunities to construct understanding socially as they developed CT practices and investigated engineering challenges that were associated directly with the standards they teach for mathematics and science. The teams were heterogeneous by means of subject (mathematics, science, and STEM), grade, and observed ability with tools (robotics and 3D printers). The sessions incorporated challenges that were structured using inquiry-based models and engineering design that had direct correlations with four of the grand challenges as described by the National Academy of Engineering (NAE, 2020). Grand challenges as outlined by the NAE with descriptions of how they were incorporated follow.

Advance Personalized Learning. Learning itself will require an engineering solution and is described by the NAE (2020) as follows:

A growing appreciation of individual preferences and aptitudes has led toward more “personalized learning,” in which instruction is tailored to a student’s individual needs. Given the diversity of individual preferences, and the complexity of each human brain, developing teaching methods that optimize learning will require engineering solutions of the future. (Advance Personalized Learning section, para. 1)

Flipped classrooms, project-based learning, and other modern pedagogies were used to instruct participants on how to personalize learning for students and, thus, progress on this engineering grand challenge.

Engineer the Tools of Scientific Discovery. Scientific tools are required for scientific discovery, and these tools must themselves be engineering discoveries. One of the great modern technologies is additive manufacturing or 3D printing, which allows an engineer to create a tangible representation of a discovery that previously existed only in the mind (Lipson & Kurman, 2013).

For this engineering grand challenge participants used the computer assisted drawing (CAD) software, Tinkercad, and 3D printers to create and build custom parts as accessories for the Lego Mindstorms EV3. These parts were used to facilitate experiments with the clean water and solar energy challenges that would otherwise be difficult to perform with currently available tools.

Provide Access to Clean Water. Many barriers exist to providing clean water to the world’s population, including desalination, transportation, and pasteurization. The pasteurization of water, the process of killing bacteria harmful to humans, does not always require a rolling boil (Ciochetti & Metcalf, 1984). To be practical in locales with varying degrees of available resources, water may need to be kept at a midpoint for a long period of time or at a high temperature for a short period of time. This project required participants to use the processing power of the Lego Mindstorms, external sensors including the temperature probe, and data analysis tools. Additional materials provided to each team included a hot plate and a 2-quart saucepan. Each team had to design and print a 3D tool to hold the temperature probe in the pan of water and a device to attach to the EV3 robot that would turn on the hot plate using the plate’s knob and adjust the temperature throughout the challenge.

The goal was to program the robot to turn on the hot plate and heat up the water remaining at a constant temperature for a period of time and then increase temperature and hold at that temperature without any further input from the team. After providing time for teams to troubleshoot their designs and their programming, all teams set up their equipment at the same time and recorded data for 3 hours. See Figure 1 to view several team designs as well as Lego EV3 code used to collect data.

Figure 1   Sample Clean Water Challenge Designs and Lego EV3 Coding

Make Solar Energy Economical. Solar power is the cleanest form of energy, but unfortunately still costs more than traditional fossil fuels, leading to the result that less than 1% of all generated energy worldwide is solar (Ritchie & Roser, 2018). This project utilized the LEGO Renewable Energy Add-on kit to explore ways to make solar power more economical, and the participants were challenged to develop an efficient way to collect solar energy. They used a solar panel included in the renewable energy kit that could be attached to the Lego EV3 brick.

They were provided with a choice of mirrors, including concave, convex, or plane surface, and challenged to design and 3D print a mirror holder to direct light toward the solar panel. After initial planning and troubleshooting for the challenge outside, the weather was rainy on the data collection day; therefore, the teams had to use clamp lights and make adjustments to their programming on the spot to collect data inside. See Figure 2 to view an example team design as well as Lego EV3 code used to collect data. All teams were required to set up their equipment along with a computer to collect data over a 3-hour period. After data collection the teams analyzed their data and wrote a technical report (See Appendix A).  

Figure 2   Sample Solar Panel Design and Lego EV3 Coding

Classroom Pedagogy Focus: Flipped Classrooms

Flipped classrooms are those in which direct instruction is delivered to the students outside of class through the use of video or a digital learning object, such as a simulation, to allow more strategic use of in-class time for group work, differentiated instruction, and project-based active learning (Bergmann & Sams, 2015a, b). Flipped instruction allows for completion of lower levels of cognitive work, such as gaining and comprehending new knowledge outside of class, while higher levels of cognitive work, such as application, analysis, synthesis, and evaluation, is completed in class with the support of peers and the teacher (Brame, 2013).

Nineteen of the 24 teachers had never used flipped classroom techniques with their students, three had tried them but felt unsuccessful due to technology constraints students had at home, and two teachers had more experience and were beginning to feel comfortable with using them with their students. Furthermore, 20 of the teachers had never made a screencast.

We provided each teacher with either a copy of Flipped Learning for Math Instruction or Flipped Learning for Science Instruction (Bergmann & Sams, 2015a, b) to read as homework and discuss during the summer institute. Participants received explicit instruction and practice with the use of various flipped classroom tools and were challenged to develop teaching modules for their own students to complete out of class in lieu of lecturing during class. The purpose was to allow opportunities for project-based, active learning such as the engineering challenges used in this training, in which students can apply their learning. 

Participants were trained to make their own video screencasts, where to post them, and how to hold students accountable for completing out of class work. We modeled how to create screencasts using Screencast-O-Matic (https://screencast-o-matic.com/), Screencastify (https://www.screencastify.com), and Educreations (https://www.educreations.com). We also modeled tools for making interactive videos such as Ted-Ed (https://ed.ted.com/lessons) and EdPuzzle (https://edpuzzle.com/).

Dialogue and discussion tools were shared as a means to help hold students accountable for the required work such as VoiceThread (https://voicethread.com/), Edublogs (https://edublogs.org/) and Flipgrid (https://info.flipgrid.com/). Other tools were modeled to encourage student accountability, either out of class or at the beginning of a class period, such as Google Forms to create quizzes, Kahoot (https://kahoot.com/) and Plickers (https://www.plickers.com/). Platforms introduced that could be used to host videos and organize out-of-class activities included Google Classroom (https://classroom.google.com/), TES Blendspace (https://www.tes.com/lessons), and their school’s current learning management system, if applicable. 

Classroom Pedagogy Focus: Critical Reading and Technical Writing

Eleven of the 24 teachers had limited experience teaching students critical reading and technical writing skills. Several teachers described incorporating reading particularly with problem-based learning activities, while others described regularly asking students to take Cornell notes, write lab reports, and write arguments using the claim, evidence, reasoning model. With a focus on literacy for science and technical subjects we discussed the importance of providing a variety of texts at different complexity levels to meet the needs of the differentiated learners in the middle and high school classroom.

We showed the teachers how to determine the Lexile level of texts, which is a quantitative measure based upon the complexity of sentence structure and other text features. Figure 3 includes the primary websites and sources for science and STEM-based informational texts shared with the teacher participants for use in their classrooms and as needed for the challenges they solved during the training.

Figure 3   Informational Text Sources Online and in Print

Participants engaged in technical writing for their Make Solar Energy Economical grand challenge using the typesetting language LaTeX, which was a new skill for all participants. To streamline the process of learning a new technical writing format, the participants used an online collaborative LaTeX editor called Overleaf (https://www.overleaf.com/). Overleaf allowed teams to work collaboratively on a single file using a template we provided (see Appendix B), which included the following headings: Abstract, Introduction, Model (or Method), Results, Conclusion, and References.

We created screencasts using Screencastify to show how to use LaTeX and Overleaf to write equations and construct lists, figures, tables, and a reference list. Participants accessed these screencasts working at their own pace and used them as needed to help them construct their technical writing documents (see Appendix A). The use of the template and screencasts made the process accessible to the teachers as they collaboratively wrote a technical paper to present their design challenge.

Fall Semester Follow-Up

Between the summer institute and the follow-up Saturday session, teachers were encouraged to use their National Science Teachers Association (NSTA) Learning Center subscription to locate resources for instruction and complete personalized interactive modules, called SciPacks, as a means to improve their science content and pedagogical knowledge. The NSTA Learning Center was developed using established research-based production and design procedures that ensure rigorous science content, compliance with national standards, and effective pedagogy (Dede et al., 2016).

Twenty-five different SciPacks are available in life, physical, and earth/space science, and each takes an average of 10 hours to complete. Nine teachers successfully completed and passed an assessment for a SciPack. Seven teachers added numerous articles and book chapters to their resource library that they will be able to refer to even after their subscription has expired.

During the Saturday follow-up session teachers took their postassessments, debriefed their explorations with the NSTA Learning Center, discussed troubleshooting issues they encountered with both 3D printing and robotics as they returned to the classroom, and planned conference proposals to disseminate information that they had learned. Five science teacher participants planned and presented a session, “Engineering Challenges with 3D Printing,” at the Tennessee Science Teachers Association conference, in which they described engineering challenges they participated in during the summer training and subsequent 3D printing challenges they posed for their students within their own classrooms.

Additionally, one middle school mathematics teacher participant planned and presented resources for flipped classroom tools with teachers within her school and district. One high school science teacher participant prepared a presentation for faculty at her school regarding a school-wide engineering design project, and she also shared her expertise with elementary teachers within her district seeking opportunities to integrate robotics. 

Methods

STEM Lit was designed using a mixed methodology approach of collecting qualitative and quantitative data, and a convergent parallel design was used to collect both types of data concurrently (Buchholtz, 2019; Creswell & Clark, 2017). Quantitative data was collected using a CT-based mathematics and science content assessment designed specifically for this institute by grant staff as well as the TPACK assessment (Schmidt et al., 2009). An additional quantitative source included a postinstitute evaluation form. Pre and post quantitative data were analyzed using two-sample t-tests, with the use of a Bonferroni correction to determine the statistical significance of changes.

Narrative analysis was used to reveal emergent themes within the qualitative data collected pre- and postparticipation (Patton, 1990). Participant responses to an open-ended prompt included on the TPACK survey as well as responses to a final project evaluation form and TEAM assessment both designed by project staff served as the qualitative data. The TPACK survey prompt was provided pre- and postparticipation and asked participants to describe a specific teaching episode in which they effectively demonstrated or modeled combining content, technologies, and teaching approaches in a classroom lesson.

The responses to the TPACK open-ended prompt were categorized into teacher-focused and student-focused use of technology and organized by stages of the Mathematics Teachers TPACK Developmental Model (Niess, et al., 2009) and included reference to the SAMR Model level (Puentedura, 2010), where applicable.

The final project evaluation form required participants to describe the “top three take-aways” from participation and what could have been done to improve their experiences. The responses to the project evaluation form were analyzed to search for similarities and differences between participant ideas to identify the emergent themes for top take-aways and what could have been improved. The TEAM assessment asked participants to describe three indicators positively impacted by participation. The responses were sorted by indicator and the top five indicators for the entire group were reported along with representative comments.

Findings

This section reports a comparison of teacher performance on the CT, mathematics and science pre- and postassessment, both the quantitative and qualitative results for the TPACK and the STEM Lit evaluation form, and qualitative analysis for the post-TEAM assessment form.

Content Knowledge Pre-and Postassessment

The content assessment consisted of 25 questions that addressed CT, mathematics, and science concepts. The questions were worth 4 points each, resulting in a 100-point scale. The pretest average was 44.4, with a range in scores from 16 to 88. The posttest average was 69.6, with a range from 42 to 93. Standard deviation for the pretest was 16.8 and for the posttest was 14.9. The 25.2 increase in the average scores was statistically significant at the p < 0.0001 level with a two-sample t-test. Twenty-three of the 24 participants completed both the pre and post content assessments.  

The assessment contained a mixture of question types using Weintrop’s CT in mathematics and science taxonomy as a guide (Weintrop et. al., 2016). To determine which questions had the largest contributions to the overall statistical significance, individual pre- and postassessment two-sample t-tests were performed. When a Bonferroni correction is applied to this collection of 25 individual t-tests, two were statistically significant at the 0.05 level, one at the 0.01 level, one at the 0.001 level, and four at the 0.0001 level. To outline their contributions, six of the 25 questions were selected for discussion (Appendix C). These questions were selected to illustrate the different types of questions included that are representative of Weintrop’s CT in mathematics and science categories. The significance levels reported may not show that improvement is independently statistically significant, but they do illustrate the relative contributions of each question to the overall significant result. 

Example A in Appendix C is an application problem that asks the participants to use a function for velocity to calculate acceleration. Correctly answering this problem demonstrates an understanding of rates of change and the relationship between velocity and acceleration. In Weintrop’s taxonomy this problem falls in the Modeling and Simulations Practices category under the subcategory of using computational models to find and test solutions. The pre/posttest improvement on this question increased from an average score of 1.13 to an average of 2.30 (p < 0.05).

Example B in Appendix C concerns the influence of an outlier in statistical data summaries. This problem falls under Data Practices in Weintrop’s taxonomy in the subcategories of manipulating and analyzing data. The pre/posttest averages increased from 3.13 to a perfect 4.0 (Not significant; p = 0.011).

Example C in Appendix C is another application problem that depends on the participant selecting the correct formula for distance and applying it to a real-world problem. The taxonomic category for this problem is, again, Modeling and Simulation Practices in the subcategories of constructing a computational model and using a computational model to find and test solutions. The average on this problem increased from 1.74 to 2.87 (Not significant; p = 0.003).

Example D in Appendix C requires the participant to convert LaTeX mathematical typesetting commands into a mathematical equation. This problem requires the participants to think more abstractly about mathematical notation and is addressed under Computational Problem Solving Practices, in the subcategory of programming and creating computational abstractions. The average on this problem improved from 0.83 to 3.04 (p < 0.0001).

Example E in Appendix C asks the participants to determine the result of a Lego Mindstorms graphical program. In Weintrop’s taxonomy this task falls under both Computational Problem Solving Practices and Systems Thinking Practices, because it requires the participant to understand programming and understand the relationships within a system (the interaction of the software with the hardware of the robot). The average scores on this problem improved from 0.35 to 2.17 (p < 0.0001).

Example F in Appendix C involves determining how an equation must be modified to reflect a horizontal shift. This problem involves two of Weintrop’s categories: Computational Problem Solving Practices (developing modular computational solutions and creating computational abstractions) and Modeling and Simulation Practices (designing computational models). The average scores on this problem increased from 0.78 to 2.52 (p < 0.0001).

TPACK Survey

TPACK Quantitative Results

The TPACK assessment included 32 Likert-scale items divided into categories taken from the Survey of Preservice Teachers’ Knowledge of Teaching and Technology (Schmidt et al., 2009). Each item response is scored with a value from 1 (strongly disagree) to 5 (strongly agree). Each participant’s responses were averaged over all 32 questions. Additionally, each participant’s responses were averaged over each construct. For example, the six questions under TK were averaged to produce one score. A two-sample t-test was computed for the participant’s average responses over all the questions to show a significant change (p = 0.0018). To determine the individual contributions, separate two-sample t-tests were performed on each construct.

Once a Bonferroni correction was imposed, four of the seven constructs showed a statistically significant increase at the p < 0.05 level, including TK, CK, TCK, and TPK. Table 1 includes the participant average results for pre- and post-TPACK and standard deviation, along with the p-value to help distinguish the contribution of each construct to the overall statistical significance. Twenty-two of the 24 participants completed both the pre- and post-TPACK assessment.

Table 1  Pre- and Post-TPACK Assessment Results

TPACK Qualitative Results

The participants were asked to describe a specific teaching episode in which they effectively demonstrated or modeled combining content, technologies, and teaching approaches in a classroom lesson. On the preassessment, participants responded in the following ways arranged in order from teacher-centered to student centered and lower cognitive demand to higher: unable to describe a teaching episode using technology (n = 3); teacher focused use of technology for instruction or assessment (n = 5); student use of internet (n = 4); student use of robotics or 3D printers in STEM classroom that was not focused on disciplinary content (n = 4); student use of subject-specific simulations (n = 3); student use of technology to construct models (n = 1); student use of technology to collect data with probe software (n = 1); and student creation of videos to explain concepts (n = 1).

On the postassessment nine participants responded as follows: no use of technology at this point in school year with students, although two had modeled STEM Lit tools for colleagues (n = 5); student use of coding that was not focused on disciplinary content (n = 1); student use of internet for research (n = 1); and use of virtual manipulatives or simulations (n = 2). Responses from the remaining teachers (n = 13) for the postassessment are included in Table 2 as representatives of how the teachers started using 3D printing and design, flipped classroom techniques, and robotics in their classrooms. The responses are organized by the five stages (Recognizing, Accepting, Adapting, Exploring, and Evaluating) of the Mathematics Teachers TPACK Developmental Model (Niess et al., 2009) and refer to a SAMR Model level (Puentedura, 2010) where applicable.

Table 2 TPACK Postassessment Response Analysis: TPACK Stage and SAMR Model Level

STEM Lit Postevaluation Form

Thirteen Likert-scale questions were asked to determine participant perceptions regarding specific practices and tools emphasized during the institute. Two open-ended questions were asked to determine three take-aways gained from their experience and what could have improved their experience. Twenty-one out of the 24 participants completed this assessment. Each Likert-scale item response was scored from 1 (strongly disagree) to 5 (strongly agree). The first 11 prompts started with, “I feel that I am better prepared to…”. The prompts and corresponding average result follow.

1. Use the Science and Engineering Practices as outlined by the K-12 Framework for Science Education. (4.24)
2. Teach using TEAM pedagogy. (4.05)
3. Collect and analyze data. (4.10)
4. Teach children how to collect and analyze data. (4.05)
5. Use technical reading and writing strategies. (3.81)
6. Teach students to use technical reading and writing strategies. (3.67)
7. Use critical thinking and problem-solving activities in my classroom. (4.29)
8. Implement differentiated instruction methods for my classroom. (4.05)
9. Use computer programming and robotics aligned with science/STEM in my classroom. (4.19)
10. Use 3D printing and modeling aligned with science/STEM in my classroom. (4.33)
11. Plan flipped classroom modules for my instruction. (4.14)
12. The NSTA Learning Center was a useful tool for my professional development. (4.05)
13. I would participate in another STEM Literacy project and recommend it to others. (4.86)

Five themes emerged as top takeaways from the 21 responses received, including 3D printing and use of technology tools, flipped classroom tools and strategies, knowledge of and increased comfort with using and coding/programming robotics, engineering challenges and problem-solving skills, and collaboration with other teachers. Table 3 includes representative quotes from teachers for each isolated theme. Each quote is labeled by subject/grade level.

Table 3   Top Takeaways From STEM Lit

Two themes emerged as suggested improvements, including time constraints and structure/organization. Twelve teachers described the need for more time and suggestions for improvement. Within the time category, four teachers referred specifically to flipped classroom tools. An eighth-grade science teacher suggested, “Additional practice time slots for the various websites. Perhaps, have the ‘steps’ for using the website and devices listed on the grant wiki.”An eighth-grade mathematics teacher suggested, “Take more time to allow us to explore website by website and not overwhelm us by showing us several at the same time.” 

Four teachers stated that they felt it was a lot of information to process in 8 days and they could have used more time during the summer institute. Additional teachers stated they wanted more time to analyze Mindstorm data (HS Science), more time to practice with the 3D printers (MS STEM), and more time for or a slow pace with the mathematics content sessions (sixth-grade science).

Four teachers suggested providing more structure/organization to meet their needs, including the use of breakout sessions by topic, the use of a timetable or rubric for the engineering challenges, and more detail about how to code the Lego Mindstorms EV3 robots. One high school science teacher requested more information about how to transfer the ideas for use in the classroom:

I think we should have had one major project with the robots and printer and then spent the rest of the time discussing how to best incorporate the tools in our classrooms. I enjoyed the projects we did and really got into them, but I am still uncertain as how to best employ them in my classroom with students whose computer skills are severely limited for being “digital natives.”

TEAM Postassessment

The TEAM Instruction rubric is used by supervisors of instruction in the state of Tennessee to evaluate teacher instruction. The teachers were asked to select three indicators out of 12 possible for which they felt they made positive changes as a result of grant participation. Table 4 includes representative quotes of the five most mentioned indicators. Note that 22 of the 24 participants completed the TEAM postassessment.

Table 4    TEAM Indicator Improvement Coding Connections

Discussion

The participants showed a statistically significant increase in both performance on the mathematics, science, and CT content assessment and their self-reported confidence ratings on the TPACK assessment, with four of the seven individual constructs showing a statistically significant increase as well. By mid-September at our fall semester follow-up session, almost half of the teachers (n = 10) described examples on the TPACK postassessment at an advanced stage of the TPACK Developmental Model, exploring teaching and learning with 3D printing, flipped instruction, or robotics at the Exploring or Implementation stage. This accomplishment was significant considering that the school year was just getting started and that the majority of these teachers had never used the tools and strategies for instruction previously.

In looking at common themes across qualitative assessments, successful aspects of STEM Lit included an increased understanding and use of 3D printing tools and modeling, flipped classroom instruction and technology tools, in general, and programming/coding with EV3 Mindstorms robotics. Participants stated that they felt better prepared to teach using TEAM pedagogy.

The participants did not describe specific examples of using technical reading and writing in the qualitative postassessments; however, several described assigning students to read to prepare for challenges they were working on. Teachers primarily focused on the use of 3D printing, flipped classroom strategies, and robotics as articulated in postreflections.

In addition, while their self-reported ratings on the two associated prompts on the STEM Lit postevaluation form showed growth, they had the lowest averages, including “I feel that I am better prepared to use technical reading and writing strategies,” with an average of 3.81/5, and “I feel that I am better prepared to teach students to use technical reading and writing strategies,” with an average of 3.67/5. Over half of the teachers (n = 13) claimed that they had already been using technical reading and writing in their classrooms prior to grant participation according to the preliminary needs assessment.

While the teachers were impressed with using the LaTeX template in Overleaf and enjoyed the process of writing their team reports in this manner, none of the participants described using this format with students by mid-September. Even though the importance of using informational texts and having students communicate their results through oral and written discourse were explored, the bulk of time was focused on developing skills with coding, robotics, 3D printing, and solving real-world challenges with these tools.   

Implications for Teacher Education

Our institute design suggests a number of specific recommendations for teacher education. The 2-week, 8-day summer institute was appropriate timing to allow for teachers to transfer use of 3D printing and flipped instructional strategies to the classroom. They shared a number of specific examples of classroom use in their postassessments. Although the teachers increased their knowledge of programming robots and ability to use them in solving challenges, they received only one robot to use in their classroom. One robot is difficult to share with a group of 30 students; however, students can effectively use and share one 3D printer. Additionally, the teachers suggested more time for learning how to use the robots and 3D printers as part of feedback.

Many of the teachers were better prepared to create screencasts and use web-based tools to interact with students inside and out of the class, which is helpful to allow for more time with problem-based challenges as explored during this institute. While the teachers used technical reading and writing with the use of an Overleaf LaTeX template during the institute, precedence was given to exploring the use of the hardware provided to the teachers rather than literacy skills as originally intended. 

As suggested for most PD experiences, extended time for training with several summer institutes, funding for more supplies, and follow-up during the school year would provide a better understanding of teacher use of these tools and strategies as well as transfer to students in the classroom (CSSS, 2018). Some STEM-specific suggestions include the need to develop disciplinary CT skills for teachers from different subject areas as they work together to solve problem-based challenges. Developing these skills can help address the need to prepare teachers in high-need STEM disciplines to address the future needs for CS skills in mathematics and science classes rather than only in CS classes (Marder et al., 2017).

Teacher participants can increase the sustainability of trainings such as these by sharing their knowledge within districts, using peer coaching and new skills such as screencasting learned during this institute to document and model how to use coding, robotics, and 3D printing tools for instruction. Virtual robots that replicate the environment of Lego Mindstorms are a good alternative for schools with limited funding. Virtual robots allow users to create settings for and program robots without having access to the expensive hardware. Robot Virtual Worlds (http://www.robotvirtualworlds.com/) and Virtual Robotics Toolkit (https://www.virtualroboticstoolkit.com/), in particular offer perpetual licenses for a set number of students that can be used year after year at much lower costs than the kits. The hands-on kits provide a certain motivation factor to students. We recommend having at least one kit on hand for groups of students to rotate using while the majority of the class programs and interacts with virtual robots.

We propose an innovative approach to teaching disciplinary CT and expansion of the SAMR model to emphasize the use of real-world challenges at a new Discovery level that can be used for mathematics and science PD and classroom instruction modeled in Figure 5.

Figure 5   STEM Lit and SAMR Levels.

This approach starts with opportunities to explore programming in general with tools such as Lightbot (https://lightbot.com/) (A) as a substitution for unplugged programming which can help students learn basic algorithm design and function calls. There are any number of online tools that allow for this type of programming such as those located at hourofcode.com. This introductory coding could then be augmented with visual coding (B) through the Lego Mindstorms EV3 robotics visual coding environment with more complex blocks and capabilities, which is what the teachers were able to explore as part of STEM Lit. Where (A) moves a virtual robot, (B) moves a robot in the world. Python is introduced (C) that matches all the capabilities of (B) but then opens up a new world of task design (Suters & Suters, 2020).

The final robot (D), as redefined by the participant to solve a challenge, was inconceivably difficult to bring to task until after (A)-(C). We propose the next step is to complete challenges similar to the National Academy of Engineering Grand Challenges (NAE, 2020). For example, the grand challenge, “Restore and Improve Urban Infrastructure,” as depicted by (E) represents our modification of the SAMR model to include an additional level, a challenge. At this level, in the context of our training, participants were provided an opportunity to explore two grand challenges, including the clean water challenge and the make solar energy economical challenge incorporating the use of designing, troubleshooting, and programming robots, as well as 3D printing design to help discover solutions to these real-world challenges. This use of the SAMR model takes into consideration critiques of the model by using technology in contexts to support learning goals and desired outcomes (Hamilton et al., 2016).

This type of thinking and learning is reflective of what is needed in the changing nature of careers in mathematics and science taking place today (Schwab & Davis, 2018). Using applied CT practices during middle and high school mathematics and science classes will help answer students’ question, “When will I ever use this?” These skills will give students a more realistic view of what is required in professional fields such as bioinformatics, chemometrics, computational statistics, cybersecurity, and climate research (Qin, 2009; Weintrop et al., 2016). In alignment with recommendations for effective STEM PD, the group field trip to Oak Ridge National Laboratory’s Manufacturing Demonstration Facility plant to see large scale examples of 3D printing and robotics was a direct means to help teachers prepare students to see applications of how their current educational experiences can lead them to future career paths (Bush et al., 2020).

Conclusions

STEM Lit addressed the need for high-quality PD in STEM with a goal of improving content and pedagogical strategies in the context of engineerable societal issues. The PD intentionally advanced research on the meaning of CT within secondary mathematics and science classrooms by including the use of innovative cyber technologies (robotics, programming, 3D printing, Web 2.0 tools for flipped classroom instruction, etc.) and by using interdisciplinary approaches incorporating technical reading and writing to solve grand challenges in the classroom. Participation in this project prepared teachers to begin using CT practices within their respective disciplines as well as provided necessary knowledge to transform their instruction over time. Future explorations with these types of learning opportunities should provide for sustained teacher experiences, allow for opportunities for supported transfer to the classroom, and opportunities to observe enacted curriculum in participants’ classrooms.

Author Note

This material is based upon work supported by an Improving Teacher Quality Grant from the Tennessee Higher Education Commission under Grant No. 33201-09317 FY 17-18.

References

Asunda, P. A. (2018). Infusing computer science in engineering and technology education. The Journal of Technology Studies, 44(1), 2-13.

Autodesk, Inc. (2019). Tinkercad: From mind to design in minutes. https://www.tinkercad.com/

Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? Inroads, 2(1), 48-54.

Bergmann, J., & Sams, A. (2015). Flipped learning for math instruction (Vol. 2). International Society for Technology in Education.

Brame, C. (2013). Flipping the classroom. Vanderbilt University Center for Teaching. Vanderbilt. http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/.

Buchholtz, N. (2019). Planning and conducting mixed methods studies in mathematics educational research. In G. Kaiser & N. Presmeg (Eds.), Compendium for early career researchers in mathematics education (pp. 131-152). Springer.

Bush, S. B., Cook, K. L., Ronau, R. N., Mohr-Schroeder, M. J., Rakes, C. R., & Saderholm, J. (2020). Structuring Integrated STEM education professional development: Challenges revealed and insights gained from a cross-case synthesis. Electronic Journal of Science Education, 24(1).

Chambers Cantrell, S., David Burns, L., & Callaway, P. (2008). Middle-and high-school content area teachers’ perceptions about literacy teaching and learning. Literacy Research and Instruction, 48(1), 76-94.

Ciochetti, D. A., & Metcalf, R. H. (1984). Pasteurization of naturally contaminated water with solar energy. Applied and Environmental Microbiology, 47(2), 223-228.

Computer Science Teachers Association. (2016). K–12 computer science framework. http://www.k12cs.org

Council of Chief State School Officers. (2017). Supplemental information for Appendix A of the Common Core State Standards for English language arts and literacy: New research on text complexity. ERIC Clearinghouse.

Council of State Science Supervisors. (2018). Science professional learning standards rubric 2018. http://cosss.org/Professional-Learning

Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage.

Dede, C., Eisenkraft, A., Frumin, K., & Hartley, A. (Eds.). (2016). Teacher learning in the digital age: Online professional development in STEM education. Harvard Education Press.

Du, W., Liu, D., Johnson, C. C., Sondergeld, T. A., Bolshakova, V. L., & Moore, T. J. (2019). The impact of integrated STEM professional development on teacher quality. School Science and Mathematics, 119(2), 105-114.

Fine, J., Zygouris-Coe, V., Senokossoff, G., & Fang, Z. (2011). Secondary teachers’ knowledge, beliefs, and self-efficacy to teach reading in the content areas: Voices following professional development. In M. S. Plakhotnik, S. M. Nielsen, & D. M. Pane (Eds.), Proceedings of the 10th Annual College of Education & GSN research conference (pp. 24-28). Florida International University. http://coeweb.fiu.edu/research_conference/

Frey, C. B., & Osborne, M. A. (2017). The future of employment: How susceptible are jobs to computerisation?. Technological Forecasting and Social Change, 114, 254-280.

Gorle, P., & Clive, A. (2013). Positive impact of industrial robots on employment. IFR International Federation of Robotics, Metra Martech. https://robohub.org/wp-content/uploads/2013/04/Metra_Martech_Study_on_robots_2013.pdf

Grover, S., & Pea, R. (2013). Computational thinking in K–12 a review of the state of the field. Educational Researcher, 42(1), 38–43.

Hamilton, E. R., Rosenberg, J. M., & Akcaoglu, M. (2016). The substitution augmentation modification redefinition (SAMR) model: A critical review and suggestions for its use. TechTrends, 60(5), 433-441.

International Society for Technology in Education & Computer Science Teachers Association. (2011). Operational definition of computational thinking for K–12 education. https://id.iste.org/docs/ct-documents/computational-thinking-operationaldefinition-flyer.pdf  

Kale, U., Akcaoglu, M., Cullen, T., Goh, D., Devine, L., Calvert, N., & Grise, K. (2018). Computational what? Relating computational thinking to teaching. TechTrends, 62(6), 574-584.

Lipson, H., & Kurman, M. (2013). Fabricated: The new world of 3D printing. John Wiley & Sons.

Loucks-Horsley, S., Stiles, K., & Hewson, P. (1996). Principles of effective professional development for mathematics and science education: A synthesis of standards. NISE Brief, 1(1), n1.

Makerbot Industries. (2019). Thingiverse: Digital designs for physical objects. https://www.thingiverse.com/

Marder, M., Brown, R. C., & Plisch, M. (2017). Recruiting teachers in high-needs STEM Fields: A survey of current majors and recent STEM graduates (Executive summary). American Physical Society Panel on Public Affairs.  https://www.aps.org/policy/reports/popa-reports/stemteachers.cfm

Marr, B. (2019, May 22). 8 things every school must do to prepare for the 4th industrial revolution. Forbes. https://www.forbes.com/sites/bernardmarr/2019/05/22/8-things-every-school-must-do-to-prepare-for-the-4th-industrial-revolution/#5be2d284670c

Mishra, P. & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.

Mouza, C., Yang, H., Pan, Y. C., Ozden, S. Y., & Pollock, L. (2017). Resetting educational technology coursework for pre-service teachers: A computational thinking approach to the development of technological pedagogical content knowledge (TPACK). Australasian Journal of Educational Technology, 33(3), 61-76.

National Academy of Engineering. (2020). NAE grand challenges for engineering. http://www.engineeringchallenges.org/challenges.aspx

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010a). Common core state standards for English language arts and literacy in history/social studies, science, and technical subjects.  Author.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010b). Common core state standards for mathematics. Authors.

National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.

National Research Council. (2013). Next generation science standards: For states, by states. The National Academies Press. https://doi.org/10.17226/18290

Nearpod. (n.d.). Bringing engagement and joy to every classroom. https://nearpod.com/

Niess, M. L., Ronau, R. N., Shafer, K. G., Driskell, S. O., Harper, S. R., Johnston, C., Browning, C., Ozgun-Koca, & Kersaint, G. (2009). Mathematics teacher TPACK standards and development model. Contemporary Issues in Technology and Teacher Education, 9(1), 4-24. https://citejournal.org/volume-9/issue-1-09/mathematics/mathematics-teacher-tpack-standards-and-development-model

ORNL. (n.d.). Advanced Manufacturing. https://www.ornl.gov/advancedmanufacturing

Patton, M. Q. (1990). Qualitative evaluation and research methods. SAGE Publications, Inc.

Puentedura, R. (2010). SAMR and TPCK: Intro to advanced practice. http://hippasus.com/resources/sweden2010/SAMR_TPCK_IntroToAdvancedPractice.pdf

Qin, H. (2009, March). Teaching computational thinking through bioinformatics to biology students. ACM SIGCSE Bulletin, 41(1), 188-191.

Ritchie, H., & Roser, M. (2018, July). Energy. https://ourworldindata.org/energy

Runde, D. F., Bandura, R., Campbell-Zausner, C., Carter, W., Hammond, M., Jorge, S., Klynge, C., Milner, A., Nakagaki, M., Raymond, P., Rubin, N., & Zausner, S. (2019). Beyond technology: The fourth industrial revolution in the developing world. Center for Strategic & International Studies. https://www.csis.org/analysis/beyond-technology-fourth-industrial-revolution-developing-world

Sands, P., Yadav, A., & Good, J. (2018). Computational thinking in K-12: In-service teacher perceptions of computational thinking. In M. Khine (Ed.), Computational thinking in the STEM disciplines (pp. 151-164). Springer International Publishing.

Schmidt, D. A., Baran, E., Thompson, A. D., Koehler, M. J., Mishra, P., & Shin, T. (2009). Survey of preservice teachers’ knowledge of teaching and technology. Récupéré le, 2.

Schwab, K., & Davis, N. (2018). Shaping the future of the fourth industrial revolution. Currency.

Spencer, M., & Bouwma-Gearhart, J. (2014). Reading strategies for secondary science teachers. Academic Exchange Quarterly, 18(3), 50-60.

Suters, L., & Suters, H. (2020). Coding for the Core: Computational thinking and middle grades mathematics. Contemporary Issues in Technology and Teacher Education, 20(3), 435-471. https://citejournal.org/volume-20/issue-3-20/mathematics/coding-for-the-core-computational-thinking-and-middle-grades-mathematics

Thibaut, L., Knipprath, H., Dehaene, W., & Depaepe, F. (2019). Teachers’ attitudes toward teaching integrated stem: The impact of personal background characteristics and school context. International Journal of Science and Mathematics Education, 17(5), 987-1007.

TN Department of Education. (n.d.). TEAM: Tennessee educator acceleration model. https://team-tn.org/

Weintrop, D., Beheshti, E. Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147.

XYZprinting Inc. (2018). Da Vinci Jr. 1.0. https://www.xyzprinting.com/en-US/product/da-vinci-jr-1-0

Yadav, A., Hong, H., & Stephenson, C. (2016). Computational thinking for all: Pedagogical approaches to embedding 21st century problem solving in K-12 classrooms. TechTrends, 60(6), 565-568.

Yore, L. D., Anderson, J. O., & Shymansky, J. A. (2005). Sensing the impact of elementary school science reform: A study of stakeholder perceptions of implementation, constructivist strategies, and school–home collaboration. Journal of Science Teacher Education, 16(1), 65-88.

Appendix A
Maximizing Voltage Output From Lego MindStorm Solar Kit

Appendix B
Overleaf Report Template

Appendix C
Sample Content Test Items