ChatGPT, representing a significant advancement in AI (Wu et al., 2023), has been available to the public since 2022. Powered by deep learning techniques and neural networks, ChatGPT, an AI language model, can understand and analyze data consolidated from multiple sources and produce human-like text and multimodal content, such as images and music (Dasborough, 2023; Fui-Hoon Nah, 2023; Ray, 2023). This technological revolution has transformed the way people work, study, and communicate with each other and with machines (Long & Magerko, 2020).
Likewise, it significantly impacts different fields and industries, including education (e.g., Baidoo-Anu & Ansah, 2023; Chiu et al., 2023). The scholarly articles on AI and ChatGPT and its impact on teaching and learning have been growing over the past years. Much research has primarily discussed the use of ChatGPT in different disciplines (e.g., literacy and language, as well as science, technology, engineering, and mathematics [STEM]) in terms of opportunities, challenges, and limitations (e.g., Chiu et al., 2023; Memarian & Doleck, 2023; Vogt & Flindt, 2023).
For example, Grassini (2023) illustrated that ChatGPT offered various potential benefits in teaching and learning. AI tools had capabilities that went beyond grading short answers and providing students with valuable teaching aids; they could also be used for translating educational materials and creating interactive, adaptive learning environments. Meanwhile, Baidoo-Anu and Ansah (2023) proposed possible drawbacks of using GenAI, like ChatGPT, which included lack of human interaction and contextual understanding, bias in the training data, and limited ability to personalize instruction.
Several other scholarly studies have addressed how researchers, practitioners, and educators should act upon these transformative changes in teaching and learning (e.g., Baskara & Mukarto, 2023; Gamage et al., 2023; Jauhiainen & Guerra, 2023). For instance, Gamage et al. proposed some innovative assessment tools to evaluate students’ performance instead of using the traditional written assessment, an area that might be most affected by ChatGPT. Jauhiainen and Guerra argued for the active integration of ChatGPT to tailor learning resources, thereby meeting the needs of learners with varying levels of knowledge and skills. Although incorporating AI in education holds great potential, there has been controversy over its use in teaching. From an optimistic viewpoint, supported by some initial evidence, AI could help reduce teachers’ workloads (Chan & Hu, 2023); On the contrary, concerns have also been raised that AI could potentially replace teaching professions (Alam, 2021).
Research on AI and education is still emerging, with most studies being conceptual, thus highlighting the need for more empirical research to explore this vast landscape of untapped potential (Zhang & Aslan, 2021). Hence, this study aimed to examine the use of GenAI-ChatGPT in the context of a teacher education program, particularly focusing on how preservice teachers (PSTs) developed their unit plans, a major learning task in a literacy course, with the assistance of ChatGPT.
Purpose of the Study
This qualitative exploratory study investigated how PSTs in a teacher education program utilized ChatGPT for developing unit plans. Two objectives of this study were as follows: (a) to understand how ChatGPT can be used intentionally to assist PSTs in constructing unit plans and (b) to explore factors that might shape the ways that PSTs’ use of ChatGPT, especially the ways they evaluated the ChatGPT output. Two following research questions (RQs) guided this qualitative research:
RQ1: How do preservice teachers make use of ChatGPT to support their unit plan design to teach reading to secondary students?
RQ2: What factors do preservice teachers consider when evaluating the output generated by ChatGPT?
Conceptual Framework
New technologies, such as the internet and AI, are constantly evolving, becoming integral to everyday lives, and have profoundly changed our practice in teaching and learning (Blankenship et al., 2022; Leu et al., 2017). To respond to this change, teacher preparation programs need to equip future educators with a solid knowledge of leveraging technology to support learning (Office of Educational Technology, 2023).
This study is informed by the technology, pedagogy, and content knowledge (TPACK) framework (Mishra & Koehler, 2006), which details how technology knowledge (TK), pedagogical knowledge (PK), and content knowledge (CK) are required for educators to integrate technology in lesson design effectively. Technology knowledge (TK) can be understood as ways of thinking about and working with technology in practice. Pedagogical knowledge (PK) is teachers’ knowledge about the processes, practices, and methods to facilitate learning. Content knowledge (CK) refers to teachers’ knowledge about the subject matter to be taught (Koehler & Mishra, 2009).
The significance of the TPACK framework emphasizes interactions and intersections of these three components — content domain, pedagogical strategies, and technological knowledge, and how these three areas collaboratively contribute to achieving a high level of educational outcomes (Mupita et al., 2018). Therefore, pedagogical content knowledge (PCK) is the knowledge of the teaching approach that is appropriate for teaching specific content. Technological content knowledge (TCK) refers to the interrelationship between learning content and technology, while technological pedagogical knowledge (TPK) pertains to the interactions between technology and pedagogy.
Technological pedagogical content knowledge (also referred to as technology, pedagogy, and content knowledge, or TPACK) is situated at the intersection of three key elements of TK, PK, and CK, which foregrounds the importance of integrating technology with effective teaching strategies for content learning (Koehler & Mishra, 2009; Koehler et al., 2014). In addition, Mishra (2019) revised the TPACK diagram, relabeling “contexts” as “ConteXtual Knowledge (XK),” which considers “organizational and situational constraints that teachers work within” (p. 77). The three key overlapping components (TK, PK, and CK), along the newly defined XK, collectively represent the knowledge that educators need to acquire in the digital age.
TPACK has been a widely used framework across a large number of teacher education studies (e.g., Karlsson & Nilsson, 2023; Tondeur et al., 2020; Weidlich & Kalz, 2023; Wen & Shinas, 2020), which primarily aims to prepare educators with the knowledge and abilities needed to integrate effective pedagogies with new technologies for specific content teaching and learning in classrooms (Harris & Hofer, 2011; Sheffield et al., 2015; Voogt & McKenney, 2017). Moreover, as a useful conceptual perspective, TPACK can be regarded as an analytical tool to examine what forms of technology educators need to utilize effectively to achieve any purpose, including supporting their own teaching planning processes (Mishra et al., 2023).
Hence, this qualitative study used TPACK as an analytical lens to explore how PSTs made use of emerging technology (ChatGPT), as well as leveraging their understanding of subject matter (content knowledge) and teaching strategies (pedagogical knowledge) to develop their unit plans. This approach uncovered how these PSTs integrated the components of the TPACK framework to enhance their instructional design.
The TPACK framework not only shaped the research direction of this study but also served as a lens through which the literature was explored. The literature reviewed in the next section identifies previous research on teacher education, AI technologies, and the application of TPACK, all of which align with the study’s goals and conceptual framework.
Literature Review
First, the literature for this study was based on previous research regarding the broad impact of AI on education. Recognizing the importance of this advancement, the U.S. Department of Education addressed the clear need to promote the use of technology to improve teaching and learning, as well as to support innovations in the fields. There is a critical need for K-12 teachers to be involved in learning, designing, and using innovative technologies such as AI to support their teaching further so they may better engage and support their students (Office of Educational Technology, 2023). Within this broad context, both pre- and in-service teachers, in the fast-evolving world of technology, need to understand how technologies change the educational ecosystem, that is, how AI technologies can support pedagogical processes for both content teaching and instructional design (Mishra et al., 2023).
For instance, Trust et al. (2023) explored the challenges and opportunities presented by ChatGPT and its implications for teacher education. One example was that ChatGPT was able to assist in creating lesson plans and learning activities, as well as developing classroom management strategies. Beyond these practical day-to-day teaching activities within a classroom, ChatGPT has also been acknowledged as an excellent resource to support curriculum design and instructional planning for educators (e.g., KV, 2023; van den Berg & du Plessis, 2023; Young, 2023).
Given the benefits that AI can bring to education, questions arise regarding what knowledge and skills are required to support educators in integrating AI into their teaching practices. To delve deeper into this subject, our study was also informed by a body of literature that highlights the importance of understanding how TPACK framework (Mishra & Koehler, 2006) plays a role in AI and teacher education. Many studies have suggested that in teacher education programs, TPACK-integrated, AI-focused training significantly improved teachers’ ability to work with AI (Kim, 2024; Kim & Kwon, 2023; Yue et al., 2024).
For example, Kim (2024) argued that preservice teachers showed the most substantial improvements in PCK and TPACK from an AI-convergence TPACK educational program. Another study conducted by Kim and Kwon (2023) maintained that identifying AI competencies within the TPACK framework could strengthen the theoretical foundation and provide a more systematic approach to K–12 AI education, particularly when offering professional development (PD) for schoolteachers.
Although the TPACK framework offers guidance in teacher education and PD programs when preparing future teachers with AI competencies, nuances also needed to be explored regarding how each component of TPACK interacts and contributes to the effective incorporation of AI. Celik (2023) emphasized that while teachers’ TK was crucial, it was not sufficient on its own. Teachers also need to understand the pedagogical affordances and ethical considerations of AI to fulfill their orchestrator role in AI-based instruction. Similarly, Hava and Babayiğit’s (2024) study indicated that in-service teachers may have limited knowledge regarding the pedagogical potential of AI applications, which could negatively impact their integration of AI-based tools into the educational process.
After reviewing the existing literature, it is evident that interest in the integration of AI in education, particularly in relation to TPACK and teacher education, is growing. However, there is a lack of empirical studies exploring how educators make use of GenAI, such as ChatGPT, by leveraging their TPACK during the instructional design process and how they respond to AI-generated products. Therefore, there is an urgent need to better understand the affordances and constraints of GenAI, like ChatGPT, its role in curriculum design, and the ways prospective educators evaluate and integrate AI-generated output. The insights gleaned from such research can be integrated into the teaching resources and curricula in the training of future educators at the higher education level.
Methods
Context and Participant
This exploratory inquiry was situated in a literacy course that prepared PSTs pursuing dual certification in early childhood education and special education at a small, private university in the northeast region of the United States. Data from 16 PSTs enrolled in the course were collected and analyzed.
The course focused on exploring the reasons behind and finding solutions for secondary students who faced reading difficulties and those who were reluctant to engage in reading. Throughout the course, PSTs examined the impact of social and political factors on literacy for secondary schoolers. PSTs were required to participate in a series of experiential activities to understand fully the sources of reading difficulties and aliteracy. Ultimately, they designed unit plans to teach literacy for those adolescents.
The Unit Plan Design Task
The capstone project of the course was for the participants to pair with one of their peers and create a GenAI-supported unit plan using high-interest adolescent literature for a group of disengaged and unmotivated readers in the secondary setting. The goal of this project was for the PSTs to apply their learning of the course content by designing unit plans to teach reading to secondary students with the support of GenAI. Participants were asked to meet online via the video conferencing platform Zoom to complete their unit plans. The whole design process and conversations were recorded by Zoom with the camera off.
The task was designed as a collaborative project for the following reasons:
- Peer support is widely used to engage learners, especially those with more learning needs. This assignment was to create authentic experiences for the PSTs so that they could create such an experience for their future school-aged students (as in Bond & Castagnera, 2006).
- Given the new AI technology that may present unforeseeable challenges, such collaborative work would provide not only cognitive but also emotional support for the participants (according to Wen, 2023).
- Conversation during collaborative work can better elicit participants’ thinking, thus allowing us to capture participants’ moments of thought and examine their skills and perceptions toward this new technology (as in Biasutti & Frate, 2018).
The unit plan was required to include lesson plans for five consecutive days. Before PSTs delved into designing unit plans, a mini lecture was offered by the course professor to introduce the basics of the application of ChatGPT in instructional design. Besides, another guideline was provided to the PSTs as a cognitive scaffold, which consisted of two major parts: Part I specified the overall goals guided by the topic/content of the chosen young adolescent literature and the relevant academic and English language arts (ELA) standards; Part II showcased the scope and sequence of the 5-day lessons in a table followed by three featured lessons. In addition, PSTs were required to detail and reflect on the processes of how their unit plans were created through the assistance of GenAI-ChatGPT by writing reflections. PSTs could also choose to opt out using AI in unit plan design, but the reflection was required.
Data Collection
Data collected for this study from multiple sources, including (a) eight pairs of PSTs’ 5-day unit plan artifacts, (b) 16 PSTs’ written reflections, (c) four pairs of PSTs’ Zoom video recordings of their collaborative processes, and (d) a post-hoc focus group online interview. The unit plan consisted of two parts. Part I was the introduction of teaching content and targeted learners. Part II detailed unit/lesson objectives, big ideas, scope, sequence, content, teaching activities and strategies, and assessment.
The written reflection asked PSTs to reflect on unit design and the processes of using or not using GenAI-ChatGPT. The Zoom video recordings captured paired PSTs’ conversations, showcasing both their thinking process and actions of developing unit plans. The interview questions were about their processes and considerations when interacting with ChatGPT.
Data Analysis
This study employed a Thematic Analysis (TA) approach (Braun & Clark, 2006; Maguire & Delahunt, 2017) with both deductive (Hsieh & Shannon, 2005; McKibben et al., 2020) and inductive coding strategy (Saldaña, 2015) to analyze and interpret the data. Braun and Clarke defined TA as “a method for identifying, analyzing, and reporting patterns (themes) within data” (p. 79), and used themes to represent some patterned meaning within data, which helped us to understand different dimensions/aspects of a phenomenon.
Before the data analysis was conducted, Zoom automatically transcribed recordings of participants’ collaborative processes and interviews. These transcriptions were then corrected and verified through member-checking. The transcripts then underwent further in-depth analysis, and all data sources were organized around two questions. To respond to RQ1, data analysis was drawn from participants’ written reflections and video recordings that captured participants’ conversations and actions during the processes of unit plan design. Data collected from students’ reflective writings and post-hoc focus group interviews were analyzed to address RQ2.
Processes of data analysis were iterative, starting with two researchers individually familiarizing themselves with the data, including reading, rereading the data, and documenting initial ideas. Then, two researchers met to exchange emerging insights and create the initial codebook consisting of a set of priori codes (Bazeley, 2013), which derived from the components of the TPACK framework (Mishra & Koehler, 2006).
After the initial codebook had been created, two researchers reread the data, especially by viewing PSTs’ unit plans and written reflections and observing their Zoom videos of interacting with ChatGPT. During this stage, another set of inductive codes emerged from the iteration of viewing and analyzing data. Subsequently, two researchers met multiple times to align PSTs’ collaborative processes with their constructed unit plans, written reflections, and interviews. Two researchers discussed how codes could be grouped, regrouped, collapsed (or expanded in some cases), and refined to understand how PSTs utilized ChatGPT and other resources to support unit plan development (as in Hsieh & Shannon, 2005). Disagreements on codes were discussed and resolved, and common codes were identified. Researcher memos were recorded after each meeting for future reference.
When completing multiple rounds of inductive coding, 15 codes were identified. They were grouped into broader categories and then compared to surface underlying relationships guided by the TPACK framework. Ultimately, themes emerged and were finalized through an iterative coding process that reached saturation across all data sources. This process included systematic coding and categorization, along with analytical reflections from the researchers (Saldaña, 2015).
The deductive coding process was guided by TPACK framework, which aligned with inductive categories/codes identified in the study. PSTs’ engagement with ChatGPT was driven by their knowledge regarding CK, TK, PK, and the interaction between them. This set of knowledge provided information, context, and understanding, enabling PSTs to take actions with ChatGPT and informing their decision about the output of ChatGPT. For example, PCK, as one framework-driven code, was used to interpret the reason why one pair of PSTs deleted the reading activity of “reading 8 chapters within 15 minutes” during their evaluation.
The appendix shows a codebook excerpt that shows the inductive codes capturing PSTs’ processes of engaging with ChatGPT. Codes and categories were informed and defined by observation of video recordings and iterative analysis of PSTs’ unit plans and reflections.
Results
Results are organized around two research questions. The first research question investigated how PSTs developed their unit plans for teaching reading to secondary schoolers by utilizing ChatGPT. The second research question explored the considerations that PSTs took into account when interacting with ChatGPT to evaluate its output.
RQ1
The PSTs’ unit plan design processes revealed that they mainly went through four stages: (a) Inquiry and Prompt, (b) Examine and Evaluate, (c) Modify and Tinker, and (d) Synthesize and Organize. This article presents these four stages in a linear manner for ease of reading. However, in the actual unit plan development processes, these stages were recursive and did not occur sequentially. PSTs’ actions at each stage were informed by the components of TPACK.
Inquiry and Prompt
At this stage, PSTs usually started to engage with ChatGPT with a purpose or query. They sought information and knowledge by entering a statement or asking a question to ChatGPT. Data revealed that the prompts provided by PSTs ranged on a continuum from being general to specific. The continuum demonstrated different levels of detail, specificity, and focus given by different PSTs. The following examples illustrate the initial prompts that three pairs of PSTs typed in to request a unit plan from ChatGPT.
Example 1: “Write me a literacy unit plan on Richard Peck’s book Are You in the House Alone?”
Example 2: “Give me a 5-day middle school unit plan on The Giver.”
Example 3: “Write a Twilight unit plan for high school students. The unit plan should include 5 lessons. Each lesson needs to include essential questions, objectives, standards, and activities. At the end of the unit, there should be a final project as a summative assessment.”
These three examples showed a progression from providing broader to more detailed initial prompts. The first pair of participants directly asked ChatGPT for a unit plan on a book without any details regarding the grade level, objectives, and context. Compared to the first example, the second pair of participants added timeframe and grade level to their prompt. The third pair provided ChatGPT with a more specific statement, which contained the key components and detailed requirements of a unit plan (e.g., essential questions, objectives, activities, and a summative assessment), besides specifying the grade level and timeframe. Based on the prompts that three pairs of PSTs entered, the pair quoted in Example 3 demonstrated how they utilized their pedagogical knowledge: the knowledge about basic components of a unit plan to form a prompt.
Although PSTs had inquiries or purposes before they started to use ChatGPT, they were still not sure whether those prompts they gave could elicit satisfied responses. Most of the PSTs in this study had little knowledge about the capabilities of ChatGPT and how to engineer an effective prompt. Lacking TK with it, many PSTs did not provide clear and specific prompts initially. As one PST shared in the written reflection, “I really did not have much of an understanding of using AI, so I had no clue with the prompts.” At this stage, PSTs engaged in typing in various prompts, yielding diverse responses, which led to the stage of examination and evaluation.
Examine and Evaluate
During this process, PSTs read and evaluated the output that ChatGPT produced. Data suggested that PSTs evaluated the ChatGPT’s output mainly on four dimensions: accuracy, relevancy, applicability, and specificity. Accuracy in this context was about the reliability of the information provided by ChatGPT, which should be verified for correctness from primary or other credible sources. Relevance referred to the degree to which information was connected to the PSTs’ specific purposes or goals for creating unit plans. Applicability emphasized the level of appropriateness or how suitable the unit plan produced by ChatGPT was for use in a specific teaching context. Specificity referred to the extent to which information could be customized to meet the learning objectives and address learners’ levels and needs. Learners denoted the target students for whom PSTs created the unit plans. Table 1 provides examples from the data to showcase different dimensions PSTs focused on when examining and evaluating ChatGPT’s output.
Table 1
Four Dimensions to Evaluate Responses Generated by ChatGPT
| Dimensions | PSTs’ Prompt | ChatGPT’s Output | PSTs’ Evaluation | Related TPACK |
|---|---|---|---|---|
| Accuracy | Give us some examples of symbolism in the book Twilight. | The apple is a recurring symbol in Stephenie Meyer’s Twilight series, and it carries several layers of meaning throughout the books. | The output is not accurate, because there is an apple on the cover of the novel, and it does have symbolic significance, but apples never appear in the book. | Content knowledge |
| Relevance | Write me a literacy unit plan on Richard Peck’s book Are You in the House Alone? | ChatGPT generated a 4-6 weeks’ unit plan for middle or high school by incorporating various reading, writing, and creative activities. | Although we utilized ChatGPT, my partner and I still had to come up with our new unit plan. I changed many items since they were not what we expected, such as the steps of the procedures, vocabulary choices, and objectives. | Pedagogical Content Knowledge |
| Applicability | Create a 5-day unit plan on Lois Lowry's novel The Giver formiddle school students. | One of the activities that ChatGPT generated was “Read-Aloud (15 minutes): Begin reading the first chapter of The Giver as a class. Stop at a cliffhanger to encourage students to continue reading on their own.” | The AI suggested that it take 15 minutes for the students to read chapters 1-8 of the book. That is by no means realistic or appropriate for any age of individuals to do that. | Pedagogical Content knowledge |
| Specificity | Create an 8th-grade unit lesson plan consisting of 3 lessons for the book Hatchet by Gary Paulsen. | ChatGPT generated a two-day lesson plan with the outline: “Hook — Introduction to Hatchet — Reading — Discussion.” For example, one discussion question that ChatGPT generated was “Have students work in pairs or small groups to complete the character analysis chart using evidence from the text.” | ChatGPT does not provide any quotes and much information from books. It will give you a little detail about the book but nothing too specific. You would really have to write the prompt a specific way or modify the results and add more details. | Pedagogical Content knowledge |
| Note. Due to the lengthy unit plans generated by ChatGPT, the table summarizes ChatGPT’s output (see the third column from the left). | ||||
PSTs’ evaluation of the unit plan produced by ChatGPT indicated that they were able to make informed decisions by leveraging their knowledge of TPACK. Four dimensions (accuracy, relevancy, applicability, and specificity) of the unit plan that PSTs assessed revealed that they considered whether the plan was appropriate and effective in delivering the targeted content.
Modify and Tinker
At this stage, participants modified and adjusted the prompts, and the output produced by ChatGPT. The word “tinker” used in this study specifically referred to the actions in which PSTs attempted to fix or improve prompts in an experimental and iterative manner (as in Vossoughi & Bevan, 2014). It involved reading, rereading, and refining the prompts for multiple times. During the process of modifying and tinkering, PSTs either experimented with their input (prompts and trying to elicit more satisfied results) or directly engaged in improving the output generated by ChatGPT.
When reworking the prompts they input, PSTs tended to (a) abandon the original prompt and create a new prompt, (b) refine the prompt to make it more specific and personalized, or (c) enter follow-up questions or provide feedback to guide ChatGPT to modify its new output. Table 2 provides three examples from the data to illustrate how one pair of PSTs created a unit plan on The Hunger Games by continuously refining their prompts to ChatGPT.
Table 2
Examples of Modification of Input
| Instance | Initial Prompt | Summary of ChatGPT’s Output | Second Prompt | PSTs’ Verbalized Rationale | Related TPACK |
|---|---|---|---|---|---|
| Abandon the original prompt and provide a new prompt | Give quotes from The Hunger Games chapters 1-4 on social inequality. | ChatGPT shows it can’t provide verbatim excerpts from specific chapters of The Hunger Games as it is copyrighted material. | Give examples of social inequality in the first four chapters of The Hunger Games. | ChatGPT can’t give the answer we requested, so we have to change the prompt. | Content knowledge |
| Refine the prompt to be more specific and personalized | Create a lesson plan on the first three chapters of the plot of The Hunger Games. | ChatGPT generated the objective, materials, 3-day lesson plans with activities, conclusion, and assessment. | Write a lesson plan on the symbolism in the first three chapters in The Hunger Games. | The output was too broad based on the plot of the book. We wanted to focus on symbolism. | Content & pedagogical knowledge |
| Provide follow-up questions or feedback as further prompt | How chapters 1-4 of The Hunger Games relate to the United States. | ChatGPT generated several themes, such as social inequality, government control, exploitation of resources, media influence and so forth. | This is great! Can you write a lesson plan on how to teach students about these themes in The Hunger Games, which can be connected to real-world issues in the United States. | We are very interested in these topics and want an in-depth exploration of that, and teach students about real world issues, like inequality. | Content & pedagogical knowledge |
| Note. Due to the lengthy responses generated by ChatGPT in some cases, the table summarizes ChatGPT’s output (see third column from the left). | |||||
Another circumstance was that PSTs directly modified the output generated by ChatGPT by (a) adding details, (b) deleting unnecessary or repetitive information, and/or (c) rewriting or editing some of the information to ensure the revised or refined output can align with their objectives. Table 3 selects the same pair of PSTs who designed the unit plan on The Hunger Game. It demonstrates how this pair actively tailored the information produced by the ChatGPT to achieve their teaching expectations. In this example, the pair of PSTs crafted instructions for analyzing chapters 5-9 from The Hunger Games, focusing on social dynamics and resistance within the narrative. Through this exploration, secondary learners targeted in the unit plan would be able to draw connections to real-world issues in the United States, fostering critical thinking and understanding of societal structures.
Table 3
Examples of Modification of Output
| ChatGPT’s Output | PSTs’ Action | PSTs’ Verbalized Rationale | Related TPACK | |
|---|---|---|---|---|
| Add details | Activity- Explore instances of resistance and defiance within chapters 5-9, particularly focusing on Katniss and Peeta's actions. | Adding: Katniss and Peeta both demonstrate instances of resistance and defiance against the Capitol’s control and the oppressive nature of The Hunger Games. Here are some key examples: Alliance with Rue, Mockingjay Gesture, Peeta's Decision to Join the Careers. | We expect students to come up with these ideas independently. These are strictly in the lesson to help facilitate and lead discussion if the class comes to a standstill. | Pedagogical Content Knowledge |
| Delete unnecessary or repetitive information | 1. Discuss how characters resist the Capitol’s control and challenge the norms in The Hunger Games. 2. Group analyzes instances in the text where characters employ their assigned resistance strategy. | Delete the first activity. | These two activities are similar, as they both involve analyzing the behavior and actions of characters, so we only keep one. We also wanted students to focus on the next project-- Independent research: Real world connection. This would take more time to do. | Pedagogical Content Knowledge |
| Rewrite/edit partial output | Divide students into groups and assign each group a specific form of resistance (e.g., collaboration, defiance, manipulation). | Have a whole group discussion on specific forms of resistance (e.g., collaboration, defiance, manipulation), instead of small group discussion. | These examples are simply to facilitate discussion and bring up ideas if necessary. Students should be coming up with these ideas on their own, which is important and saves time for independent projects. | Pedagogical Content Knowledge |
During the modify and tinker stage, PSTs not only needed to employ their TPACK knowledge to evaluate the unit plans generated by ChatGPT but also used this set of knowledge to make revisions. They aimed to use their professional knowledge and skills to ensure the unit plans were comprehensive, engaging, and tailored to the needs of the target learners.
Synthesize and Organize
In this stage, PSTs strategically selected, combined, and synthesized information across multiple sources to create the unit plans that could be realistically implemented, since some PSTs were not limited to using ChatGPT as the only tool. Data revealed that if PSTs were not satisfied with the output they utilized other tools to search for their desired information, applied their own TPACK, and integrated new ideas and strategies into their unit plans design. For example, one pair of participants incorporated ideas from Google Search, the textbook, and Magic School AI to complete a unit plan design. Table 4 demonstrates how a pair of PSTs synthesized information from multiple sources to complete a unit plan.
Table 4
An Example of Information Synthesis From Multiple Sources
| Tools | Information Incorporated in the Unit Plans |
|---|---|
| ChatGPT | Keep the partial responses from ChatGPT, mainly the outline: Introduction to the novel; Exploring the setting; Character development; and Themes and symbolism |
| Google Search | Examples of unit plan of The Giver, learning activities, story element worksheet |
| Magic School AI | Summative assessment for The Giver-- Multiple choice questions & short answers |
| Textbook | Reading strategies/Anticipation guide |
Furthermore, given that the study was situated in a teacher education program, PSTs were required to follow the professor-created template to organize the unit plans. This process was further supported by the reflection of one pair of PSTs:
When finalizing our unit plan, we put our heads together and utilized our knowledge to craft this unit plan. We are both very familiar with what elements go into a lesson to make it beneficial. We also incorporate some ideas that ChatGPT generated and other resources like the textbook. We are very used to the structure of a university lesson plan format, so we made sure the AI-generated unit plan followed that similar format and had the same elements. We ultimately wanted to make sure that the procedure part of the lesson was detailed and sequential.
All these four stages were closely intertwined, where PSTs engaged in trial-and-error through a series of actions. Getting a satisfied output required an iterative process and effort. Data showed that PSTs went through several rounds of trying different prompts, examining the responses, learning from the previous attempts, and making adjustments and modifications. In the meantime, PSTs needed to fully make use of their own knowledge and skills regarding the teaching content and pedagogy to better support their evaluation and modification of ChatGPT’s output. Figure 1 depicts PSTs’ construction of information to develop unit plans, in which they actively interact with the tools by selecting, examining, evaluating, and synthesizing information from various sources within the context they are situated.
Figure 1
Processes of Constructing Unit Plans
This figure illustrates the nuances when PSTs constructed unit plans by leveraging multiple resources, information, and TPACK. The information can be drawn from GenAI technologies, like ChatGPT, as well as from various other resources, both digital (e.g., Google search) and nondigital (e.g., textbooks). However, all the information cannot be directly used to build unit plans.
PSTs need to fully use their TPACK to critically evaluate and synthesize information across multiple resources, ranging from their understanding and knowledge of subject matters, teaching content and strategies aligned with curriculum standards, and targeted learners’ backgrounds and needs. The knowledge of TPACK actively interacts with the iterative processes that PSTs go through. For example, if PSTs have a good understanding of how ChatGPT generates results, a good command of the content they teach, and knowledge of effective teaching methods, they could create clearer and more detailed prompts for ChatGPT to elicit better answers.
Moreover, PSTs’ knowledge of content, pedagogy, and technology informed them to engage in a series of actions — analyze the output generated by ChatGPT, make decisions to revise the output as needed and synthesize the output to reflect their thinking and ideas about what constitutes a good unit plan. Additionally, the immediate instructional environment, referring to the specific situation in which PSTs create their unit plans, also plays a role during the instructional design process. For example, in this study, PSTs created the unit plans for a course assignment, so their plans were also influenced by the contextual factor, the professor’s expectations, which is explained in more detail in RQ2.
RQ2
Observations from PSTs’ recordings of their unit plan design processes and interviews revealed that the evaluation of ChatGPT’s output might be influenced by multiple factors, which were related to PSTs’ TPACK in several ways. Data from this study suggested that the factors that stood out were PSTs’ CK, PK, PCK, and XK. In addition, other factors included their personal preferences and the professor’s rubrics in the course.
Content Knowledge
Data suggested that PSTs assessed the information generated by ChatGPT based on their prior knowledge about the target content and subject matter. PSTs’ CK allowed them to verify whether the information was current, correct, or appropriate for teaching their students. In this study, PSTs who chose adolescent literature for teaching reading should have a thorough understanding of the content of the literature. For instance, when a pair of PSTs searched for examples of symbolism in the book Twilight, ChatGPT informed them that the apple was a recurring symbol in the book. Those two PSTs immediately identified it as an error, since they knew the content of the book well, with the apple symbol only showing on the cover of the book, not the entire book. Moreover, by understanding the subject matter, PSTs were able to adjust instructions to target specific content for their students rather than simply following the ChatGPT’s output.
Within this study, since PSTs had taken multiple literacy methods courses, it was highly likely that they had well developed their CK in ELA for adolescent students. Namely, what knowledge, concepts, skills, and practices should be included in teaching reading through literature for adolescents. A case in point was that a pair of PSTs created the unit plan to teach a piece of literature, Too Many Tamales? They did not completely follow the outline produced by ChatGPT. Instead, they mapped out their own unit plan outline by chunking it into five major components based on their knowledge of the literature analysis (i.e., teaching story grammar in literature), which involved (a) introduction of the setting, plot, and themes of the story, (b) careful reading of selected paragraphs and focused vocabulary learning, (c) character analysis, (d) cultural exploration in Mexican and other Latinx cultures, and (e) culminating activity: create a poster or PowerPoint slide presentation to address a specific problem in the story. As they reflected,
AI is not the “end-all-be-all” for generating lesson plans. Teachers need to critically evaluate and modify these AI-generated lessons to meet their students’ needs. The real effectiveness of teaching comes from teachers because teachers have the knowledge about what students need to learn and how they learn best.
Another example was one pair of PSTs who used the novel Third Grade Angels by Jerry Spinelli to teach sixth graders language arts. They decided to focus more on vocabulary learning as an important component of literacy education. As they reflected,
I see using AI as a resource to create and help with the outline of lesson plans, but we decided to include vocabulary into our lesson plan and fluency since we wanted to focus more on the 5 aspects of language, and focus more on vocabulary, comprehension, and fluency.
Pedagogical Knowledge
With PK, PSTs can deliberately make choices and use strategies when designing and implementing learning experiences (LaVelle et al., 2020). This set of knowledge includes but is not limited to understanding targeted students’ backgrounds and needs, learning objectives, curriculum requirements, and learning sequences and activities.
Targeted Students’ Background. Understanding the background of targeted students is essential to knowing what they have already known about the learning subjects, their levels, interests, and any specific needs they may require. When a PST designed a unit plan with ChatGPT, they could provide detailed and contextualized prompts. However, ChatGPT still could not fully understand a particular group of targeted students and their specific needs. PSTs themselves had the autonomy to decide resources and activities that catered to students’ varying skill levels, learning preferences, and needs based on their pedagogical knowledge. As a PST reflected in the interview,
I also find that AI can be quite dull in the fact that it does not always provide engaging activities for students to develop their knowledge, and it also does not consider specific students with disabilities (i.e., students in your current classroom). ChatGPT provides brief suggestions of accommodations and modifications, but nothing specifically tied to a student’s IEP goals or personal experiences.
Alignment with Learning Objectives and Curriculum Requirements. Knowledge of learning objectives and curriculum is found to be a crucial component of PK (Hudson, 2004). PSTs could leverage this knowledge not only to guide ChatGPT in creating specific unit plans but also to assess if the unit plan was aligned with learning objectives and curriculum. For example, when dissatisfied with ChatGPT’s initial response to their request for a unit plan on the book Twilight, this pair of PSTs refined and reentered their prompts by including learning objectives, essential questions, and state subject standards. The updated prompt is shown in Figure 2, demonstrating how these two PSTs aimed to prompt ChatGPT in light of the curriculum and educational goals.
Figure 2
Updated Prompt for Creating a Unit Plan for Teaching Twilight
Other Pedagogical Consideration. In the study, other PST pedagogical considerations included the learning sequencing and procedures, time allocation, the design and implementation of engaging activities, and the methods of assessment. Most of the time, PSTs demonstrated their concern about the generic unit plans that ChatGPT produced, which could not be used and implemented directly in practice. The following two quotations illustrate PSTs’ opinions about ChatGPT’s responses in an educational setting and suggest ways to improve them.
We then were able to embed aspects of our textbook into our lesson, which helped strengthen it even more. The lesson plans developed by ChatGPT were generalized. Since they can’t use specific examples from the book, my partner and I went in to make it more personalized and geared to the book using the Beers Textbook.
It [ChatGPT] provided us with good ideas and a general outline of a lesson plan, However, there definitely needs to be modifications made to make it more personalized to the class as well as the class content. Some ways that we want to make it more specific is by providing visual aids, working with partners, offering a glossary of words, scaffolded questions, an audiobook, and many other examples.
Pedagogical Content Knowledge
PCK supports PSTs in deciding which instructional strategies are suitable for teaching specific content. In the study, one pair of PSTs decided to use the novel Holes, written by Louis Sachar, to teach seventh-grade special education students, believing it to be appropriate content for them. After evaluating the unit plan generated by ChatGPT, they decided to add several strategies that were tailored for teaching literature and reading, such as KWL, a strategy to guide the learning process, journaling, and multimodal expressions. The following quotation illustrates how and why they taught young adult literature to seventh graders by considering a series of these teaching strategies.
Holes is a great story for middle schoolers due to its simple language and relatable themes. ChatGPT was beneficial to generate the outline of the unit plan, but we need to add the activity of “It says, I say, and So” since using learning organization charts are great for students. I also used “KWL (know, want to know, learned) charts” because this was good for teaching reading comprehension. We also used journaling and drawing pictures. The lesson plan we modified based on ChatGPT reflects what we know about special education and what the story can bring to students.
ConteXtual Knowledge
Another factor that influenced PSTs’ evaluation of ChatGPT’s output was the immediate context that they were situated, which was related to the XK component of TPACK. Since this study was conducted within a course in a teacher education program with preservice teachers, the PSTs constructed the unit plans with careful consideration of how the professor in the course would grade their unit plans. They tended to adapt their unit plans to meet the criteria provided by the professor in the course. One pair of PSTs said,
My partner and I used ChatGPT as more of an outline, rather than a resource when creating our unit plan. We both already had a good plan about what we wanted to do with our unit plan. We wanted to frame it using what we learned in the college course and the professor’s rubric about creating the unit plan.
Personal Preferences
PSTs’ personal preferences could be a factor in influencing their evaluation of ChatGPT’s output. This type of judgment may not be based on concrete evidence but rather informed by instinct. For instance, one pair of PSTs reflected,
We then asked it to list activities that could be completed based on the range of chapters it provided. We ultimately decided to turn to Google or our own ideas after seeing the ones that ChatGPT provided. There wasn’t anything inherently wrong with the ones it gave, we just did not like them all that much.
Drawing from PSTs’ reflections, interviews, and unit plans they designed, this study highlighted how TK, CK, PK, PCK, and XK came into play with PSTs’ interaction with ChatGPT. Because PSTs barely integrated technologies into their unit plan design and, therefore, minimally drew upon their TPK and TCK when evaluating and revising unit plans, it did not mean that these two components were less important. TPACK guided PSTs to adjust their prompts to obtain more satisfactory output, as well as support them in analyzing and revising the output of ChatGPT. With the understanding of this set of knowledge, PSTs strived for the best practice to teach reading to secondary students.
Discussion
This study makes timely contributions to the field by extending our current emerging understanding of PSTs’ practices with GenAI in a teacher education program. Particularly, it explores PSTs’ complex and nuanced relationship with ChatGPT and factors to consider when developing unit plans for literacy instruction. It responds to the current debate on whether GenAI, such as ChatGPT, should be banned in educational settings and whether new emerging AI technologies could replace teaching professions.
Findings suggest that PSTs did not passively receive the information produced by GenAI. Instead, they tended to go through complex processes rather than completely relying on the output of GenAI, which is engaged in four distinct stages in an iterative and nonlinear way. This iterative process indicates that the use of GenAI technology is as complex as digital reading and writing (Coiro, 2021), both of which exert great cognitive and socially constructed demands, requiring the coordination of multiple knowledge, skills, and social practice to accomplish the task.
Each of the four stages of (a) inquiry and prompt, (b) examine and evaluate, (c) modify and tinker, and (d) synthesize and organize requires PSTs to apply their higher order thinking skills, such as synthesizing and critical thinking. Particularly, PSTs need to understand fully what constitutes a high-quality unit plan and effective teaching practices, which helps them critically evaluate the information they obtain from various resources, especially the responses generated by AI.
This finding also indicates that the current GenAI, like ChatGPT, which relies upon a large amount of training data to generate responses (Dasborough, 2023), shows remarkable capabilities to support educators with some daily teaching tasks, such as creating lesson plans, coming up with rubrics, and brainstorming learning activities, but cannot entirely replace the teaching profession. While GenAI can supplement these aspects to some extent, an effective unit plan still requires teachers’ critical thinking, creativity, contextual judgment, and integration of teachers’ expertise, including understanding and leveraging of TPACK and other human traits, which cannot be provided by current GenAI.
This study suggests the complex nature of using GenAI technology in developing unit plans, which hinges upon the multiple bodies of knowledge and skills required on the part of PSTs. This complexity is highlighted by the major obstacles reported by the participating PSTs of not knowing how to create and refine prompts to ChatGPT. At times, PSTs could struggle when using GenAI. This finding is aligned with the findings from earlier studies showing technology use is by no means a simple task in the educational setting (De Rossi & Trevisan, 2018; Muilenburg & Berge, 2015). As teacher educators, we cannot assume young teachers in training would know how to use new technologies effectively for educational purposes because of the ubiquitousness of technologies in today’s world. Transformative GenAI technology, in a way, presents more pressing challenges to classroom teachers, as it asks for more sophisticated skill sets, such as problem-solving and critical thinking skills. Teacher education programs, thus, need to respond to the findings of the study.
Develop PSTs’ TPACK
In this study, the TPACK framework served an analytical purpose lens for examining the necessary knowledge of educators in today’s educational environment. As Mishra and Koehler (2006) asserted, the new technologies “often disrupt the status quo, requiring teachers to reconfigure not just their understanding of technology but all three components” (p. 1030). GenAI, as an emerging technology, requires future educators not only to utilize it but also to do so in a critical and effective manner, which should be supported by their strong PK, TK, and CK. This set of knowledge not only guides PSTs to craft more effective prompts to ChatGPT but also equips them with the knowledge to evaluate the output generated by ChatGPT.
Hence, teacher education programs must design a series of curricula for PSTs by taking these questions into consideration, such as the following: How can PSTs be supported in developing TPACK? What strategies can be used to enhance understanding and practice TPACK in teaching and learning? And how can the effectiveness of TPACK training in teacher education programs be assessed?
Promotes AI Literacies
The saturation of AI-led technologies in education highlights the importance of equipping future educators with necessary AI literacies. AI literacies not only refer to knowledge and skills of using AI tools but also include critical engagement and social practice. This approach defines AI literacies as situated within the context of its use, shape, and shaped by specific norms and practices. Teacher education programs should explicitly engage future teachers in the conversations and offer authentic experiences to explore the affordance and limitations of utilizing GenAI to support unit plan design. Programs should situate future educators into practical scenarios in which teacher candidates can gain opportunities to use their AI literacies and reflect on what they have learned and experienced.
PSTs can only become flexible with using AI, including prompt engineering, only after practices, reflection, and internalization since there might not be a perfect formula for them to simply follow. Importantly, AI literacies for educators need to marry with TPACK (Mishra & Koehler, 2006), which involves the interaction of AI-specific knowledge, skills, and each component of TPACK. Table 5 presents some suggestions for educators to consider when utilizing ChatGPT by leveraging the TPACK framework. These suggestions and insights can be applied to a different educational context by integrating AI technologies.
Table 5
Suggestions for Evaluating AI-generated Unit Plans Guided by TPACK Framework
| Process of Use AI for Unit Plan Design | Suggestions or Instructional Guidelines |
|---|---|
| Prompt engineering for eliciting AI-generated content | - Understand the capacities of different AI tools - Understand how to use AI technologies to best support teaching and learning goals, such as unit plan development. - Understand how to use clear language to craft effective prompts by considering learning objectives, standards, specific content, best learning strategies, and other contextual information, such as classroom dynamics. |
| Evaluate output generated by AI | - Analyze the AI-generated unit plans by considering four dimensions (accuracy, relevance, applicability, specificity). - Reflect whether AI-generated unit plans well present learning objectives, learning context, process (e.g., sequence of instruction, technology integration, learning scaffolding) and products- the evidence of students’ learning. |
| Modify & synthesize AI-generated content for specific learning objectives | - Understand how to adjust represent clear objectives, effective learning materials, resources, instructional strategies, and assessment methods. - Understand what needs to be improved or revised to meet students’ needs. - Choose appropriate technology to support the instructional design that enhances student learning by having a good alignment of AI technology to instructional goal and pedagogy choice. |
Limitation and Future Research
This qualitative exploratory study was situated in an instructional setting with preservice teachers enrolled in a teacher program at a private university. PSTs created unit plans for an assignment, not for actual teaching. Therefore, further research is needed to understand the process by which in-service teachers develop unit plans for real-life teaching purposes. In addition, as aforementioned, AI technologies are still emerging. Educational fields call for more empirical research exploring the affordances, limitations, and applications of AI.
Based on the findings and understanding of this exploratory inquiry, more questions remain for further exploration, such as the following: How do educators perceive their roles in the AI age versus the roles played by AI in teaching and learning? What are the ethical issues and potential barriers when using AI in designing unit plans?
Given that TPACK is a useful theoretical perspective guiding the lien of research on AI, it is necessary to conduct an in-depth investigation on what aspects of TPACK are essential to develop the effective use of AI technologies in teaching and learning; Similarly, such questions are equally important regarding how AI technologies can be integrated within the TPACK framework to enhance educators’ instructional practices and how professional development and training programs can be designed to improve TPACK and AI literacy of practicing and prospective teachers.
Conclusion
Technology is a rapidly changing landscape, continuously impacting the ways people teach and learn. The integration of AI-led technologies in education brings up both optimism and uncertainty. There is an urgent need to understand how educators interact with AI technologies to make meaning, as well as how they integrate technology into the curriculum in ways to enhance the learning process and outcomes. The findings and implications of this study can inform teacher education programs to better prepare educators for the technology-rich educational settings of today. As technologies evolve in this fast-changing age, each and every educator must be a lifelong learner, continuously engaging in this ongoing process of adaptation and learning.
References
Alam, A. (2021, December). Should robots replace teachers? Mobilisation of AI and learning analytics in education. Proceedings of the 2021 International Conference on Advances in Computing, Communication, and Control (ICAC3; pp. 1–12). IEEE.
Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62.
Baskara, R., & Mukarto, M. (2023). Exploring the implications of ChatGPT for language learning in higher education. Indonesian Journal of English Language Teaching and Applied Linguistics, 7(2), 343- 358
Bazeley. (2013). Qualitative data analysis: Practical strategies. SAGE
Blankenship, R., Mourlam, D., Berson, I., Berson, M., Lee, C.Y., Peng, L.W., Jin, Y., Lyublinskaya, I., Du, X., Warr, M., Mishra, P., Williams, M., & Hatfield, T. (2022). Reimagining practical applications of the TPACK framework in the new digital era. In E. Langran (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 2198-2203). https://www.learntechlib.org/primary/p/221011/
Biasutti, M., & Frate, S. (2018). Group metacognition in online collaborative learning: Validity and reliability of the group metacognition scale (GMS). Educational Technology Research and Development, 66, 1321-1338. https://doi.org/10.1007/s11423-018-9583-0
Bond, R., & Castagnera, E. (2006). Peer supports and inclusive education: An underutilized resource. Theory into Practice, 45(3), 224-229.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.
Celik, I. (2023). Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468.
Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in High Education 20, 43. https://doi.org/10.1186/s41239-023-00411-8
Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, 100118. https://doi.org/10.1016/j.caeai.2022.100118
Coiro, J. (2021). Toward a multifaceted heuristic of digital reading to inform assessment, research, practice, and policy. Reading Research Quarterly, 56(1), 9–31. https://doi.org/10.1002/rrq.302
Dasborough, M. T. (2023). Awe‐inspiring advancements in AI: The impact of ChatGPT on the field of organizational behavior. Journal of Organizational Behavior, 44(2), 177–179. https://doi.org/10.1002/job.2695
De Rossi, M., & Trevisan, O. (2018). Technological pedagogical content knowledge in the literature: How TPCK is defined and implemented in initial teacher education. Italian Journal of Educational Technology, 26(1),7–23.
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023). Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research, 25(3), 277-304.
Gamage, K. A., Dehideniya, S. C., Xu, Z., & Tang, X. (2023). ChatGPT and higher education assessments: more opportunities than concerns? Journal of Applied Learning and Teaching, 6(2), 1-12. https://doi.org/10.37074/jalt.2023.6.2.32
Grassini, S. (2023). Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7), 692. https://doi.org/10.3390/educsci13070692
Harris, J. B., & Hofer, M. J. (2011). Technological pedagogical content knowledge (TPACK) in action: A descriptive study of secondary teachers’ curriculum-based, technology-related instructional planning. Journal of Research on Technology in Education, 43(3), 211-229.
Hava, K., & Babayiğit, Ö. (2024). Exploring the relationship between teachers’ competencies in AI-TPACK and digital proficiency. Education and Information Technologies. https://doi.org/10.1007/s10639-024-12939-x
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.
Hudson, P. (2004). Toward identifying pedagogical knowledge for mentoring in primary science teaching. Journal of Science Education and Technology, 13, 215-225.
Jauhiainen, J. S., & Guerra, A. G. (2023). Generative AI and ChatGPT in school children’s education: Evidence from a school lesson. Sustainability, 15(18), 14025.
Karlsson, G., & Nilsson, P. (2023). Capturing student teachers’ TPACK by using T-CoRe and video-annotation as self-reflective tools for flexible learning in teacher education. Technology, Pedagogy and Education, 32(2), 223-237.
Kim, S. W. (2024). Development of a TPACK educational program to enhance pre-service teachers’ teaching expertise in artificial intelligence convergence education. International Journal on Advanced Science, Engineering & Information Technology, 14(1), 1-9.
Kim, K., & Kwon, K. (2023). Exploring the AI competencies of elementary school teachers in South Korea. Computers and Education: Artificial Intelligence, 4, 100137.
Koehler, M. & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. https://citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge
Koehler, M. J., Mishra, P., Kereluik, K., Shin, T. S., & Graham, C. R. (2014) The technological pedagogical content knowledge framework. In J. M. Spector, M. Merrill, J. Elen, & M. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 101-111). Springer.
KV, P. (2023, August 14). Harnessing the power of ChatGPT for instructional design: Transforming education through AI. Linkedin. https://www.linkedin.com/pulse/harnessing-power-chatgpt-instructional-design-transforming-philip-kv/
LaVelle, J. M., Lovato, C., & Stephenson, C. L. (2020). Pedagogical considerations for the teaching of evaluation. Evaluation and Program Planning, 79, 101786.
Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2017). New literacies: A dual level theory of the changing nature of literacy, instruction, and assessment. Journal of Education. Reprinted from Alvermann, D.E., Unrau, N.J., & Ruddell, R.B. (Eds.). Theoretical models and processes of reading (6th ed.). (pp. 1150-1181). International Reading Association.
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–16). ACM. https://doi.org/10.1145/3313831.3376727
Maguire, M., & Delahunt, B. (2017). Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars. All Ireland Journal of Higher Education, 9(3). 3351-33514.
McKibben, W. B., Cade, R., Purgason, L. L., & Wahesh, E. (2020). How to conduct a deductive content analysis in counseling research. Counseling Outcome Research and Evaluation, 13(2), 156–168. https://doi.org/10.1080/21501378.2020.1846992
Memarian, B., & Doleck, T. (2023). ChatGPT in education: Methods, potentials and limitations. Computers in Human Behavior: Artificial Humans, 100022.
Mishra, P. (2019). Considering contextual knowledge: The TPACK diagram gets an upgrade. Journal of Digital Learning in Teacher Education, 35, 76-78. https://doi.org/10.1080/21532974.2019.1588611
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teachers’ knowledge. Teachers College Record, 108(6), 1017–1054. doi:10.1111/j.1467-9620.2006.00684.x
Mishra, P., Warr, M., & Islam, R. (2023). TPACK in the age of ChatGPT and generative AI. Journal of Digital Learning in Teacher Education, 39(4), 235-251.
Mupita, J., Widiaty, I., & Abdullah, A. G. (2018, November). How important is technological, pedagogical, content knowledge? A literature reviews. In IOP Conference Series: Materials Science and Engineering (Vol. 434, No. 1, p. 012285). IOP Publishing.
Muilenburg, L. S. M. S. Y., & Berge, Z. L. (2015). Revisiting teacher preparation. Quarterly Review of Distance Education Journal Issue, 16(2), 93–105.
Office of Educational Technology. (2023). Artificial intelligence and the future of teaching and learning. https://tech.ed.gov/ai-future-of-teaching-and-learning/
Ray, P. P. (2023). Benchmarking, ethical alignment, and evaluation framework for conversational AI: Advancing responsible development of ChatGPT. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(3), 100136.
Saldaña, J. (2015). The coding manual for qualitative researchers. Sage.
Sheffield, R., Dobozy, E., Gibson, D., Mullaney, J., & Campbell, C. (2015). Teacher education students using TPACK in science: A case study. Educational Media International, 52(3), 227-238.
Tondeur, J., Scherer, R., Siddiq, F., & Baran, E. (2020). Enhancing pre-service teachers’ technological pedagogical content knowledge (TPACK): A mixed-method study. Educational Technology Research and Development, 68, 319-343.
Trust, T., Whalen, J., & Mouza, C. (2023). Editorial: ChatGPT: Challenges, opportunities, and implications for teacher education. Contemporary Issues in Technology and Teacher Education, 23(1), 1-23. https://citejournal.org/volume-23/issue-1-23/editorial/editorial-chatgpt-challenges-opportunities-and-implications-for-teacher-education
van den Berg, G., & du Plessis, E. (2023). ChatGPT and generative AI: Possibilities for its contribution to lesson planning, critical thinking and openness in teacher education. Education Sciences, 13(10), 998. https://doi.org/10.3390/educsci13100998
Vogt, K., & Flindt, N. (2023). Artificial intelligence and the future of language teacher education: A critical review of the use of AI tools in the foreign language classroom. In P. Hohaus & J-F. Heeren (Eds.), The future of teacher education (pp. 179-199). Brill.
Voogt, J., & McKenney, S. (2017). TPACK in teacher education: Are we preparing teachers to use technology for early literacy? Technology, Pedagogy and Education, 26(1), 69-83.
Vossoughi, S., & Bevan, B. (2014). Making and tinkering: A review of the literature. Board on Science Education, National Research Council. http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_089888.pdf.
Weidlich, J., & Kalz, M. (2023). How well does teacher education prepare for teaching with technology? A TPACK-based investigation at a university of education. European Journal of Teacher Education, 1-21. https://doi.org/10.1080/02619768.2023.2243645
Wen, H., & Shinas, V. H. (2020). Using a multidimensional approach to examine TPACK among teacher candidates. Journal of Digital Learning in Teacher Education, 37(1), 30-47.
Wen, W. (2023). Exploring teaching and learning in an undergraduate online digital literacies course: Instructor’s scaffolding, students’ performance and perspectives [Unpublished doctoral dissertation]. University of Arizona, Tuscon, AZ, United States.
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q. L., & Tang, Y. (2023). A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5), 1122-1136.
Young, N. (2023, November 15). ChatGPT for instructional design: Unleashing game-changing tactics. Teachfloor. https://www.teachfloor.com/blog/chatgpt-for-instructional-design
Yue, M., Jong, M. S. Y., & Ng, D. T. K. (2024). Understanding K–12 teachers’ technological pedagogical content knowledge readiness and attitudes toward artificial intelligence education. Education and Information Technologies, 1-32. https://doi.org/10.1007/s10639-024-12621-2
Zhang, K., & Aslan, A. B. (2021). AI technologies for education: Recent research & future directions. Computers and Education: Artificial Intelligence, 2, 100025.
Appendix
Codebook Excerpt
Four themes below indicate the actions that PSTs took when engaging with ChatGPT, guided by their knowledge of the technology, content, and pedagogy. The emerging codes highlight the specific strategies and/or considerations that PSTs focused on, reflecting how TPACK has influenced their approach to handling ChatGPT’s output.
| PSTs' processes of developing unit plans | |
|---|---|
| Description: It refers to various steps that pre-service teachers engage in to create unit plans. | |
| Theme #1: Inquiry & Prompt | |
| Codes | Descriptions |
| General prompts | PSTs provide a broad request to ChatGPT without much detail |
| Specific prompts | PSTs offer more details to make requests more focused |
| Theme #2: Examine & Evaluate | |
| Codes | Descriptions |
| Accuracy | The accuracy and/or reliability of information generated by ChatGPT |
| Relevance | The level of pertinence of the information generated by ChatGPT in relation to PSTs’ goals |
| Applicability | The level of usefulness of the information generated by ChatGPT in relation to PSTs’ goals |
| Specificity | The level of detail of information generated by ChatGPT in relation to PSTs’ goals |
| Theme #3: Modify & Tinker | |
| Codes | Descriptions |
| Work on the input | PSTs rewrite, revise, or refine the prompts to elicit ChatGPT’s responses |
| Work on the output | PSTs rewrite, revise, or refine the output generated by ChatGPT |
| Theme #4: Synthesize & Organize | |
| Codes | Descriptions |
| Select | PSTs intentionally choose information from multiple sources |
| Synthesize | PSTs filter, make connection, and integrate multiple resources |
| Organize | PSTs arrange selected resources to form a unit plan |
