As digital educational technologies (edtech) become more sophisticated, teacher educators must prepare teachers to make thoughtful and well-informed decisions about their use in the classroom (Cardona et al., 2023). Teachers can use technologies to expand learning opportunities for students (Kimmons et al., 2020), but as is the case with any technology, the effects on teaching and learning often extend beyond what was originally intended. The growing field of critical edtech scholarship has drawn attention to the unintended and often negative impacts of edtech in schools (Macgilchrist, 2021).
For instance, research has documented how technologies whose purpose is to transform instructional practices often perpetuate traditional ones (Manolev et al., 2019; Selwyn, 2016; Sims, 2017). Scholars have also identified racial biases embedded in technologies that exacerbate educational inequities (Broussard, 2023, p. 65; Dixon-Román et al., 2020; Macgilchrist, 2019; Selwyn, 2022). Others have raised concerns over student privacy amid digital systems that gather copious amounts of student data (Boninger et al., 2019; Lupton, 2021; Marachi & Quill, 2020; Selwyn et al., 2021; Williamson, 2017a, b).
Collectively, critical edtech research challenges the notion that technology is an unqualified good, and that more technology is necessarily better (Boninger et al., 2019; Heath et al., 2022; Macgilchrist, 2021). Yet, there is a need to connect this scholarship to the work of prospective and practicing teachers. To date, much of the research on technology and teacher education has focused on supporting teachers’ technology integration competencies (e.g., via frameworks such as technological pedagogical content knowledge, also known as technology, pedagogy, and content knowledge [TPACK]; Koehler & Mishra, 2009) and willingness to utilize technologies in their classrooms (e.g., Granić, & Marangunić, 2019).
Meanwhile, little empirical research has focused on how to prepare teachers to critically analyze and make critically informed decisions about edtech, even though scholars have argued that such capacities are essential for classroom teachers (Heath et al., 2022; Henderson et al., 2023; Krutka et al., 2019; Rodriguez et al., 2020; Schroeder & Curcio, 2022; Weisberg & Dawson, 2023).
In this article, we present the results of our efforts to prepare preservice teachers (PSTs) to critically assess edtech. We define this critical practice as the ability to use multiple perspectives to examine and consider the varied effects that technologies bring to diverse classrooms, both positive and negative, intended and unintended (Heath et al., 2022; Krutka et al., 2019; Pleasants et al., 2023). Teachers need such perspectives to select, adapt, and implement technologies that do no harm while supporting students’ development and growth, including in ways that are culturally responsive and sustaining (e.g., Ladson-Billings, 2021).
To promote critical perspectives, we developed a series of learning activities for our edtech courses in which PSTs used conceptual scaffolds to critically examine the use of technologies in schools and classrooms. We specifically examined how PSTs applied their emergent critical perspectives to two common classroom technologies: Teachers Pay Teachers (https://www.teacherspayteachers.com/) and Class Dojo (https://www.classdojo.com/). We explored how PSTs utilized the conceptual scaffolds that we presented to them, the depth of their critical inquiries, and the areas of their thinking that require further development.
Literature Review
Mainstream research related to technology and teacher education typically focuses on developing teachers’ technology integration competencies and dispositions (e.g., Ertmer & Newby, 2016; Granić & Marangunić, 2019; Koehler & Mishra, 2009). Comparatively little attention, however, has been given to how to prepare teachers to make critical decisions related to educational technology. In a recent review, Henderson et al. (2023) examined empirical teacher education literature related to developing PSTs’ critical perspectives on technology. Although they identified a handful of studies that described raising PSTs’ critical awareness in technological contexts, they found few in which the technologies themselves were taken as the objects of critical inquiry.
For example, Riley et al. (2021) used Critical Race Media Literacy in an elementary literacy education course to help PSTs critically examine the content of YouTube comments. Similarly, Degand (2020) reported a study in which PSTs critically examined how race is represented or misrepresented across different media forms. In both studies, PSTs critiqued the content of media platforms but did not critically examine the underlying technologies (e.g., YouTube).
A study by Rodriguez et al. (2020) further illustrates this trend. They implemented a series of learning activities to help PSTs critically appraise lessons obtained from user-based platforms such as Pinterest (https://www.pinterest.com/) and Teachers Pay Teachers. The authors concluded that their activities would likely have been more effective if PSTs critically analyzed the platforms themselves (i.e., the technologies) instead of focusing primarily on their content (i.e., the lessons). In other words, the PSTs would have been better prepared to make better decisions related to the platforms had they developed a critical perspective toward them.
In a separate scoping review of the literature, Weisberg and Dawson (2023) identified studies addressing equity pedagogies and technology integration. Most of the studies they reviewed used technologies as tools to support equity pedagogies rather than serve as the objects of inquiry. Of the few studies they identified that centered technological issues, most raised PSTs’ awareness of inequitable access to digital technologies (e.g., Hall et al., 2020). They found only three studies in which PSTs applied critical perspectives to edtech — an approach they aligned with the notion of “technoethics” (Krutka et al., 2019). In one of those studies (Voithofer, 2005), PSTs were tasked with making ethical and responsible decisions when bringing technologies into a local urban school. Interestingly, they found that PSTs often resisted fully engaging in the kinds of critical reflections they were asked to do, emphasizing the need to further develop teacher education approaches on this topic.
Our survey of the literature revealed a substantial need for research that examines ways to develop PSTs’ critical views of edtech. It also demonstrated a disconnect between the fields of teacher education and critical edtech scholarship (e.g., MacGilchrist, 2021; Selwyn, 2016; Williamson, 2017a), a divide that needs amelioration (Heath et al., 2022). Our goal in this study was to contribute to bringing these fields together.
Conceptual Framework: Defining a Critical Perspective on EdTech
As a starting point for our work, we recognize multiple ways in which individuals can be critical of edtech. Macgilchrist (2021), for instance, identified multiple distinct types of critical edtech studies in the academic literature, from ones that focus on the multifarious transformations they bring to educational environments to ones that show how they uphold and reinforce existing patterns of power, oppression, and inequity. Those heterogenous works and perspectives inform our conceptualization of what it means to be “critical” of edtech, but we are also keenly aware that teachers need not become critical edtech scholars. Thus, we use technoskepticism as a holistic and teacher-accessible guiding framework that leverages and translates critical technology scholarship (Pleasants et al., 2023).
Technoskepticism is an orientation that involves questioning technology, approaching it with deliberateness and caution. However, it is not inherently antitechnology; technoskepticism, rather, emphasizes that technology has complex effects (both positive and negative) on individuals, communities, and society. Critical researchers have revealed the multifaceted effects of edtech platforms such as Canvas (Boninger et al., 2019; Marachi & Quill, 2020) and Class Dojo (Manolev et al., 2019, 2024; Williamson, 2017b), as well as educational apps such as Grasshopper (Decuypere, 2019), and even classroom furniture (Selwyn, 2024).
Like research that has examined the intersection of technology and society more generally, these studies showed that technologies are not neutral tools and always do more than what was intended by their designers and adopters (Ciccone, 2022; Garcia & Nichols, 2021; Selwyn, 2016, 2022). Technologies interact with myriad sociocultural and technical systems, bringing forth varied unanticipated effects that are rarely distributed equally or equitably across individuals or communities (Benjamin, 2019; Broussard, 2023; Chachra, 2023; Feenberg, 1999; Verbeek, 2015).
Technoskeptical thinking is challenging because the myriad effects of technology are not readily perceptible or easy to predict (Pleasants et al., 2023). The intended uses and outcomes of technologies are typically clear and straightforward, whereas the unintended effects are, by their nature, always less evident. To facilitate technoskeptical analyses, we use the Technoskepticism Iceberg (Pleasants et al., 2023) shown in Figure 1 as a conceptual scaffold. The metaphor of an iceberg conveys how important aspects of technology are not immediately visible. At the surface level, technologies present as tools with clear intended uses and outcomes. Tool-level evaluations of technology are narrowly confined to how well they accomplish what they are meant to do (e.g., Does this thermos keep my coffee warm? Does this projector work with my computer?). Yet viewing technologies only as tools obscures their more complex effects, which are revealed through deeper analyses (Broussard, 2023; Feenberg, 1999; Pleasants et al., 2019; Verbeek, 2015).
Figure 1
The Technoskepticism Iceberg
The Iceberg framework identifies two layers that are essential for deeper critical analyses of technology. First, technologies operate as complex sociotechnical systems: assemblages of interacting sociocultural and technical elements. What technologies do is a product not only of their technical characteristics, but also of the ways they interact with interconnected systems (e.g., economic systems, cultural customs, and language systems). Second, technologies are not neutral; instead, they are designed and used in ways that reflect and reinforce social and cultural values such as efficiency, cooperation, control, or autonomy.
The framework also lays out three dimensions along which technologies can be investigated. Technical inquiries focus on the material structures of technologies to examine how they bring about their myriad effects. Psychosocial inquiries address how technologies both enable but also affect the ways we think, act, and interact with one another, often in subtle or unseen ways. Political inquiries examine how decisions are made about technologies and who has the power to make those decisions.
Overview of Focal Technologies
In this study, we focused on ways PSTs critically analyzed two technologies (Teachers Pay Teachers and Class Dojo). We chose to address these technologies because they are both common in schools (Doan et al., 2023) and have been identified by prior research as demanding critical analysis due to their wide-reaching effects on teaching and learning (Manolev et al., 2019, 2024; Rodriguez et al., 2020; Shelton et al., 2022; Williamson, 2017b). Because of their central role in this study, we briefly describe each one and illustrate how the constructs of the Iceberg framework can be applied to facilitate critical inquiry.
Teachers Pay Teachers (TPT) is a digital marketplace that publishes user-generated lesson plans and instructional resources (Shelton et al., 2022). According to its website, “TPT is the world’s largest marketplace of PreK-12 resources, powered by a global community of experienced educators” (Teachers Pay Teachers, 2024, para. 1). The platform provides a search bar that allows users to locate resources based on topic and grade level. On the marketplace, resources have a price (which could be zero) set by the content creator, display star ratings that show user feedback, and include a brief description. Taking a Tools perspective, we might ask how well the platform achieves its intended function of connecting a user with whatever instructional resource they are looking for. We would likely focus on the Technical dimension and investigate the quality of the content that we might find for various topics or audiences. From the Psychosocial dimension, we might ask about the user experience and whether obtaining resources from the platform eases the stress on teachers.
Deeper layers of the Iceberg reveal more complex effects of the platform. To examine the Values embedded in the platform, we might ask how the need for the platform to be profitable could have influenced its Technical structure. We could question the way that that sales income is allocated to content creators versus the platform itself, the way that the platform relies on crowdsourced reviews of content rather than professional curation, or the priorities of the platform’s search algorithm.
From the Psychosocial dimension, we would raise questions about how the platform affects how teachers search for and locate resources within a broader System of curriculum materials. User-based rating systems can give teachers the impression that resources have been “vetted,” but they can also be easily manipulated (e.g., Ovide, 2021). Since users can only truly evaluate a resource after purchasing it, what happens if a lesson isn’t what was promised? Through lines of inquiry such as these, we can gain a better sense of whether TPT “works,” how and for whom it works, and how it might influence teachers’ planning practices.
Class Dojo is a digital classroom community in which each student is represented by a cartoon monster avatar. Teachers can use the platform to upload pictures or documents and associate them with students’ avatars. Teachers can also give (or take away) dojo points to students’ avatars. Depending on the platform settings, parents can view their children’s associated documents and points. Teachers and parents can also send messages through the platform.
As with TPT, viewing Class Dojo as a Tool means examining how well it performs those intended functions. From a Psychosocial dimension, we could ask how easily parents and teachers can communicate with each other through the platform. We could also examine how well the point system reinforces positive student behaviors.
Looking more deeply at the Psychosocial dimension, additional issues emerge. While teachers can use dojo points in many ways, the technology is largely aligned with behaviorist perspectives and Values (Williamson, 2017b). If a teacher uses Class Dojo, how might those embedded values affect the way that they interact with their students? For instance, Manolev et al. (2024) discovered that Class Dojo can shift teachers’ attention toward what the technology measures and away from other unmeasured aspects of their classrooms. Assigning points to certain behaviors over others inevitably reveals what is valued in our school System (and what is not).
There are also important Political questions. A core function of the technology is the communication of student information between teachers and parents. Who gets to decide what information is shared? What priority, if any, is placed on the Value of student privacy? These questions represent only a few key starting points for further investigation.
Contribution of the Study
Our work is part of a larger design experiment (Cobb et al., 2003) to understand how PSTs develop critical perspectives on classroom technologies as they transition from student users to teacher decision-makers. Given the paucity of research in this domain (Henderson et al., 2023; Weisberg & Dawson, 2023), our study was necessarily exploratory. We documented how our PSTs used technoskepticism and the Iceberg framework to consider the potential benefits and issues the focal technologies might bring. By examining PSTs’ critical analyses, we aimed to better understand the kinds of issues that PSTs most readily perceived, where their critical inquiries were more constrained, and how we might further support their critical thinking. Our goal was to build our understanding of PSTs’ thinking to guide our ongoing teacher education practice (Cobb et al., 2003). The following research questions guided our study:
- What benefits and issues did PSTs identify during their TPT and Class Dojo analyses, and how did those align with the elements of the Technoskepticism Iceberg?
- To what extent did PSTs’ analyses demonstrate technoskeptical thinking?
Methods
Our study used a conversion mixed methods design (Teddlie & Tashakkori, 2009) to address our research questions. The data we collected from our PSTs were their written analyses of the two focal technologies, using the Iceberg framework as a point of departure. To analyze those qualitative responses, we began with a qualitative content analysis phase (Schreier, 2012) to create descriptive and evaluative categories that captured commonalities in our data set. We then used frequency counts of those codes to quantify our data for a subsequent phase of our analysis (as recommended in Teddlie & Tashakkori, 2009). The quantitative analysis allowed us to describe broader patterns in PSTs’ responses and compare their analyses of the two technologies.
Participants and Study Context
Participants in this study were 65 undergraduate PSTs enrolled in two different edtech courses taught by two of the authors in spring 2023 at two different public institutions in the southwestern and northeastern United States. We conducted this work at two sites to explore ways similar learning activities might play out across different contexts. Table 1 gives an overview of the site characteristics and participant demographics.
Table 1
Participant and Study Context Characteristics
| Site | Site/Course Details | PST Programs | Gender Identities |
|---|---|---|---|
| Southwest Site 20 Participants (1 Course Section) | Large public research university. The “Teaching with Technology’ course is typically taken 1-2 semesters before student teaching. Most PSTs also complete field experience for a separate, unrelated course. | Early Childhood Elementary Secondary Special Education | 13 female 6 male 1 nonbinary |
| Northeast Site 45 Participants (2 Course Sections) | Medium-sized public teaching college. The “Critical Media Literacy” course is typically taken two semesters before student teaching. Most PSTs also complete field experience for a separate, unrelated course. | Elementary | 39 female 6 male |
The courses at the two sites had similar roles in their respective teacher education programs, in that they occurred at similar times and are the only courses that focused on educational technology. At the Southwest site, the course is required for all teacher certification programs, from secondary education to special education. On occasion, a master’s-level student takes the course as part of teacher certification, but it is an undergraduate-level course and all students in this study were undergraduates. At the Northeast site, the course was required only for the undergraduate elementary teacher education program. We did not gather detailed demographic information, but at both sites, most PSTs identified as White and were traditional college-aged students, reflective of recent national teacher trends (National Center for Education Statistics, 2023).
We collaboratively planned a set of learning experiences to incorporate technoskepticism into the courses at both sites in similar ways. In an introductory activity, technoskepticism was defined and the Iceberg framework dimensions were presented and applied to an everyday technology to illustrate how the framework can reveal its myriad effects. PSTs then worked in small groups to practice applying the framework to a classroom technology. After that initial introduction, at regular intervals throughout the semester, we engaged PSTs in a sequence of two types of technoskeptical inquiry tasks. They included case study tasks, in which PSTs had to take a position on an issue involving technology and schools (e.g., ensuring internet access), and iceberg tasks, in which PSTs analyzed classroom technologies using the Iceberg framework (e.g., Class Dojo).
For this study, we focused on the latter task type. Both research sites followed the same overall timing and format for the Iceberg tasks but addressed slightly different sets of classroom technologies. TPT and Class Dojo were used for Iceberg tasks in both courses, which is one reason why we focused on those technologies for the study. In addition, these technologies are salient because they are both common in schools and have been the subject of prior critical edtech research (Manolev et al., 2019; Rodriguez et al., 2020; Shelton et al., 2022; Williamson, 2017b).
Each Iceberg task briefly presented background information about the technology in question. Then, in groups of three to five, PSTs used the Iceberg framework to generate technical, psychosocial, and political questions they wanted to explore about the technology. PSTs investigated their questions using internet sources in their groups and then shared their findings during a whole-class discussion. As instructors, we took a facilitator role, in that we helped PSTs pursue their lines of inquiry and clarified definitions of the Iceberg framework, as needed. Yet, we were careful not to impose our own assessments of the technologies and made sure to encourage all investigations and conclusions.
After class, each PST completed a homework assignment, in which they composed a written reflection (approximately one to two paragraphs) that summarized their perceived benefits and issues with the technology and explained how they would use it (or not) in their future classroom. PSTs’ written Iceberg task reflections were used as the primary data source for this study.
Data Analysis
Our transformative mixed methods analysis (Teddlie & Tashakkori, 2009) began with a qualitative phase followed by a qualitative content analysis approach (Schreier, 2012). Aligned with our two research questions, we used this analysis to generate two sets of codes:
- a descriptive set of codes to capture the benefits and issues that PSTs identified for each technology and
- an evaluative set of codes to capture the extent to which PSTs’ responses demonstrated technoskeptical thinking.
After coding our data, our analysis moved to a quantitative phase, in which we focused on the frequencies of codes assigned during the qualitative phase. We used those frequencies to determine which arguments were most common in PSTs’ inquiries and how PSTs’ arguments regarding TPT and Class Dojo differed from one another — in terms of the kinds of arguments presented and the depth of technoskepticism exhibited.
Qualitative Phase
We created a set of descriptive codes for each technology by analyzing those data sets separately. We used an inductive process, beginning with a phase in which the three of us independently created first-cycle codes to capture claims PSTs made in their responses about the benefits and issues with the technologies (as recommended in Miles et al., 2020). We then compared and combined our respective codes to create second-cycle codes that captured common claims in PSTs’ responses, then iteratively tested and refined those codes against our data (as recommended in Schreier, 2012).
We independently used our provisional coding frame to analyze subsets of the data, met to resolve disagreements and update our coding frame as needed, and eventually reached a consensus on all codes assigned to our data set. Our final coding frameworks from each phase of analysis are described in the results section. During our descriptive analysis, an emergent theme we noticed was that a significant portion of students referred to personal experiences as students, teachers, and classroom observers with the technology during their analysis. While not itself a claim about the technology, we found those references interesting and thus noted those instances during coding.
After creating our descriptive codes, we aligned each coded claim with the Iceberg framework. While some of the claims are potentially associated with multiple dimensions and layers, we aligned each claim with the principal dimension and layer with which it was most closely related. For example, a common claim regarding Class Dojo is its potential to negatively affect students’ motivation by introducing an extrinsic reward system (dojo points). This claim is principally aligned with the Psychosocial dimension, given that it raises questions about how the technology will affect students’ thinking and acting. It is principally aligned with the Systems layer, as it considers interactions between social and technical elements of the classroom.
To create our evaluative codes, we used a similar qualitative content analysis process (Schreier, 2012), except in this case, we began with a set of ordinal a prioricategories corresponding to increasing depths of technoskeptical analysis. We used those a prioricategories as a starting point but then revised and refined them through iterative rounds of testing the categories against our data. During that process, we modified both the number of categories and the category definitions to best capture distinctions between PSTs’ responses. As with the descriptive codes, we created our final coding frame through an iterative process of independently coding segments of the data, discussing disagreements, updating our codes, and eventually coming to a consensus on the coding of the data set.
We took several steps to establish the trustworthiness of our interpretations. We utilized our different perspectives by engaging in iterative phases, during which we coded data independently before coming together to synthesize our independent findings and come to a consensus. Those collaborative processes supported the reliability of our analysis (Schreier, 2012).
Following our analysis, we shared our coding frameworks and findings with a colleague who holds expertise in the theoretical foundations of our work but was not involved in this empirical study. That colleague served as an expert check to confirm the validity of our interpretations (as recommended in Miles et al., 2020). Finally, in this paper we give detailed descriptions our analytical processes, codes, and exemplar data to provide an audit trail that transparently establishes how we came to our conclusions.
Quantitative Phase
During the quantitative phase of our analysis, we focused on the frequencies of the claims we coded during the qualitative phase. We were especially interested in determining which dimensions and layers of the Iceberg framework were most (and least) represented in PSTs’ analyses. To do this, we calculated the proportion of all coded claims aligned with each of the Iceberg dimensions (Technical, Psychosocial, and Political) and the three layers (Tools, Systems, and Values). Those proportions were calculated separately for TPT and Class Dojo, and we compared the proportions for the two technologies using two-proportion Z-tests. This strategy allowed us to determine how the characteristics of PSTs’ analyses varied by the technology in question.
We also compared the two technologies with respect to the evaluative codes to test whether PSTs demonstrated deeper technoskeptical analyses for one than the other. In this case, we used a X-squared test to determine whether the distribution of PSTs’ evaluative codes (across the ordinal categories) differed between the two technologies.
Results
To organize our findings, we first report results that addressed the first research question, which aimed to describe the claims PSTs made about the two focal technologies. Results are presented for each technology separately before the two are compared. Results are then presented pursuant to the second research question, which focused on the degree to which PSTs’ analyses demonstrate technoskepticism.
For the analyses reported here, we aggregated the results from the two study sites, as we found no statistically significant differences between the patterns of student responses. To support our quantitative findings, illustrative quotes from participants’ reflections are provided. All names reported alongside those quotes are pseudonyms.
Descriptive Results for TPT
Table 2 shows the descriptive codes that characterized PSTs’ analyses of TPT. Each code was holistically applied to a response to indicate the presence of a claim regarding the technology’s benefits or issues. We also calculated the proportion of responses assigned to each code (out of 59 responses we analyzed; six PSTs did not submit a response).
Table 2
Descriptive Codes for PSTs’ Analyses of TPT with PST Reflection Exemplars
| Claim | Proportion n = 59 | Description | Exemplar |
|---|---|---|---|
| Get Lesson Ideas | 0.54 | The platform provides valuable ideas and resources for lessons. | Platforms like Teachers Pay Teachers are easy to turn to for inspiration and guidance. They offer curriculum ideas when maybe you are at a school where there is limited support, and there aren’t resources/materials available for you to use in your classroom. (SW Site) |
| Lesson Quality | 0.44 | The lessons on the platform may not be of high quality. | TPT is a cool resource, but it is essential to use it with discretion. Teachers cannot use whatever lesson they find and be assured of quality. (SW Site) |
| Contexts | 0.39 | The lessons on the platform may not be appropriate for all contexts (e.g., state standards, cultural contexts, ages). | There are a lot of questions about if it follows state standards and if the lessons meet the needs of the children in your room. (NE Site) |
| Eases Workload | 0.34 | Using the platform saves teachers time and energy and makes their job easier. | A pro would be that it would save me time with making lessons/worksheets for little activities. (NE Site) |
| Creator Credibility | 0.34 | Because anyone can post content on the platform, there is no guarantee that creators are credible or trustworthy. | One of the biggest caveats is being unsure of who posts these materials as it often doesn't say who created them… how can teachers ensure that materials are accurate and made with good intent if they don't know who made them? (NE Site) |
| Quality Control | 0.32 | The platform does not assess the quality of the lessons. It also makes assessing quality difficult for users. | It seems like you would end up spending quite a bit of money because you can’t really preview materials to see if they are what you actually need. (SW Site) |
| Too Costly | 0.25 | The lessons may not be worth the price. | Some of the activities can be expensive. I know so many teachers who buy their own school supplies for their students so how can teachers also buy worksheets offline and still make a living? (NE Site) |
| Unfair Pay | 0.24 | Content creators only keep a fraction of the proceeds from their content sales; the platform takes an unfairly large cut. | Teachers on the website also don’t get 100% of the money. TPT takes 60% of the profits that would have gone to the teachers who made them. The website makes more than the creators do, which seems really backward and unfair. (NE Site) |
| Earn Money | 0.17 | As content creators, teachers can earn supplemental income. | I think it could be a good way to make passive income and financially benefit from the work you’re already doing. (NE Site) |
| Copyright | 0.15 | Creators can potentially post and profit from content that they did not originally produce. | There have been reports of copyright infringement and stealing lessons from other teachers to be sold on TPT. (SW Site) |
| Overuse | 0.14 | Teachers may come to rely on the platform, eroding their own creativity and expertise. | …I will not be using this in my future classroom since I believe I will end up relying on it too much. (NE Site) |
| Algorithm Bias | 0.12 | The search algorithm on the platform is not neutral and may be biased toward lessons that are not necessarily of the highest quality. | What are the first lessons that show up? The more expensive lessons because TPT wants to profit off the lessons. (NE Site) |
| Profit Motives | 0.08 | The motive to maximize profits can cause both creators and the platform itself to engage in dubious practices. | With the incentive of making money for teachers selling lesson plans, it makes me wonder if they are pumping out random lesson plans to make extra money instead of putting quality lesson plans on there. (SW Site) |
In their reflections, many PSTs discussed the benefits of the platform as a labor-saving technology (Eases Workload) and a resource for lesson planning (Get Lesson Ideas). For instance, Alice (Northeast Site) wrote, “TPT is a great resource that provides teachers with lots of tools and lessons. Teachers already have a lot of responsibilities and limited time to prep for the day or week and this tool comes in handy.”
PSTs also recognized, though, a range of possible issues with TPT. The most common issues (Lesson Quality; Contexts; and Creator Credibility) were concerns with the content that PSTs might encounter on the platform. As Joy (Northeast Site) explained, “It’s hard to know if materials you purchase are worth the money because you can’t preview it or see if the information presented is accurate. You are taking a risk with the materials found on this site.”
While still prevalent, comparatively fewer claims were made about the platform’s structure. The most common of those claims was, again, regarding lesson quality (Quality Control), but in this case, PSTs recognized the role that the platform itself played (or failed to play) in curating the resources on the site. Summer (Southwest Site) wrote, “I don’t think the website does the best job of regulating lessons on if the content even fits with the standards.” Concerns about other platform design elements, such as the algorithm (Algorithm Bias) or the monetary structure (Unfair Pair; Profit Motives), were less common. Additional PST examples are provided in Table 2.
Table 3 shows how the coded claims made by PSTs about TPT aligned with the Iceberg framework. The table shows the dimensions of the framework that are most relevant to the claim being made, with illustrative questions that connect the claims to the framework. Also shown are the layers of the Iceberg at which the claims primarily operated. For instance, the “Too Costly” claim is aligned with the Technical dimension because it primarily concerns how the system works (i.e., the price of content). It operates at the Tools layer because the focus is on the quality of the output of the technology (a lesson) and whether it is worth the required input (price). Much as one might wonder whether a pair of scissors is worth the price tag, this claim treats TPT as a technical resource-locating tool. In contrast, a claim about Algorithm Bias addresses the Values layer by examining how those values are embedded in the structure and priorities of the algorithm (thus, placing it in the Technical dimension).
Table 3
Alignment of PSTs’ Claims About TPT and the Technoskepticism Iceberg
| Claim | Proportion n = 59 | Principal Iceberg Dimension With Illustrative Question | Principal Iceberg Layer |
|---|---|---|---|
| Get Lesson Ideas | 0.54 | Psychosocial How can teachers use it while planning their instruction? | Tools |
| Lesson Quality | 0.44 | Technical Does TPT provide a good product? | Tools |
| Contexts | 0.39 | Technical Will the lessons work in my context? | Tools |
| Eases Workload | 0.34 | Psychosocial How can it alleviate the stress of teaching? | Tools |
| Creator Credibility | 0.34 | Technical Is the content produced by experts? | Tools |
| Quality Control | 0.32 | Technical What structures are in place on the platform to enable users to evaluate the quality of the content? | Systems |
| Too Costly | 0.25 | Technical Does the cost match the quality? | Tools |
| Unfair Pay | 0.24 | Political Who decides how money from content sales is distributed, and is that distribution a fair one? | Values |
| Earn Money | 0.17 | Technical How can teachers earn income from posting content? | Tools |
| Copyright | 0.15 | Political How do copyright laws constrain what creators can post? | Systems |
| Overuse | 0.14 | Psychosocial How might the platform alter the ways that teachers go about planning? | Systems |
| Algorithm Bias | 0.12 | Technical What is prioritized by the algorithm, and why? | Values |
| Profit Motives | 0.08 | Political How does the desire for profit affect how decisions are made about the design and use of the platform? | Values |
Notably, PSTs’ claims tended to focus on the Technical dimension and operated mainly at the Tools layer. We did identify several claims that addressed TPT in terms of Systems and Values, but they tended to be relatively infrequent.
Descriptive Results for Class Dojo
Table 4 shows the descriptive codes for PSTs’ claims about Class Dojo. As with TPT, PSTs noted the beneficial intended uses of the technology, pointing out that the point system, as well as the communication functions of the platform, could be useful (Benefits of Point System and Benefits of Communication). Neils (Northeast Site) said, “Class Dojo can be used to reinforce appropriate classroom behaviors and help educators with classroom management. It’s like an email or text program without providing personal information to families.”
Table 4
Descriptive Codes for PSTs’ Analyses of Class Dojo with PST Reflection Exemplars
| Claim | Proportion n = 63 | Description | Exemplar |
|---|---|---|---|
| Negative Student Experiences | 0.44 | The point system is likely to cause students to have negative experiences such as shame, embarrassment, or frustration. | Students may be shamed by their peers for doing too well or too poorly. I feel it does more harm than good to students’ self-esteem and creates an individualistic classroom environment rather than a community-building experience. (NE Site) |
| Benefits of Communication | 0.43 | The platform eases communication with parents and caretakers. | Parents are able to have a record of their student’s performance -strengths and weaknesses - both academically and behaviorally, and can contact - or be contacted by - their children’s teacher. (SW Site) |
| Student Competition | 0.40 | The point system will encourage students to compete with one another, causing negative interactions between students. | Class Dojo may spark competition, but not necessarily in a healthy manner. Points may even become an issue between students, leaving room for jealousy, bullying, teasing, embarrassment, etc. (NE Site) |
| Benefits of Point System | 0.32 | The point system can reinforce and encourage positive student behaviors. | I would use the points to help with behavior management. Also, it is important to not just provide points for “standard” behavior. It needs to be individualized based on the student. (SW Site) |
| Overinvolved Parents | 0.32 | Too much information is shared with parents, and parents may demand more information than is good for students or teachers. | I could see parents become obsessive about seeing what their child is doing in the classroom, how they are performing, and constantly having access to their grades, and I felt it could encourage helicopter parenting and mistrust in the child-parent relationship. (NE Site) |
| Undermine Motivation | 0.27 | The point system can erode intrinsic motivation by causing students to focus on points. | This could affect a child because if one doesn't have as many points as one of their classmates, they can begin to beat themself up about it and become discouraged. (NE Site) |
| Data Harms | 0.21 | Surveilling students and recording behavioral data produces harms and risks related to student privacy | I think this app brings up big issues with tracking behavior with points and being able to share all the points of the students with the class. (NE Site) |
| Inequitable | 0.14 | The point system is likely to be punitive for students in inequitable ways | It's unfair to have a standard behavior as children all face circumstances that allow for different behaviors to be acceptable. (SW Site) |
| Inaccessibility | 0.10 | The useful features of the system may not be accessible to all families. | If they do not have a way to use Class Dojo at home, can the teacher still contact them? If all students cannot use the technology, is it fair for all students to use Class Dojo? (NE Site) |
| Contexts | 0.08 | Class Dojo is not appropriate for all contexts or students (e.g., better for elementary) | I personally will not be incorporating Class Dojo into my future classroom, mostly because it is really marketed to young children, and I want to teach high school chemistry. (SW Site) |
At the same time, they often also recognized that those features could lead to adverse outcomes. PSTs most often argued that the Class Dojo point system would have negative psychological (Negative Student Experiences) and interpersonal (Student Competition) or even motivational (Undermine Motivation) impacts on students. As Alex (Northeast Site) said, “Putting the points on display creates a shame and praise type of relationship and classroom environment that can be toxic between students and teachers. Displaying this information can traumatize students and create competition.”
PSTs also often pointed out that parental communication is not an unalloyed good and that parents might become overinvolved via the platform. Ash (Northeast Site) wrote, “I would not use the point system. I would use the communication aspect [of Class Dojo], but I would try only to involve parents when necessary to prevent too much involvement or helicopter parenting.” Linking many of those claims was a common concern that Class Dojo could have negative rather than positive effects on the classroom environment.
Table 5 shows how PSTs’ claims about Class Dojo aligned with the Iceberg framework. Unlike with TPT, PSTs’ claims were overwhelmingly situated within the Psychosocial dimension. The claims were also more likely to operate at the deeper Systems layer of the iceberg.
Table 5
Alignment of PSTs’ Claims About Class Dojo and the Technoskepticism Iceberg
| Claim | Proportion n = 63 | Principal Iceberg Dimension With Illustrative Question | Principal Iceberg Layer |
|---|---|---|---|
| Negative Student Experiences | 0.44 | Psychosocial How might it alter the classroom environment in ways that lead to negative psychological impacts on students? | Systems |
| Benefits of Communication | 0.43 | Technical What can be communicated using the system? | Tools |
| Student Competition | 0.40 | Psychosocial How might competition lead to conflicts between students? | Systems |
| Benefits of Point System | 0.32 | Psychosocial How can it be used to reinforce positive behavior? | Tools |
| Overinvolved Parents | 0.32 | Psychosocial How might it alter parent-teacher-student dynamics? | Systems |
| Undermine Motivation | 0.27 | Psychosocial How might it lead to problems with student motivation? | Systems |
| Data Harms | 0.21 | Political Who decides how data are collected and made visible? | Systems |
| Reinforce Inequities | 0.14 | Psychosocial What kinds of behaviors are likely to be valued, and how will that affect students from nondominant groups? | Values |
| Inaccessibility | 0.10 | Technical What is required to access the platform? | Systems |
| Contexts | 0.08 | Technical What students does it work with? | Tools |
Table 6 compares the overall distribution of claims for the two technologies across the three Iceberg dimensions and layers. For TPT, we coded a total of 208 claims across PSTs’ responses. Table 6 shows that 66% of those responses were principally aligned with the Technical Dimension, and 70% were principally located at the Tools layer. We coded a total of 170 claims for Class Dojo that show a distinctly different distribution. To determine whether the apparent differences in distributions were statistically significant, we used a X-squared test of independence.
Table 6
Distribution of Claims Across the Iceberg Dimensions and Layers by Counts (and %)
| Framework Category | TPT | Class Dojo |
|---|---|---|
| Technical | 120 (58%) | 38 (22%) |
| Psychosocial | 60 (29%) | 119 (70%) |
| Political | 28 (13%) | 13 (8%) |
| Tools | 146 (70%) | 52 (31%) |
| Systems | 36 (17%) | 109 (64%) |
| Values | 26 (13%) | 9 (5%) |
Note that even though Political and Values claims were rare for Class Dojo, we still had adequate counts (more than five) to perform this statistical test. The results of the test supported a decision to reject the null hypothesis that the distribution of PSTs’ claims regarding TPT and Class Dojo were the same across dimensions, X2(df = 2) = 64.32, p < .0001, and layers, X2(df = 2) = 86.69, p < .0001. In other words, the differences in distributions were statistically significant.
Results From Evaluative Analysis
Our second research question concerned the extent to which PSTs’ analyses demonstrated technoskeptical thinking. Here, our analysis classified PSTs’ responses into four ordinal categories, shown in Table 7. The four levels represent an increasing depth of technoskepticism, with a significant separation occurring between Levels 2 and 3. Responses categorized as 1 or 2 remained only at the “surface” layer of the Iceberg, in that they approached the technologies as Tools. Responses categorized as 3 or 4 included analyses that addressed the deeper layers of Systems and Values.
Table 7
Coding Framework Used to Analyze Degree of Technoskepticism in PSTs’ Responses
| Level | Description | Exemplar from PST Reflection |
|---|---|---|
| 4 | The analysis focuses on and describes unintended effects of the technology. The analysis examines and explains how and why the technology will interact with relevant systems (e.g., school systems, economic systems) and values to produce those unintended/unwanted consequences. | ClassDojo rewards students with points for completing tasks programmed in by the teacher or parents, depending on whether a student’s parent pays for the extra software. The whole class’s points are put on the board for everyone to see, and students watch themselves and their peers actively gain and lose points throughout the day. The negative consequences that this incurs heavily outweigh the benefits of easily motivating students to complete tasks. For one, publicly humiliating someone by taking away their points in front of the class is more likely to discourage their learning. Another negative consequence that may happen is “Pavloving” students using this program. With points extrinsically motivating the completion of tasks, a growth mindset cannot be fostered, and significant learning can’t take place. (SW Site) |
| 3 | The analysis identifies unintended effects of the technology. The concerns are not confined to how well the technology “works” in terms of its intended effects. However, the analysis does not fully explain how the technology will interact with systems and values to produce those effects. | I think ClassDojo is a great method of managing behavior for a younger set of students, but I don’t think it is something I would use in a high school setting. Implementing this service in a high school classroom could make students feel as if they are being treated younger than their age, and I want to be able to have a more trusting relationship with them. There also comes the issue of a “point hierarchy,” as points being visible to the entire class could encourage comparison and make some students feel bad about their classroom behavior. (SW Site) |
| 2 | The analysis raises questions about the quality of the technology in relation to its intended effects (how well the technology “works”). However, the analysis does not consider any unintended effects. | Thinking about my own classroom, I would like to use Teachers Pay Teachers. I think that it is a great resource to have and can help take both teaching and learning to the next level. This could also be a beneficial resource to use in my classroom because it can give me creative ideas that I may not have thought of before. However, we learned in class that anyone can sign up for this website and sell lesson plans. Having this in mind, I will need to be extra sure to look deeply at each of the lesson plans on the website and make sure that it looks reliable before bringing it into my classroom. (NE Site) |
| 1 | The analysis describes only the intended uses and benefits of the technology. The response might consider the monetary cost of the technology but treat it as a narrow issue of cost-versus-benefit. | I believe Teachers Pay Teachers is a great resource for teachers everywhere. Teachers Pay Teachers help teachers find lessons and resources fast. Purchasing lessons and resources from TPT can help teachers pay more attention to helping their students than taking a majority of their time on coming up with and creating lessons. I have personally used TPT for past homework assignments. I also run an account with my cousin, who is currently a second-grade teacher. She uses her own lesson slide shows, worksheets, and more to upload to TPT. I help her promote her account by posting to an Instagram account I created specifically for the TPT account. It can be costly at times, depending on which type of material you are purchasing. (NE Site) |
Our descriptive results suggested that PSTs conducted deeper analyses of Class Dojo than of TPT, as many of the claims regarding TPT remained at the level of Tools. Trends in our evaluative analysis confirmed that pattern. Table 8 shows how PSTs’ analyses of the two technologies were distributed across the four levels of technoskepticism. Here as well, we used a X-squared test of independence to determine whether the differences in those distributions were statistically significant.
Table 8
Distribution of Iceberg Responses at Each Technoskepticism Level by Counts (and %)
| Technoskepticism Level | TPT | Class Dojo |
|---|---|---|
| 4 | 7 (12%) | 21 (33%) |
| 3 | 17 (29%) | 26 (41%) |
| 2 | 24 (41%) | 7 (11%) |
| 1 | 11 (19%) | 9 (14%) |
Our data met the assumptions for this test in that at least five counts were observed at each level for both technologies. The results of the X-squared test indicated that the difference in distributions was indeed statistically significant, X2(df = 3) = 18.3, p = .0004. Notably, 74% of PSTs’ Class Dojo responses were at Level 3 or 4, whereas only 41% of TPT responses achieved those levels. Most often, PSTs’ TPT responses occupied Level 2, in which the only concerns presented took a “Tools” perspective; in this case, those concerns were largely ones regarding the quality of the lessons. In contrast, most Class Dojo responses occupied Level 3, in which PSTs identified one or more unintended consequences of the technology; often described ways in which Class Dojo might undermine the classroom environment rather than simply reinforce positive behaviors.
Discussion
Technoskepticism is not a typical way of thinking about edtech (Krutka et al., 2019; Pleasants & Radloff, 2024; Schroeder & Curcio, 2022), but rather an approach that needs to be developed and reinforced over time. Overall, our findings indicated that the Iceberg framework and our series of instructional activities supported PSTs’ technoskeptical thinking, but this outcome varied based on the technology under study. The types of claims and the depth of technoskepticism present in PSTs’ analyses of TPT differed greatly from those present in their analyses of Class Dojo.
Understanding the reasons for those differences is essential to better support PSTs’ technoskeptical thinking. A core element of technoskeptical thinking is identifying the unintended consequences that might occur when introducing a technology into a classroom. This skill requires PSTs to use their imagination to think through the impacts that technologies might have, not only on classroom learning environments (e.g., interactions between students and teachers), but also on the social and cultural environments in which those classrooms are embedded (e.g., schools, families, and neighborhoods). The unique characteristics of TPT as a marketplace and Class Dojo as a behavioral system may have rendered their unintended effects easier or more challenging for PSTs to perceive.
Thinking through the unintended effects of Class Dojo may have been easier for PSTs because it overtly aims to shape the social and behavioral dynamics of the classroom environment. Those aims were obvious to PSTs and evidenced by the fact that their claims were primarily aligned with the Psychosocial dimension. It might be comparatively easy for PSTs to imagine different ways that a technology might play out when it has a clear psychosocial orientation. The PSTs, for instance, we able to describe several ways that the point and communication systems might backfire and cause negative rather than positive outcomes. In addition, unlike platforms where data collection processes are often hidden, Class Dojo records and displays student data in highly visible and public ways (Lupton, 2021; Marachi & Quill, 2020; Selwyn et al., 2021; Williamson, 2017a, 2017b), inviting PSTs to imagine the associated problems that could arise.
In contrast, analyzing the unintended consequences of a digital marketplace like TPT can be more challenging, because PSTs can easily focus on the content rather than the platform (Rodriguez et al., 2020). Most of the concerns that PSTs raised were, in one form or another, doubts about the quality of the platform’s content: the lessons. In addition to directly questioning lesson quality, PSTs also wondered if they were worth the price, if the content creators had sufficient expertise, and if the lessons would be useful in their own teaching contexts.
Appraising the quality of the lessons on TPT is undoubtedly worthwhile, and multiple researchers have documented how the quality instructional resources on online marketplaces are mixed at best (e.g., Aguilar et al., 2022; Shelton et al., 2022; Summers, 2024; Xu et al., 2023). However, fixating on the content rather than the medium misses many of the most important effects of the technology. When the products are placed in the foreground, TPT itself easily fades into the background and becomes a neutral infrastructure. However, engaging in technoskeptical analysis requires examining the platform itself and the ways that it structures and patterns both the content and the users’ interactions with that content (Nichols & Garcia, 2022).
Some of our PSTs did engage in deeper forms of technoskeptical analysis by raising questions about the structure of the TPT platform. They asked how the search algorithm might be designed to favor profitable rather than quality resources. They similarly wondered how profit motives might lead the platform designers and content creators to make questionable decisions. Ultimately, these concerns tended to circle back to the question of content quality, but in these cases, the claims identified the characteristics of the platform pattern and structure of the content that appears on it. Because relatively few of our PSTs raised these issues, our results indicate that PSTs likely require further scaffolding to analyze the platform itself.
Although we saw relatively high degrees of technoskepticism in PSTs’ analyses of Class Dojo, it is worth noting some fruitful lines of inquiry that PSTs tended not to pursue. One consequence of the technology that they tended not to consider was its potential effects on teachers; PSTs primarily focused their attention on student impacts. Because teachers manage the data inputs on the platform, Class Dojo demands a great deal of teacher labor. What might be the psychosocial impacts of those work processes?
It is also important to consider the educational values and assumptions built into the technical characteristics of the technology (only 5% of the Class Dojo claims were principally aligned with Values). The capabilities designed for Class Dojo reflect behaviorist values (via its point system) and a prioritization of recordable data (Manolev et al., 2019). While teachers can use it in multiple ways, those values inevitably play a role in the effects of Class Dojo on teachers, students, and parents.
For both Class Dojo and TPT, then, PSTs were less able to imagine and describe the mediating roles that these technologies play. That is, they tended not to consider how technologies pattern and structure classroom and school environments, in turn, affecting the decisions and interactions that occur within them. It is through those patterns and structures that the values embedded in technologies become evident (Benjamin, 2019; Broussard, 2023; Feenberg, 1999; Verbeek, 2015). The capitalist, profit-seeking logic of the TPT marketplace positions users as consumers/customers or as creators/sellers. Rather than a space where teachers come together to share instructional resources, the underlying economic values produce a commercialized environment with all its associated benefits and drawbacks (Shelton et al., 2022).
The point system of Class Dojo positions the teacher as a gatherer of student behavioral data, fostering a monitored, behaviorist environment in which certain measurable actions are assigned value while others are ignored or punished (Manolev et al., 2019; Williamson, 2017b). PSTs likely need further scaffolding to perceive these ecological effects (Decuypere, 2019).
Limitations and Directions for Further Study
As an exploratory project in a nascent research area (Henderson et al., 2023), our study raises several questions that we are not yet able to answer. First, it is crucial to understand better what experiences and conceptual resources (apart from those we introduced) PSTs draw upon when conducting their critical inquiries of classroom technologies. PSTs’ prior experiences with the technologies are undoubtedly meaningful. A handful of the PSTs in our study discussed their earlier experiences as being formative to their analyses, and other PSTs may have had experiences that they simply did not mention.
Less clear is precisely what role PSTs’ prior experiences played in shaping their thinking. For one, those experiences came in different forms. Some PSTs, for instance, experienced Class Dojo as students, whereas others observed Class Dojo as being used by mentor teachers during field experiences. In the case of TPT, some PSTs had used the platform to find lessons, while others had family members who were content creators. The types of experiences PSTs have had, the extent of those experiences, and whether those experiences were ultimately considered positive or negative are all likely to shape their thinking. PSTs may, for instance, place particular weight on observations made during their field experiences (as noted in Capobianco & Radloff, 2022). Because our study did not systematically examine our participants’ prior experiences, we were not able to explore these possibilities.
There are features of our study’s context that limited our findings. By implementing a similar set of technoskeptical inquiries in two different contexts, our aim was to test the crosscontext applicability of our instructional approach and increase the generalizability of our findings. Although we did not detect any differences in our results across our two sites, we were limited in that we had a smaller number of participants at the Southwest site (with only one section of the course). We, thus, may simply have lacked the statistical power to detect differences that may have existed. Further, our similar findings for our two sites does not mean that different results could be obtained in different contexts.
Another limitation of our study was that at both sites, we incorporated technoskepticism into only a single edtech course, situated in a much larger context of teacher preparation. While our intention is for PSTs to take the technoskeptical practices they developed in our courses with them into their future coursework and professional practice, we were not able to assess those broader impacts. Further research is necessary to assess how PSTs use technoskeptical perspectives when making decisions about edtech during classroom instruction. Future studies could also examine how technoskepticism could be woven into multiple courses (e.g., teaching methods courses) so that it is more holistically integrated into teacher preparation.
Our study was also limited by its focus on PSTs’ analyses of only two focal technologies. While those technologies were purposefully chosen, our results imply that different technologies would likely have lent themselves to different lines of critical inquiry. It would be worth examining how PSTs analyze a broader range of classroom technologies to better understand how they approach technologies with other structures and purposes. How, for instance, might PSTs appraise hardware such as smart boards or codable robots or accessibility technologies such as translation or transcription apps or instructional software such as math or language tutors? In our future work, we plan to expand the set of technologies that PSTs investigate in our courses.
Implications for Practice
Although critical engagement with technology is rare in undergraduate teacher education programs (Heath et al., 2022; Pleasants et al., 2024), our study showed one way that it can be accomplished and that critical perspectives can be fostered in PSTs. Our study indicates both the value of the Technoskepticism Iceberg as a tool for facilitating PTSs’ critical inquiries and the need for additional scaffolds.
For platforms like TPT, PSTs likely need targeted support to move past the Tools layer and consider their mediating effects on teachers and students. For example, learning management technologies such as Canvas are ubiquitous in schools and can give the appearance of being neutral platforms. Critical examinations of those technologies are essential (Boninger et al., 2019; Marachi & Quill, 202; Nichols & Garcia, 2022), but our results suggest that PSTs require guidance beyond what the Iceberg framework provides. The general-purpose categories of the Iceberg framework are valuable in their wide applicability, but conceptual resources that specifically target classroom technologies are also likely needed.
One tool we intend to use and refine as we continue our design experiment (Cobb et al., 2003) is an “audit guide” that draws PSTs’ attention to common technological features and directs them to consider issues often associated with them. We drew inspiration for this guide from the techno-ethical audits suggested by Gleason and Heath (2021) and Krutka et al. (2019).
Table 9 shows our initial version of this guide, which drew upon a wide range of critical technology and edtech scholarship. No guide can be universal or comprehensive, but our aim is that it might serve as a heuristic for PSTs to focus their attention on key areas of concern, while at the same time providing space for additional curiosities and inquiries. As we continue our work with PSTs, our intention is to study the strengths and limitations of this guide as a scaffolding tool.
Table 9
Audit Guide to Facilitate Critical Inquiries into Classroom Technology
| Feature | Potential Lines of Inquiry | Potential Issues |
|---|---|---|
| Algorithms | Identify where algorithms are used (e.g., search, automated decision-making). Investigate how those algorithms function. | -Favor certain results over others -User behavior shaped by algorithms -Algorithms replace teacher judgment |
| Data | Determine what user data are generated and where those data are stored. Investigate who can access those data and to what ends. | -Violations of student privacy -Usage of student data for profit -Equates measurability with importance |
| Model of Learning and Teaching | Describe the assumptions about how students learn and how teachers should teach that are evident when it is used “as intended.” | -Faulty or simplistic assumptions -Conflict with teacher’s principles -Reshape beliefs about teaching/learning |
| Communication | Describe the communication channels (e.g., messaging, commenting), who can use them, and which types of interactions are facilitated (and which are not). | -Encourages/discourages certain forms of student/teacher interaction -Inequitably includes/excludes students from interactions |
| Usage Requirements | Consider what is required for teachers, students, and/or parents to use it. Consider requirements that are physical, financial, linguistic, etc. | -Potential inaccessibility -Inequitable usage opportunities and challenges for students |
| Inputs | Identify the financial, material, and labor costs needed to make it function. Investigate how those costs are distributed across stakeholders. | -Directs teacher labor towards the technology rather than students -Consumes limited school resources |
Last, as we have engaged in this work, we have become more mindful of our own technological discourses and practices. As teacher educators, we need to reflect on what we are modeling to future and current teachers. This reflection means applying technoskeptical inquiry to our own classroom technology practices and communicating our decision-making processes to our PSTs. We have further begun to place greater focus on the differential impacts of technologies on students from diverse cultural, linguistic, and socioeconomic backgrounds (as recommended by Benjamin, 2019; Broussard, 2023). While rarely considered by PSTs in our study, those issues are essential areas of emphasis in order to prepare PSTs for work in diverse classrooms. It is through ongoing and coherent educational approaches that we are most likely to cultivate critical perspectives in our future teachers.
References
Aguilar, S. J., Silver, D., & Polikoff, M. S. (2022). Analyzing 500,000 TeachersPayTeachers.com lesson descriptions shows focus on K-5 and lack of Common Core alignment. Computers and Education Open, 3, 100081. https://doi.org/10.1016/j.caeo.2022.100081
Benjamin, R. (2019). Race after technology: Abolitionist tools for the new Jim Code. Polity.
Boninger, F., Molnar, A., & Saldaña, C. (2019). Personalized learning and the digital privatization of curriculum and teaching. National Education Policy Center. https://nepc.colorado.edu/publication/personalized-learning.
Broussard, M. (2023). More than a glitch: Confronting race, gender, and ability bias in tech. MIT Press.
Capobianco, B. M., & Radloff, J. (2022). Elementary preservice teachers’ trajectories for appropriating engineering design–based science teaching. Research in Science Education, 52(5), 1623-1641. https://doi.org/10.1007/s11165-021-10020-y
Cardona, M. A., Rodriguez, R. J., & Ishmael, K. (2023). Artificial intelligence (AI) and the
future of teaching and learning: Insights and Recommendations. U.S. Department of Education
Office of Educational Technology. https://www2.ed.gov/documents/ai-report/ai-report.pdf
Chachra, D. (2023). How infrastructure works: Inside the systems that shape our world. Riverhead Books.
Ciccone, M. (2022). Surveillance and the edtech imaginary via the mundane stuff of schooling. In B. S. De Abreu (Ed.), Media literacy, equity, and justice (pp. 197-205). Routledge.
Cobb, P., Confrey, J., DiSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13. https://doi.org/10.3102/0013189X032001009
Decuypere, M. (2019). Researching educational apps: Ecologies, technologies, subjectivities and learning regimes. Learning, Media and Technology, 44(4), 414-429. https://doi.org/10.1080/17439884.2019.1667824
Degand, D. (2020). Introducing critical race media literacy in an undergraduate education course about technology and arts-based inquiry. International Journal of Multicultural Education, 22(3), 96–117. https://doi.org/10.18251/ijme.v22i3.2461
Dixon-Román, E., Nichols, T. P., & Nyame-Mensah, A. (2020). The racializing forces of/in AI educational technologies. Learning, Media and Technology, 45(3), 236-250. https://doi.org/10.1080/17439884.2020.1667825
Doan, S., Eagan, J., Grant, D., & Kaufman, J. H. (2023). American instructional resources surveys: 2023 technical documentation and survey results. RAND. https://www.rand.org/pubs/research_reports/RRA134-20.html
Ertmer, P. A., & Newby, T. J. (2016). Learning theory and technology: A reciprocal relationship. In R. Rushby & D. W. Surry (Eds.), The Wiley handbook of learning technology (pp. 58-76). Wiley. https://doi.org/10.1002/9781118736494.ch4
Feenberg, A. (1999). Questioning technology. Routledge.
Garcia, A., & Nichols, T. P. (2021). Digital platforms aren’t mere tools—they’re complex environments. Phi Delta Kappan, 102(6), 14-19. https://doi.org/10.1177/0031721721998148
Gleason, B., & Heath, M. K. (2021). Injustice embedded in Google Classroom and Google Meet: A techno-ethical audit of remote educational technologies. Italian Journal of Educational Technology, 29(2), 26-41. https://doi.org/10.17471/2499-4324/1209
Granić, A., & Marangunić, N. (2019). Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology, 50(5), 2572–2593. https://doi.org/10.1111/bjet.12864
Hall, J., Roman, C., Jovel-Arias, C., & Young, C. (2020). Pre-service teachers examine digital equity amidst schools’ COVID-19 responses. Journal of Technology and Teacher Education, 28(2), 435–442. https://www.learntechlib.org/p/216180/
Heath, M., Asim, S., Milman, N., & Henderson, J. (2022). Confronting tools of the oppressor: Framing just technology integration in educational technology and teacher education. Contemporary Issues in Technology and Teacher Education, 22(4), 754-777. https://citejournal.org/volume-22/issue-4-22/current-practice/confronting-tools-of-the-oppressor-framing-just-technology-integration-in-educational-technology-and-teacher-education
Heath, M. K., & Segal, P. (2021). What pre-service teacher technology integration conceals and reveals: “Colorblind” technology in schools. Computers & Education, 170, 104225. https://doi.org/10.1016/j.compedu.2021.104225
Henderson, J., Milman, N. B., Asim, S., & Heath, M. K. (2023, April). A literature synthesis examining critical educational technology teacher education research [Paper presentation]. American Educational Research Association Conference 2023, Chicago, IL.
Kimmons, R., Graham, C. R., & West, R. E. (2020). The PICRAT model for technology integration in teacher preparation. Contemporary Issues in Technology and Teacher Education, 20(1), 176-198. https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model-for-technology-integration-in-teacher-preparation
Koehler, M., & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. https://citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge
Krutka, D. G., Heath, M. K., & Willet, K. B. S. (2019). Foregrounding technoethics: Toward critical perspectives in technology and teacher education. Journal of Technology and Teacher Education, 27(4), 555-574. https://www.learntechlib.org/p/208235/
Ladson-Billings, G. (2021). I’m here for the hard re-set: Post pandemic pedagogy to preserve our culture. Equity & Excellence in Education, 54(1), 68–78. https://doi.org/10.1080/10665684.2020.1863883
Lupton, D. (2021). “‘Honestly no, I’ve never looked at it’: Teachers’ understandings and practices related to students’ personal data in digitised health and physical education.” Learning, Media and Technology, 46(3), 1–13. https://10.1080/17439884.2021.1896541
Macgilchrist, F. (2021). What is ‘critical’ in critical studies of edtech? Three responses. Learning, Media and Technology, 46(3), 243-249. https://doi.org/10.1080/17439884.2021.1958843
Manolev, J., Sullivan, A., & Slee, R. (2019). The datafication of discipline: ClassDojo, surveillance and a performative classroom culture. Learning, Media and Technology, 44(1), 36-51. https://doi.org/10.1080/17439884.2018.1558237
Manolev, J., Sullivan, A., & Tippett, N. (2024). Reshaping school discipline with metrics: An examination of teachers’ disciplinary practices with ClassDojo. British Journal of Sociology of Education, 1-15. https://doi.org/10.1080/01425692.2024.2407874
Marachi, R., & Quill, L. (2020). The case of Canvas: Longitudinal datafication through learning management systems. Teaching in Higher Education, 25(4), 418-434. https://doi.org/10.1080/13562517.2020.1739641
Miles, M. B., Huberman, M. A., & Saldaña, J. (2020). Qualitative data analysis: A methods
sourcebook (4th ed.). Sage Publications.
National Center for Educational Statistics. (2023). Characteristics of public school teachers. Condition of education. U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/programs/coe/indicator/clr.
Nichols, T. P., & Garcia, A. (2022). Platform studies in education. Harvard Educational Review, 92(2), 209-230. https://doi.org/10.17763/1943-5045-92.2.209
Ovide, S. (2021, June 18). “Amazon’s open secret.” The New York Times. https://www.nytimes.com/2021/06/18/technology/amazon-reviews.html
Pleasants, J., Clough, M. P., Olson, J. K., & Miller, G. (2019). Fundamental issues regarding the nature of technology: Implications for STEM education. Science & Education, 28(5), 561-597.https://doi.org/10.1007/s11191-019-00056-y
Pleasants, J., Gui, X., Krutka, D. G., Logan, C., & Heath, M. K. (2024). Coming to critical technology consciousness: A phenomenological study of educators. Learning, Media, and Technology (online first). https://doi.org/10.1080/17439884.2024.2438925
Pleasants, J., Krutka, D. G., & Nichols, T. P. (2023). What relationships do we want with technology? Toward technoskepticism in school. Harvard Educational Review, 93(4), 486-515. https://doi.org/10.17763/1943-5045-93.4.486
Pleasants, J., & Radloff, J. (2024). Promoting a critical perspective on educational technology in a pre-service teacher education course. Journal of Technology and Teacher Education, 32(2), 187-215. https://www.learntechlib.org/p/223830
Riley, J., Slay, L. E., & Revelle, C. (2021). The elephant in the classroom: Using YouTube comments to address the essential but unacknowledged topic of race. International Journal of Multicultural Education, 23(1), 131–145. https://doi.org/10.18251/ijme.v23i1.2459
Rodriguez, N., Brown, M., & Vickery, A. (2020). Pinning for profit? Examining elementary pre-service teachers’ critical analysis of online social studies resources about Black History. Contemporary Issues in Technology and Teacher Education, 20(3), 497–528. https://citejournal.org/volume-20/issue-3-20/social-studies/pinning-for-profit-examining-elementary-preservice-teachers-critical-analysis-of-online-social-studies-resources-about-black-history
Schreier, M. (2012). Qualitative content analysis in practice. Sage.
Schroeder, S., & Curcio, R. (2022). Critiquing, curating, and adapting: Cultivating 21st-century critical curriculum literacy with teacher candidates. Journal of Teacher Education, 73(2), 129–144. https://doi.org/10.1177/00224871221075279
Selwyn, N. (2016). Is technology good for education? Polity Press.
Selwyn, N. (2022). The future of AI and education: Some cautionary notes. European Journal of Education, 57(4), 620–631. https://doi.org/10.1111/ejed.12532
Selwyn, N. (2024). The modern classroom chair: Exploring the ‘coercive design’ of contemporary schooling. Power and Education, 16(1), 63-77. https://doi.org/10.1177/17577438231163043
Selwyn, N., Pangrazio, L., & Cumbo, B. (2021). Attending to data: Exploring the use of attendance data within the datafied school. Research in Education, 109(1), 72-89. https://doi.org/10.1177/0034523720984200
Shelton, C. C., Koehler, M. J., Greenhalgh, S. P., & Carpenter, J. P. (2022). Lifting the veil on TeachersPayTeachers.com: An investigation of educational marketplace offerings and downloads. Learning, Media and Technology, 47(2), 268-287. https://doi.org/10.1080/17439884.2021.1961148
Sims, C. (2017). Disruptive fixation: School reform and the pitfalls of techno-idealism. Princeton
University Press.
Summers, R. (2024). Appraising instructional materials from TeachersPayTeachers for features of NGSS design and nature of science representations. Research in Science Education, 54(3), 523-546. https://doi.org/10.1007/s11165-023-10146-1
Teachers Pay Teachers (2024). Who we are. https://www.teacherspayteachers.com/About-Us
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage Publications.
Verbeek, P. P. (2015). Toward a theory of technological mediation: A program for postphenomenological research. In J. K. B. O. Friis & R. P. Crease (Eds.), Technoscience and postphenomenology (pp. 189-204). Lexington Books.
Voithofer, R. (2005). Integrating service-learning into technology training in teacher preparation. Journal of Computing in Teacher Education, 21(3), 103–108. https://doi.org/10.1080/1 0402454.2005.10784519
Weisberg, L., & Dawson, K. (2023). The intersection of equity pedagogy and technology integration in pre-service teacher education: A scoping review. Journal of Teacher Education, 74(4), 327-342. https://doi.org/10.1177/00224871231182129
Williamson, B. (2017a). Big data in education: The digital future of learning, policy and practice. Sage Publications.
Williamson, B. (2017b). Decoding ClassDojo: Psycho-policy, social-emotional learning and persuasive educational technologies. Learning, Media and Technology, 42(4), 440–453. https://doi.org/10.1080/17439884.2017.1278020
Xu, F., Hewlett, D., & Correia, A. P. (2023). Exploring the landscape of computational thinking resources on teachers-pay-teachers: Resources characteristics and user evaluations. Journal of Interactive Learning Research, 34(1), 9-36. https://www.learntechlib.org/primary/p/221807/
