Thomas, A., & Edson, A. J. (2019). A framework for teachers’ evaluation of digital instructional materials: Integrating mathematics teaching practices with technology use in K-8 classrooms. Contemporary Issues in Technology and Teacher Education, 19(3). https://citejournal.org/volume-19/issue-3-19/mathematics/a-framework-for-teachers-evaluation-of-digital-instructional-materials-integrating-mathematics-teaching-practices-with-technology-use-in-k-8-classrooms

A Framework for Teachers’ Evaluation of Digital Instructional Materials: Integrating Mathematics Teaching Practices with Technology Use in K-8 Classrooms

by Amanda Thomas, University of Nebraska–Lincoln; & Alden J. Edson, Michigan State University

Abstract

The study explored the evaluation of digital instructional materials (DIMs) by K-8 teachers of mathematics, positing that a useful perspective for evaluating DIMs by K-8 teachers of mathematics is considering how technology integrates with research-based practices for teaching mathematics. This paper describes the study that drew on the documentational approach of didactics and reports on analyses of teacher-generated frameworks that encompass research-informed mathematics teaching practices combined with three levels of technology integration. Analyses revealed several themes in how technology could transform effective mathematics teaching practices: (a) from one-size fits all toward differentiating for student needs, (b) from static displays toward dynamic representations, and (c) from teacher-centered toward student-centered practices. The framework and themes offer opportunities for mathematics teacher educators to support teachers in making technology integration choices that positively impact pedagogy.

Teachers of mathematics have long engaged in the selection, modification, and enactment of published textbooks and curriculum materials (Remillard, 2005; Sherin & Drake, 2009; Son & Senk, 2014). More recently, teachers have faced increasing expectations to incorporate various technologies in the teaching and learning of mathematics (Association of Mathematics Teacher Educators, 2015; National Council of Teachers of Mathematics [NCTM], 2014). This new challenge differs from the past curriculum adoption processes, as access to digital devices and internet resources is growing.

The selection, modification, and enactment of educational resources in the 21st century often emphasizes resources drawn from or directly conveyed through technology (Usdan & Gottheimer, 2012). Curriculum analysis tools, objective measures, and adoption resources (e.g., Common Core State Standards Mathematics Curriculum Materials Analysis Project) that guide the curriculum adoption processes are available for teachers. Yet, little attention explicitly targets how technology impacts classrooms beyond appropriate use of particular mathematical technology tools such as graphing calculators, computer algebra systems, dynamic geometry, and data analysis tools.

While teachers are involved in the curriculum adoption processes at the district or state level, many teachers often evaluate and select digital resources at the classroom level. In many countries, practicing teachers often lead the selection of digital resources (Pepin, Gueudet, & Trouche, 2017; Pepin, Xu, Trouche, & Wang, 2017). This selection includes both evolving digital versions of curriculum programs and texts, as well as new emerging digital forms of educational resources. We broadly refer to this intersection of curriculum, instruction, and technology use as digital instructional materials (DIMs).

The purpose of this research was to explore how elementary/middle school teachers evaluated DIMs in mathematics. Given the power and potential of technology to impact teaching, learning, curriculum, and assessment (e.g., Lantz-Anderson, Linderoth, & Säljö, 2009; Roschelle et al., 2010), we posited that a useful perspective for evaluating DIMs is considering how technology integrates with research-based practices for teaching mathematics.

Theoretical and Conceptual Perspectives

Technology integration can be a tool for innovation in teaching and learning, and shifts to digital resources have the potential to be transformative (Choppin, Carson, Borys, Cerosaletti, & Gillis, 2014; Pepin, Gueudet, Yerushalmy, Trouche, & Chazan, 2015). Emphasizing instructional planning with technology has been shown to impact teaching and teacher knowledge more effectively than do approaches that foreground technology (Harris, Mishra, & Koehler, 2009). Mathematics teacher education and professional development must support teachers in selecting, planning, and enacting technologies that can support student learning of rich mathematics content and be incorporated within effective mathematics teaching practices (Edson & Thomas, 2016; NCTM, 2014). Thus, rather than integrating technology for technology’s sake, technology integration must serve to advance mathematics teaching and learning.

A Documentational Approach to Didactics

We theoretically framed this study in the documentational approach to didactics (Gueudet & Trouche, 2009). In the documentational approach, the selection, planning, and enactment of resources (e.g., DIMs), “is at the core of teachers’ professional activity and professional development” (p. 199). Resources combined with schemes of use comprise a process of documentational genesis. Teachers’ incorporation of technology as part of their documentational work occurs within a complex curricular system, including curricular objectives, instructional materials, teachers’ intentions, and enactment of mathematics curriculum in the classroom (Remillard & Heck, 2014).

From the documentational approach perspective, resources are defined as “the variety of artifacts we consider: a textbook, a piece of software, a student’s sheet, a discussion with a colleague, etc. A resource is never isolated; it belongs to a set of resources” (Gueudet & Trouche, 2009, p. 205). Through a process of genesis, teachers combine schemes of utilization with resources to produce documents (p. 205). In the documentational approach, DIMs might be viewed as a specific resource that, through schemes of utilization, become documents. While we conceptualized DIMs as the broad intersection of curriculum, instruction, and technology use, we were also interested in knowing how a sample of practitioners viewed these intersections.

Defining Digital Instructional Materials (DIMs)

Thomas and Edson (2018a) examined how elementary/middle school teachers interpreted DIMs by asking teachers to generate their own specific definitions of digital instructional materials. Findings from that study indicated that teachers used the terms technology, used, students, tool, learning, classroom, and teaching most frequently to define DIMs. Drawing from definitions established by Gueudet and Grouche (2009), Thomas and Edson (2018a) found that teachers’ definitions tended not to distinguish between the resource and the genesis through which it becomes a document. Instead, the summary of teachers’ terms identified both resources and elements of a scheme of utilization, which are indicative of documents. (p. 342)

Teacher-generated definitions implied that the interactive nature of DIMs may make consideration of resources inseparable from their use (Thomas & Edson, 2018a). In other words, evaluating DIMs is consistent with evaluating how DIMs could be used in mathematics classrooms. This implication is a departure from print instructional materials in mathematics such as textbooks, which may be defined and evaluated based on static content. Similarly, Usiskin (2013) observed that digital mathematics textbooks are opaque, when contrasted with the print, more transparent versions. This suggests that the way in which teachers conceive of DIMs acknowledges the complexity of interacting with digital resources.

Drawing from definitions established by Gueudet and Grouche (2009), Thomas and Edson found

teachers’ definitions tended not to distinguish between the resource and the genesis through which it becomes a document. Instead, the summary of teachers’ terms identified both resources and elements of a scheme of utilization, which are indicative of documents. (2018a, p. 342)

Edson, 2018a). In other words, evaluating DIMs is consistent with evaluating how DIMs could be used in mathematics classrooms. This implication is a departure from print instructional materials in mathematics such as textbooks, which may be defined and evaluated based on static content. Similarly, Usiskin (2013) observed that digital mathematics textbooks are opaque, when contrasted with the print, more transparent versions. This suggests that the way in which teachers conceive of DIMs acknowledges the complexity of interacting with digital resources.

In this study, we drew upon Thomas and Edson (2018a) to define DIMs based on teachers’ own definitions which emphasize use. From the perspectives of teachers of mathematics, DIMs refer to technologies students use as a tool for learning and/or technologies teachers use as a tool for classroom teaching. These tools may be mathematics-specific or general resources used for the purposes of mathematics teaching. A broad definition of DIMs invites teachers to evaluate and provide examples of resources that align with diverse technological knowledge, experiences, beliefs, and access to resources.

Conceptualizing the Evaluation of Digital Instructional Materials

To explore the evaluation of DIMs centered on how technology integrates with research-based practices for teaching mathematics, we were interested in familiarizing a group of K-8 teachers with relevant research literature in mathematics and technology education. Given mathematics educators’ growing attention to decomposing the complexities of mathematics teaching into activities that could be collectively discussed and practiced (e.g., Ball & Cohen, 1999; Lampert, 2001), we drew upon eight effective teaching practices set forth by the NCTM (2014). In addition, as our stance of DIMs at the onset of the project was the broad intersection of curriculum, teaching, and technology, it was important that we framed the evaluation of DIMs in a way that did not restrict the scope of what counted as technology.

We were also interested in focusing on the use of technology in classroom teaching practices, rather than the features purported in the related resources. To this end, we examined the evaluation of DIMs using two dimensions: (a) high-quality teaching practices necessary for mathematically proficient students, and (b) a model for assessing the integration of technology in classrooms. This section elaborates on the two dimensions of the framework.

Dimension 1: Effective Teaching Practices in Mathematics

For the first dimension of the framework to evaluate DIMs, we drew upon eight research-informed teaching practices articulated in Principles to Actions: Ensuring Mathematical Success for All (NCTM, 2014). The eight teaching practices and their descriptions are articulated in Appendix A. These practices represent ways in which all students could be proficient learners in mathematics (National Research Council [NRC], 2001). Specifically, NCTM (2014) identified that these teaching practices support students to have experiences that enable them to

  • engage with challenging tasks that involve active meaning making and support meaningful learning;
  • connect new learning with prior knowledge and informal reasoning and, in the process, address preconceptions and misconceptions;
  • acquire conceptual knowledge as well as procedural knowledge, so that they can meaningfully organize their knowledge, acquire new knowledge, and transfer and apply knowledge to new situations;
  • construct knowledge socially, through discourse, activity, and interaction related to meaningful problems;
  • receive descriptive and timely feedback so that they can reflect on and revise their work, thinking, and understandings; and
  • develop metacognitive awareness of themselves as learners, thinkers, and problem solvers, and learn to monitor their learning and performance. (p. 9)

Dimension 2: Using Technology to Replace, Amplify, and Transform Teaching

For the second dimension of the framework to evaluate DIMs, we were interested in a framework that responds to Wertsch’s (2007) call that “we should be on the lookout for qualitative transformation of that action rather than a mere increment in efficiency or some other quantitative change” (p. 105). Thus, we drew upon the framework of Hughes, Thomas, and Scharber (2006) of the assessment of technology integration into classrooms, whose conceptualization they based on  past research, educational theories on technology, and empirical data from classrooms. The purpose of the RAT framework is to guide examination of a technology as to whether it replaces (R), amplifies (A), or transforms (T) the teaching and learning context, as described in Table 1.

Table 1
RAT Framework (Hughes, Thomas, & Scharber, 2006) With Descriptions and Examples

IntegrationDescriptionExample
ReplacementThe technology replicates “an already functioning instructional method, learning process, or content goal in the classroom” (p. 2).Interactive whiteboards used as a “glorified whiteboard” to draw and write about graphing linear equations.
AmplificationThe technology serves to enhance or extend instructional methods, student learning processes, and curriculum goals.Interactive whiteboards used to more precisely draw graphs of linear equations and make observations about features of graphs.
TransformationThe instructional method, student learning processes, and curriculum goals are fundamentally different because of the use of technology.Using an interactive whiteboard combined with dynamic geometry affordances used to explore how graphs of linear equations change when slope and y-intercept vary.

Methods

In this study, professional learning opportunities focused on teachers’ documentational work involving DIMs (Thomas & Edson, 2018a). Professional development and preparation programs that emphasize instructional planning with digital resources align with the core of teachers’ work as theorized in the documentational approach (Gueudet & Trouche, 2009). Emphasizing planning with digital resources may also impact teaching and TPACK more effectively than more technocratic efforts (Harris et al., 2009).

The larger study from which we drew employed qualitative and quantitative methodologies to explore the posited idea that considering both research-based mathematics teaching practices and technology integration is a useful approach to teachers’ purposeful evaluation and selection of DIMs. In this paper, we report on findings from data sources that addresses the following research question: What indicators do K-8 teachers of mathematics use to describe the intersection of the two dimensions of the framework?

Research Setting and Data Sources

Eleven K-8 teachers who were enrolled in a summer graduate course focused on technology for teaching elementary mathematics consented to participate in this study. The course was taught by one of the researchers and hosted on the campus of a large, Midwestern university. For the three-credit graduate course, participants met face-to-face in a classroom for five 8-hour days, with additional out-of-classroom work extending for 2 weeks. All participants were female and taught in the local suburban and nearby rural schools, with between 2 and 12 years of classroom experience. Nine participants taught in elementary (K-5) grades, one participant taught in middle grades (6-8), and one participant was transitioning from teaching high school mathematics to middle grades in a different school.

Throughout the duration of the summer course, participants explored a variety of technology apps and tools, read and discussed research and problems of practice relating to technology use in mathematics, and worked collaboratively in one of three randomly assigned groups to develop indicators in a framework for evaluating DIMs. Appendix B summarizes the apps and resources teachers explored during the course as well as course readings. Additionally, participants explored and presented about resources of their own choosing, examined a variety of digital curriculum resources identified online at the EdSurge Math Curriculum Products Index, and chose additional technology-related articles to read and share during a jigsaw activity.

During the course, participants worked in groups of three to four to generate indicators and examples of DIMs and uses of DIMs that characterized replacement, amplification, and transformation of each of the effective mathematics teaching practices identified in Principles to Actions (NCTM, 2014) and provided written justifications of their choices. This in-class group work was audio-recorded and documented in a collaborative online document via Google Docs.

After generating indicators and examples for all cells in the theorized DIM framework, each group presented their resulting framework to the whole class, providing elaboration and reasoning for many of the indicators as well as insight into their process of completing the framework. These presentations were video-recorded. Immediately after completion of in-class coursework, participants individually used their frameworks to evaluate DIMs and articulated written plans to integrate DIMs purposefully in two math lessons. Thomas and Edson (2018b) reported findings related to participants’ individual follow-up assignments based on the group-generated DIM frameworks. Here, we focus on the in-class group assignment to generate DIM framework indicators and presentations thereof.

Analysis Procedures

To identify the indicators and descriptors teachers used to describe the intersection of mathematical teaching practices and technology integration, we began our document analysis of the teacher-generated frameworks by combining the frameworks from the three groups into a single document, using color-coded text to identify the contributions of each group. Color coding allowed us to analyze within and among groups, identifying what was common across groups and idiosyncratic to a single group. The resulting framework document combined each of the groups’ frameworks. Each row represented one of the eight effective teaching practices (NCTM, 2014), with three columns to represent the second dimension of technology integration–replacement, amplification, and transformation (as defined by Hughes et al., 2006).

We independently and systematically analyzed the document by first examining individual cells within each row. We noted general observations, commonalities, and differences among how the three groups characterized replacement, amplification, and transformation of each teaching practice.

Next, we independently analyzed each group’s contributions to each of the eight rows, documenting how groups described progressions across three levels of technology integration. Finally, we independently analyzed each column, or level of technology integration, to identify any trends in the indicators that groups provided for replacement, amplification, and transformation across the teaching practices. Following independent analyses of the cells, rows, and columns of the framework table, we discussed our observations, identifying areas of consensus and reconciling any differences. A further level of analysis involved identifying themes we observed throughout the initial analysis of cells, rows, and columns.

To better triangulate the data and support or negate these themes, we then examined groups’ written justifications and explanations of the framework, transcripts of audio recordings, and video-recordings of presentations. Our examination focused on whether the secondary data supported or refuted our primary analysis of the written frameworks. Statements that directly related to research observations and themes provided additional insight regarding participants’ conceptions and descriptions of the indicators and dimensions of the DIM evaluation framework. For example, teachers in Group 1 said that their group saw changes in the classroom roles, thereby providing additional insight into one of the themes.

We did not find statements that directly refuted the emerging themes and research observations. Thus, we report the themes and research observations that reflect analyses from the different data sources.

Results

Indicators and Descriptions in the Framework for Evaluating DIMs

Appendix C displays participants’ indicators and descriptions for the intersections of effective mathematics teaching practices and levels of technology integration. Throughout our use of the framework, we noted instances of teachers indicating the same tool for the three levels of technology integration but focusing on increasingly transformational uses of that tool. Group 1 explicitly acknowledged this phenomenon in their written justification of the framework indicators; for instance,“We noticed that certain digital materials could be used to achieve different levels of replacing, amplifying, or transforming dependent on their level of sophistication.”

Content analysis of the three groups’ indicators within the framework revealed three major themes in how teachers indicated technology integration could impact teaching practices.

Theme 1

Analysis revealed that teachers indicated technology tools and uses that might transform Teaching Practice 1 (TP1), TP6, TP7, and TP8, from a one-size-fits-all approach toward more differentiation for individual student needs. This trend was particularly apparent in TP1 (establish mathematical goals to focus learning), as teachers suggested tools and indicators that replace the display of objectives or goals for all students in a lesson, amplify TP1 by using technology to monitor individual students’ progress toward established goals, and transform by tailoring or articulating goals to individual students’ needs (Figure 1).

Figure 1
Figure 1. Summary of replacing, amplifying, and transforming TP1.

With respect to TP8, participants explained how technology might help transform eliciting and using evidence of student thinking by providing data that facilitates in-the-moment changes on a group or individual level. For example,

Something that was transformative would allow you to change what you’re doing based on, “Ok the whole group isn’t getting it,” or change instruction for one student if they were struggling or needed enrichment in that area. (Group 1, video presentation of framework).

Theme 2

Analysis of TP1, TP2, TP3, TP4, TP5, TP6, and TP7 consistently indicated a shift from using technology to project or display static, often textual images, toward using dynamic and interactive technologies to transform the teaching of a practice. Referring to the transformation of TP5 (posing purposeful questions), participants explained, “In transforming we felt that “three-act problems” and interactive presentations (through NearPod or PearDeck) were more interactive and you could pose meaningful questions at any time we felt it was needed” (Group 2, written justification of framework).

Figure 2 summarizes participants’ articulation of this notion of transformation with respect to using and connecting mathematical representations (TP3). In describing the spectrum of replacement to transformation for TP3, Group 2 focused on a shift from merely showing representations to using and connecting among multiple representations. Analysis further suggested that TP3 progressed from displaying a representation (TP3-Replacement), to displaying or interacting with multiple representations (TP3-Amplification), toward connecting and discussing among multiple or dynamic representations (TP3-Transformation). In the overall justification of their framework, Group 3 referred to static representations in the replacement category, “The replacement category included technologies that we could use as basic functions such as interactive whiteboards and document cameras,” whereas amplification technology integration included more dynamic representations, “that provide discussion among students or models that they can manipulate” (Group 3, written justification of framework).

FIgure 2
Figure 2. Summary of replacing, amplifying, and transforming TP3.

Theme 3

Analysis suggested that technology might be used to transform from teacher-centered pedagogy to more student-centered pedagogy. One group explicitly identified this conception of transformation, “We see transformation as changing the roles of the classroom from teacher-centered to student-centered”(Group 1). This progression of technology use was observed particularly with respect to TP1, TP3, TP4, TP5, TP6, and TP8. For instance, when considering how technology could replace, amplify, or transform posing of purposeful questions, indicators shifted from digital ways of posting or displaying teacher questions (TP5-Replacement) to transforming the posing of questions by positioning students to generate their own mathematical questions (TP5-Transformation).

In their reflection, the same group wrote, “Within this category [transformation], teachers facilitate instruction by allowing students to be in charge of their own learning, motivating them to ask questions that deepen their thinking, and using justifications to challenge others and make their thinking visible” (Group 1).

Furthermore, participants indicated how the same Base Ten Block tool could be used to replace physical manipulatives (TP6-Replacement) during typical teacher-led instruction or to amplify if the teacher used the app to display student work with virtual manipulatives (TP6-Amplification). It could be used to transform to more student-led instruction if students used the app while leading discussion about concepts that might build toward procedural fluency (TP6-Transformation). This progression is illustrated in Figure 3.

Figure 3
Figure 3. Summary of replacing, amplifying, and transforming with Base Ten Blocks app.

Discussion

Overall, results supported our posited idea that the eight effective teaching practices and RAT framework on technology integration are useful dimensions for teachers’ purposeful evaluation and selection of DIMs. Through this lens, teachers were able to generate numerous ideas for purposeful technology integration that served mathematics teaching goals. All three themes that emerged from teachers’ ideas described the potential for positive transformations of effective teaching mathematics practices through technology integration. Furthermore, written justifications for each group’s framework articulated usefulness for teachers’ own practice. For instance:

Prior to this experience, we were not aware of a framework that could evaluate the technology we were using. We now feel like we have an effective tool to take back to our classrooms and use to support our use of technology in math instruction. (Group 1)

Teachers experienced challenges in the process of generating indicators. Participants identified that some teaching practices seemed more difficult to transform than others. For instance, all groups described difficulties in articulating how TP1 (establish mathematical goals to focus learning) might be replaced, amplified, or transformed through the use of technology. Teachers found some practices difficult to consider in isolation, for example, TP4 (facilitate meaningful mathematical discourse) and TP2 (implement tasks that promote reasoning and problem solving). Consequently, the resulting framework includes duplication across categories.

In considering overlap and duplication of technologies across teaching practices, participants relied not only on the tools themselves, but on how they could be used for different teaching purposes. The transcript of discussion among Group 1 articulated participants’ challenges with overlap among categories: “I think that they are meant to overlap? Am I the only one who thinks that? I think that a lot of these overlap…These tools serve a lot of different purposes.” Similarly, Group 2 wrote, “We discovered that many tools can fall under multiple categories depending on how they are used within instruction (written justification of framework).”

The third group also emphasized technologies according to their use: “Some technologies fit into multiple categories depending on the way in which they are used within the classroom (Group 3, written justification of framework).” The intermingling of technologies and technology use while evaluating DIMs with this framework corresponds with how teachers defined DIMs (Thomas & Edson, 2018a), further suggesting that technology resources may often be considered inseparable from their use.

Teachers often provided specific examples of tools or apps as indicators of how a teaching practice could be transformed with technology. Some of these specific tools and apps were repeatedly identified as indicators for multiple practices and technology integrations, whereas other specific technologies appeared once in the framework. With the exception of Dan Meyer’s 3-Act resources and LearnZillion, indicators did not reference digital textbooks or collections of materials and resources. That is, teachers did not identify any magical sandbox or single collection of resources that spanned all dimensions of the framework.

While teachers identified that examples helped them conceive of how practices could be amplified or transformed through technology, emphasis was placed on creating a framework and indicators that could apply across current and emerging DIMs. As one group wrote,

As new technology is created or as more tools become available, the framework would need to be edited to reflect these changes. The descriptors of each mathematical teaching are effective, but more tools could be added or the way that tools are used could change. (Group 1, written justification of framework)

Implications

We hypothesized that a two-dimensional framework composed of replacement-amplification-transformation technology integration levels (Hughes et al., 2006) and effective mathematics teaching practices (NCTM, 2014) would be a productive lens for teachers’ evaluation of DIMs for teaching mathematics. Findings from the resulting teacher-generated framework supported this conjecture, resulting in a framework with implications for both research and practice in mathematics education.

Teachers in this study were readily able to utilize the theorized framework to generate examples and indicators for purposeful technology use that aligns with research-backed mathematical teaching practices. Themes across these indicators also revealed transformative possibilities for how technology could be used to support mathematics teaching. This result aligns with research by Pepin et al. (2015) and Choppin et al. (2014), suggesting that a shift toward digital resources has the potential to be transformative.

Furthermore, the teachers engaged in the work examined in this study envisioned ways in which a wide variety of DIMs (both mathematics-specific and general use) could be used in the classroom. When viewed through the theoretical lens of the documentational approach, the teachers’ brought to bear their schemes of use as they considered and planned for using digital resources. Findings from this study lend further methodological and empirical support to the documentational approach to didactics theorized by Gueudet and Trouche (2009) and utilized by researchers worldwide to examine and understand the work of teachers (see, for instance, Gitirana, Miyakawa, Rafalska, Soury-Lavergne, & Trouche, 2018).

In this study, teachers had an opportunity to focus on technology for teaching mathematics and collaborate to consider and evaluate DIMs. In general, teachers of mathematics need similar opportunities not only to learn about technologies (mathematical and general), but purposefully to weigh their usefulness in relation to practices for teaching mathematics. The DIM framework in this study provides one possible approach for doing so.

The framework in Appendix C includes indicators and examples that teachers and teacher educators may find useful for their own classrooms. Moreover, the two-dimensional DIM framework could be used as a flexible tool for evaluating a variety of current and emerging digital resources that could be used for teaching mathematics. As practicing teachers assume leadership in selecting digital resources for mathematics classrooms (Pepin et al., 2017; Pepin, Xu, Trouche, & Wang, 2017), incorporating the DIM framework in mathematics teacher preparation and professional development may help prepare and support teachers in this work.

The DIM framework is readily available to incorporate in practice, engaging teachers in planning and selection of digital resources which the documentational approach positions as the crux of teachers’ professional work. When given the two dimensions of the framework and an opportunity to work collaboratively to create their own indicators and examples, teachers can leverage their authentic knowledge, beliefs, experiences, and access to technologies, together with schemes of use, in a documentational approach to evaluating DIMs (Gueudet & Trouche, 2009). Findings in this study suggest that engagement with the DIM framework may generate discussion opportunities around the transformational possibilities of technology, as well as big ideas that exist with and without technology.

In addition to discussions about technology itself, teacher educators should be prepared to facilitate contextualized discussions of differentiating for student needs, using and connecting among mathematical representations, and student-centered teaching, in order to support the transformative potential of DIMs (Choppin et al., 2014). Just as curriculum materials have the potential to impact teachers’ pedagogy (Herbel-Eisenmann, Lubienski, & Id-Deen, 2006), findings from this study also suggest that planning for practice-focused technology integration might also have the potential to transform teaching toward more student-centered practice. Activities that engage preservice and in-service teachers in evaluation and lesson planning with DIMs, particularly to support high-quality curriculum materials may be productive for supporting shifts toward student-centered pedagogy and differentiated instructional practices.

Conclusion

While this study provides some empirical evidence that the framework holds promise for helping teachers evaluate DIMs and make technology integration choices that positively impact pedagogy, the study was limited to a small number of teachers and a fixed set of resources, so additional work is necessary. Further research is needed to examine the usefulness of the framework among teachers in a variety of contexts, as well as the utility of the evaluation framework for teachers who do not engage in developing the indicators.

Additional research is also needed to understand better how teachers interpret and attend to the overlap participants identified among categories in the two dimensions of the DIM framework. Future work with larger samples of teachers will help researchers refine the framework, potentially developing subcategories for the two dimensions of the framework to help teachers more effectively evaluate the pedagogical affordances of DIMS. While this study provided evidence of teachers’ intended use of DIMs, further study is also needed to determine the extent to which using this framework might impact how teachers enact DIMs in the classroom.

Findings from this study pose important implications for teachers, researchers, and mathematics teacher educators. Considering and evaluating DIMs through the lens of teaching practices and technology integration was a productive activity for teachers and generated an evaluation framework that teachers could use to examine DIMs for their own classrooms. Given the increase in and diversity of digital resources (Choppin et al., 2014), equipping teachers with tools for evaluating DIMs is important.

We offer an approach that foregrounds technology and teaching, but research should explore and consider additional tools for evaluating DIMs. Whereas technology in mathematics education has traditionally focused on specific mathematical tools, it is essential that mathematics teacher educators prepare and support teachers to evaluate and enact digital resources that move beyond strictly mathematical tools (Edson & Thomas, 2016).  While additional research is needed, the DIM evaluation framework may also inform classroom observations, offering potential “look fors” when teachers integrate technology with current practices.

Author Note

A prior version of this paper received the 2017 NTLI Fellowship Award from the Association for Mathematics Teacher Education and can be found in the 2017 SITE Conference Proceedings.

References

Association of Mathematics Teacher Educators. (2015). Position of the Association of Mathematics Teacher Educators on Technology. Retrieved from https://amte.net/sites/default/files/technologypositionstatement-nov2015.pdf

Ball, D.L., & Cohen, D.K. (1999). Developing practice, developing practitioners: Towards a practice-based theory of professional education. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession (pp. 3-22). San Francisco, CA: Jossey-Bass.

Choppin, J., Carson, C., Borys, Z., Cerosaletti, C., & Gillis, R. (2014). A typology for analyzing digital curricula in mathematics education. International Journal of Education in Mathematics, Science and Technology, 2(1), 11-25.

Edson, A.J., & Thomas, A. (2016). Transforming preservice mathematics teacher knowledge for and with the enacted curriculum: The case of digital instructional materials. In M. Niess, S. Driskell, & K. Hollebrands (Eds.), Handbook of research on transforming mathematics teacher education in the digital age (pp. 215-240). Hershey, PA: IGI Global.

Gitirana, V., Miyakawa, T., Rafalska, M., Soury-Lavergne, S., & Trouche, L. (Eds.). (2018). Proceedings of the Re(s)sources 2018 International Conference. Lyon, France: ENS de Lyon.

Gueudet, B., & Trouche, L. (2009). Towards new documentation systems for mathematics teachers? Educational Studies in Mathematics. 71(3), 199–218.

Harris, J., Mishra, P., & Koehler, M. (2009). Teachers’ technological pedagogical content knowledge and learning activity types: Curriculum-based technology integration reframed. Journal of Research on Technology in Education, 41(4), 393-416. doi: 10.1080/15391523.2009.10782536

Herbel-Eisenmann, B. A., Lubienski, S. T., & Id-Deen, L. (2006). Reconsidering the study of mathematics instructional practices: The importance of curricular context in understanding local and global teacher change. Journal of Mathematics Teacher Education, 9(4), 313–345.

Hughes, J., Thomas, R., & Scharber, C. (2006). Assessing technology integration: The RAT- replacement, amplification, and transformation-framework. In C. Crawford, R. Carlsen, K. McFerrin, J. Price, R. Weber, & D. Willis (Eds.), Proceedings of SITE 2006–Society for Information Technology & Teacher Education International Conference (pp. 1616-1620). Orlando, FL: Association for the Advancement of Computing in Education.

Lampert, M. (2001). Teaching problems and the problems of teaching. New Haven, CT: Yale Press.

Lantz-Anderson, A., Linderoth, J., & Säljö, R. (2009). What’s the problem? Meaning making and learning to do mathematical word problems in the context of digital tools. Instructional Science, 37(4), 325–343.           

National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring mathematical success for all. Reston, VA: Author.

National Research Council. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press.

Pepin, B., Gueudet, G., & Trouche, L. (2017). Refining teacher design capacity: Mathematics teachers’ interactions with digital curriculum resources. ZDM, 49(5), 799–812

Pepin, B., Gueudet, G., Yerushalmy, M., Trouche, L., & Chazan, D. (2015). e-Textbooks in/for teaching and learning mathematics: A disruptive and potentially transformative educational technology. In L. English & D. Kirshner (Eds.), Handbook of international research in mathematics education (pp. 636–661). New York, NY: Taylor & Francis.

Pepin, B., Xu, B., Trouche, L., & Wang, C. (2017). Developing a deeper understanding of mathematics teaching expertise: An examination of three Chinese mathematics teachers’ resource systems as windows into their work and expertise. Educational Studies in Mathematics, 94(3), 257-274.

Remillard, J. T. (2005). Examining key concepts in research on teachers’ use of mathematics curricula. Review of Educational Research, 75(2), 211–246.

Remillard, J.T., & Heck, D.J. (2014). Conceptualizing the curriculum enactment process in mathematics education. ZDM, 46(5), 705–718.

Roschelle, J., Schechtman, N., Tatar, D., Hegedus, S., Hopkin, B., Empson, S., & Gallagher, L. (2010). Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies. American Educational Research Journal, 47(4), 833–878.

Sherin, M., & Drake, C. (2009). Curriculum strategy framework: Investigating patterns in teachers’ use of a reform-based elementary mathematics curriculum. Journal of Curriculum Studies, 41(4), 467–500.

Son, J.W., & Senk, S.L. (2014). Teachers’ knowledge and the enacted curriculum. In D. R. Thompson & Z. Usiskin (Eds.), Enacted mathematics curriculum: A conceptual framework and research needs (pp. 75-95). Charlotte, NC: Information Age Press.

Thomas, A., & Edson, A.J. (2018a). An examination of teacher-generated definitions of digital instructional materials in mathematics. In V. Gitirana, T. Miyakawa, M. Rafalska, S. Soury-Lavergne, & L. Trouche (Eds.), Proceedings of the Re(s)sources 2018 international conference. (pp. 340-343). Lyon, France: ENS de Lyon.

Thomas, A., & Edson, A.J. (2018b). Integrating mathematics teaching with digital resources: Where to begin? In H.-G. Weigand, A. Clark-Wilson, & A. Donevska (Eds.), Proceedings of the 5th ERME Topic Conference: Mathematics education in the digital age. (pp. 281-288). Copenhagen, Denmark: University of Copenhagen.

Usdan, J., & Gottheimer, J. (2012). FCC Chairman: Digital textbooks to all students in five years. Retrieved from: http://www.fcc.gov/blog/fcc-chairman-digital-textbooks-all-students-five-years

Usiskin, Z. (2013) Studying textbooks in an information age—a United States perspective. ZDM, 45(5), 713-723.

Wertsch, J.V. (2007). Computer mediation, PBL, and dialogicality. Distance Education, 23(1), 105–108.


Appendix A
Mathematics Teaching Practices and Descriptions (adapted from NCTM, 2014, p. 10)

Effective Teaching PracticeDescription
Establish mathematics goals to focus learning.Effective teaching of mathematics establishes clear goals for the mathematics that students are learning, situates goals within learning progressions, and uses the goals to guide instructional decisions.
Implement tasks that promote reasoning and problem solving.Effective teaching of mathematics engages students in solving and discussing tasks that promote mathematical reasoning and problem solving and allow multiple entry points and varied solution strategies.
Use and connect mathematical representations.Effective teaching of mathematics engages students in making connections among mathematical representations to deepen understanding of mathematics concepts and procedures and as tools for problems solving.
Facilitate meaningful mathematics discourse.Effective teaching of mathematics facilitates discourse among students to build shared understanding of mathematical ideas by analyzing and comparing student approaches and arguments.
Pose purposeful questions.Effective teaching of mathematics uses purposeful questions to assess and advance students’ reasoning and sense making about important mathematical ideas and relationships.
Build procedural fluency from conceptual understanding.Effective teaching of mathematics builds fluency with procedures on a foundation of conceptual understanding so that students, over time, become skillful in using procedures flexibly as they solve contextual and mathematical problems.
Support productive struggle in learning mathematics.Effective teaching of mathematics consistently provides students, individually and collectively, with opportunities and supports to engage in productive struggle as they grapple with mathematical ideas and relationships.
Elicit and use evidence of student thinking.Effective teaching of mathematics uses evidence of student thinking to assess progress toward mathematical understanding and to adjust instruction continually in ways that support and extend learning.

Appendix B
Apps, Resources, and Readings in Graduate Course


Course Component: Technology Apps and Resources

Osmo Numbers & Tangrams

TouchCounts

LearnZillion

Tiggly Chef, Cardtoons & Addventures

Pieces Basic by Math Learning Center

ToDo Math

Xyla & Yabu

Solve Me Mobiles

Braining Camp Fractions

Math Shake

Llama Drama

ActivePrompt

Screencasting apps (e.g., ShowMe, Educreations)

National Library of Virtual Manipulatives

Braining Camp suite of virtual manipulatives

MTBoS Scavenger Hunt

Course Component: Readings

Blackboard. (2015). Trends in Digital Learning: Empowering Innovative Classroom Models for Learning. Project Tomorrow.

Clements, D.H. & Sarama. J. (1997). Computers support algebraic thinking. Teaching Children Mathematics, 3, 320-25.

Drijvers, P., Doorman, M., Boon, P., Reed, H., & Gravemeijer, K. (2010). The teacher and the tool: instrumental orchestrations in the technology-rich mathematics classroom. Educational Studies in Mathematics, 75, 213-234.

Foughty, Z., & Keller, J. (2011). Implementing digital math curricula. Principal Leadership, 64-66.

Goldenberg, E.P. (2000). Thinking (and talking) about technology in math classrooms. The K-12 Mathematics Curriculum Center: Issues in Mathematics Education. Waltham, MA, USA: Education Development Center, Inc.

Hughes, J., Thomas, R., & Scharber, C. (2006). Assessing Technology Integration: The RAT- Replacement, Amplification, and Transformation-Framework. In C. Crawford, R. Carlsen, K. McFerrin, J. Price, R. Weber & D. Willis (Eds.), Proceedings of SITE 2006–Society for Information Technology & Teacher Education International Conference (pp. 1616-1620). Orlando, Florida, USA: Association for the Advancement of Computing in Education (AACE).

Kang, C. (February 16, 2016). Bridging a digital divide that leaves schoolchildren behind. The New York Times

McGinty, J.C. (May 23, 2016). Calculators in class: Use them or use them? The Wall Street Journal

NCTM (2014). Principles to Actions: Ensuring Mathematical Success for All.  Reston, VA, USA: Author.

Rochelle, J.M., Pea, R.D., Hoadley, C.M., Gordin, D.N., & Means, B.M. (2000). Changing how and what children learn in school with computer-based technologies. The Future of Children, 10(2), 76-101).

Usiskin, Z. (1978). Are calculators a crutch? The Mathematics Teacher, 71(5), 412-413.

Zbieck, R.M. & Hollebrands, K. (2008). A research-informed view of the process of incorporating mathematics technology into classroom practice by inservice and prospective teachers. In M.K. Heid and G.W. Blume (Eds). Research on Technology and the Teaching and Learning of Mathematics (pp. 287-344). Charlotte, NC, USA: Information Age Publisher.

Appendix C
Resulting Framework for Evaluating DIMs With Teacher-Generated Examples and Indicators

Loading