Listen and learn: Research paper Research papers are a pairing of two 18 minute presentations followed by 18 minutes of Discussion led by a Discussant, with remaining time for Q & A.
This is presentation 1 of 2, scroll down to see more details.
Other presentations in this group:
Design-based implementation research (DBIR; Fishman et al., 2013; LeMahieu, Nordstrum, & Potvin, 2017; Penuel et al., 2011) is a collaborative approach to research and development that includes stakeholder participation across multiple levels and groups. We chose DBIR because it is focused on addressing a shared problem of practice – teaching generalization in mathematics – and because it incorporates an iterative approach to development using implementation evidence to inform design, which will help prepare it for wide use and sustainability. The key principles of DBIR and how we addressed them in this study are:
Focusing on a persistent problem of practice with a multiple-stakeholder team. This study focused on how activity design and instructional approaches support students’ productive engagement with the activities that lead to intended cognitive processes and learning, and involve CPR2 developers, researchers, teachers, and students.
Committing to iterative, collaborative design. The team developed, implemented, revised, and re-implemented CPR2 PD and lessons based on feedback from teachers and students (Penuel, 2014).
Developing theory for learning and implementation through inquiry. While our work on CPR2 was guided by our initial theory of action, we revisited the theory continuously and updated it as evidence emerged to inform changes to it.
Developing capacity for sustaining change. So that CPR2 use can extend beyond the study period and expand to new users, we collaboratively developed the CPR2 PD materials to ensure that they are easy to use and can support new implementation of CPR2 in the future.
We conducted five DBIR sessions with the faculty who developed CPR2 and a group of experienced teachers who have delivered CPR2 instruction for years. Three in-person sessions took place in 2019-2020 and two in 2020-2021 (virtually due to COVID). The first three sessions involved the university faculty, the researchers, and three teachers who were experienced CPR2 implementers. The final two sessions had the university faculty, the researchers, two of the three teachers who participated previously, and three new teachers who were recruited from the initial implementation.
In each DBIR session, participants were expected to do some pre-work: individual reflections on questions for discussion in the whole group. The topics of each session reflected aspects of using CPR2 to address teaching generalization as a problem of practice. The COVID-19 pandemic hit near the start of the third session, which forced a shift away from the original plans for two weeks of in-person intensive PD. Therefore the third session centered around updating and tailoring the PD to a virtual setting without losing content and rigor. The DBIR group settled on one week of intensive PD and four weekly follow-up sessions with homework throughout to keep attendees engaged and learning.
The two DBIR sessions in 2020-2021 focused heavily on data from the first year’s implementation in preparation for the next cohort of teachers. The sessions addressed middle school math standards alignment with the CPR2 lessons, how to incorporate opportunities for teachers to practice teaching the content virtually, and how to generate student-centered learning environments. At the end of each session, the researchers synthesized the ideas so the group could identify actionable next steps.
Data sources on implementation used by the DBIR team included observations, focus groups, artifacts, and interviews. In the co-design year, the DBIR team reviewed results of past implementations of CPR2 in which the university faculty presented the lessons. In the initial implementation year, 10 teachers participated in the PD and 9 teachers taught CPR2 lessons during the school year. We collected data from 8 PD observations, 4 teacher focus groups, 70 daily PD feedback forms, 10 summative PD feedback forms, 9 teachers’ classroom observations across fall and spring, and 9 teacher interviews. These data were used in the last two DBIR sessions. The data allowed us to assess how well-prepared teachers unfamiliar with CPR2 and programming felt after the PD and as they taught the CPR2 lessons, as well as what other obstacles teachers faced when delivering CPR2 lessons that we may not have addressed from the first three DBIR sessions.
In the efficacy study year, 20 teachers participated in the SI and 13 teachers taught CPR2 lessons during the school year. We collected data from these sources: 8 SI observations, 4 teacher focus groups, 126 daily SI PD feedback forms, 15 summative SI PD feedback forms, 8 teachers’ classroom observations across fall and spring, and 6 teacher interviews. These data allowed us to assess the extent to which the changes we made to the PD and CPR2 lessons in the DBIR sessions resulted in improvement for teacher readiness and student learning in teaching mathematical generalization with Python as a tool—in other words, how well the theory of action worked.
As a result of the DBIR approach, several changes were made to the CPR2 PD and materials. From the beginning of the DBIR process, the team identified teachers’ programming skills and self-efficacy as the largest obstacles to strong implementation of CPR2 lessons. Generalization was already a difficult topic to teach to students, and to add into the mix programming, a skill math teachers tended to be less familiar with, was often something teachers found overwhelming. From a DBIR approach, we discussed with the participating teachers what they would need to feel more prepared in programming, and brainstormed various ways the PD could incorporate their ideas. After the session, the University of North Alabama faculty created a set of debugging activities that allowed teachers to practice identifying and fixing common Python syntax and logic errors. From data collected from the focus groups, we learned that design study teachers appreciated the daily debugging activities which helped prepare them feel confident in fixing programming errors students made.
Because participating schools had varying technology, we discussed in DBIR sessions the challenges teachers faced using Python on devices such as Chromebooks. After the sessions, the University of North Alabama faculty created a resource listing compilers for various devices. In interviews, teachers found it easy to use the resource to implement CPR2.
The University of North Alabama faculty had to shift the 2020 SI from being held in-person to via Zoom because of the pandemic. DBIR was crucial in planning to transfer intensive, in-person PD sessions into virtual sessions. The team also decided based on implementation data to shift from the original lesson structures of a couple days teaching programming only, then math only, then the conjectures only to a more integrative approach in which all three subjects were covered each day. To help ensure teachers felt comfortable in all three of the components, the team also decided that the DBIR teachers would become mentors for the design study teachers. They volunteered to lead a small group of teachers through their homework daily. In focus groups teachers said that they appreciated the mentors, found them helpful, and liked the way the PD included the mentor role throughout the PD.
After the initial implementation, DBIR sessions focused on tying math standards to CPR2 lessons and generating more student discussions. Through both DBIR sessions, the team worked to brainstorm tangible ways to change the SI and CPR2 lessons to address standards and increase student voice in the classroom. The DBIR participants created opportunities for teachers to teach CPR2 in the summer before having to lead their class of students in the CPR2 lessons. The practice teaching sessions would allow for teachers to feel more confident, feel comfortable about making mistakes, and practice strategically placing student discussions into the lessons before the school year. From feedback forms, efficacy study teachers noted that they felt confident to teach the content and appreciated the practice teaching sessions.
The DBIR sessions in this project enacted the AERA 2023 theme of interrogating consequential education research in pursuit of truth. DBIR principles and practices inherently involve interrogating data and seeking truth from multiple perspectives presented by multiple stakeholders. The iterative nature of DBIR lends itself to frequent re-interpretation of the truth that emerges from implementation data. In the session, we will share documents from our DBIR sessions so that others may implement their own.
Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. O. R. A. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education, 112(2), 136-156.
LeMahieu, P. G., Nordstrum, L. E., & Potvin, A. S. (2017). Design-based implementation research. Quality Assurance in Education, 25(1), 26-42.
Penuel, W. R. (2014). Emerging forms of formative intervention research in education. Mind, Culture, and Activity, 21(2), 97-117.
Penuel, W. R., Fishman, B. J., Cheng, B. H., and Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331-337.
Return
Trips and Tours