Fostering Equitable Teaching Practices through a Simulated Teaching Environment

Framework

This study is focused on increasing teacher self-efficacy to improve student learning. Self-efficacy is rooted in social cognitive theory (Bandura, 1977) that highlights the perspective that people are their own change agents. Teachers with a strong sense of efficacy believe student learning outcomes are within their control and are based on teacher behavior rather than outside influences (Tschannen-Moran, Hoy, & Hoy, 1998). Researchers have identified connections between teachers’ sense of efficacy, culturally responsive pedagogy (Callaway, 2016), and student achievement (Oyerinde, 2008; Tucker et al., 2005). Efforts to increase teacher efficacy are vital in remediating the low academic achievement among culturally diverse students (Callaway, 2016; Tucker et al., 2005). Highly efficacious teachers have more persistence when helping struggling students, and they create lessons designed to engage their students (Bandura, 1997; Kitsantas, 2012; Protheroe, 2008).
It is imperative that educators provide a culturally responsive environment for all students to be confident in their learning (Derman-Sparks & Ramsey, 2000). Being a culturally responsive educator first requires recognition of existing or potential bias, a metacognitive skill of teaching but these are often difficult to recognize. This study focuses on using an unobtrusive environment where underlying biases can be identified, acknowledged and remediated. A simulated teaching environment allows for comparing self-reported bias indicators to objective measures produced by teaching within the simulator.
Badiee (2012) identified four advantages to simulation based learning: (a) classroom decision-making, (b) practice through repeating, receiving feedback and advice, (c) self-efficacy in classroom teaching, and (d) collaborations and social interactions. Fischler (2006) added that simulation based learning has great potential in education by allowing educators to act within virtual environments, immediately applying theory to realistic yet controlled settings.
 SimSchool is a dynamic, online classroom simulation program that allows the opportunity to practice teaching in a safe environment for experimenting and practicing techniques, especially methods of addressing different learning needs, and wide variations in academic and behavioral performance of students. Using student profiles, teachers need to be able to plan and deliver culturally responsive instructional challenges and supports that build on the strengths of students to address their learning needs (Sianjina, 2000). Teachers need to be able to assess students, analyze the results, and enable adjustments to their instruction to ensure that all students are learning and achieving higher results (Girod & Schalock, 2002).
 SimSchool’s inference engine draws upon several instructional models and frameworks to simulate the authentic human behaviors and reactions that one experiences when teaching in simSchool (simSchool, 2018-19). These extensively researched and validated models include: 1) Cattell-Horn-Carroll Theory of Intelligence (Schneider & McGrew, 2012); 2) OCEAN model of Emotion (McCrae & Costa, 1996); 3) Interpersonal Circumplex Theory (Smith, 2013); 4) Standard models of language learning and language proficiency used to diagnose ELL students (Phakiti, Hirsh, & Woodrow, 2013); and 5) Structural functional (Case, 1993) and social constructivist theories of learning (Dweck, 1999; Vygotsky, 1962). These models are distilled into “cognitive and behavioral states” within simulated students and “cognitive and behavioral requirements” within instructional tasks. How the student performs and behaves is a direct reflection of how well-matched expectations are to students’ capabilities. Figure 1 depicts the interface of the simSchool classroom highlighting the student profiles.
Research on the use of simSchool has shown improved educator understanding in teaching skills (Christensen, Knezek, Tyler-Wood, & Gibson, 2011; Knezek, Hopper, Christensen, Tyler-Wood, & Gibson, 2015), classroom management (Christensen et al., 2007), motivation (Tyler-Wood et al., 2017), multicultural awareness (Collum, Christensen, Delicath, & Johnson, 2019), literacy (Collum, Christensen, Delicath, & Knezek, 2020), self-reported educator bias (Collum et al., 2020) and instructional self-efficacy (Knezek & Christensen, 2009). The key innovation of the program is that it provides teachers and teacher trainees many learning trials with simulated students, thereby increasing teacher confidence and competence, which in turn improves student learning. Repetition of many trials is important in changing habit complexes such as implicit bias (Malone, 2016).

Methods

Research Question
The focus of the current study is on the changes that occurred pre-post for the teachers during completion of modules in the teaching simulator. The research question guiding this paper is: 
 To what extent do changes occur during multiple sessions (iterations) of participation in simulated teaching modules?

Methods
Participants
Classroom teachers were recruited from schools in a two large public school systems. Thirty-nine teachers completed all the modules and data collection. Incomplete data from the educators who partially completed project requirements were not included in this study.
Teacher distribution by gender was 28 (71.8%) females and 11 (28.2%) males. The teachers reported ethnic identities included a majority White (28, 71.8%), followed by Hispanic and Asian (each 4, 10.3%), Black/AA (2, 5.1%) and Latinx (1, 2.6%). Teachers were paid a stipend for completing all components of the project. Modules are described below and were selected by district leaders to provide professional development that matched district goals.

Activities
The 39 teachers participated in one group of modules depending on the grade level taught. The listing of the modules by grade level band is shown in Table 1. Each participant first completed a tutorial module (2 sessions) to understand how to navigate simSchool. Each of the content modules required completion of at least five sessions in each module. Each session consisted of teaching the provided lesson for at least 15 minutes, reviewing provided feedback and reteaching the lesson with the goal of improvement based on the feedback. The average total amount of time spent actively completing simulations was 4 hours and 39 minutes.
 At the end of each simulation session, participants received graphical feedback displaying degree of success at promoting academic, emotional and equity performance in the simulated class overall, as well as feedback regarding the degree of suitability of the instructional activities selected for each individual simulated student in the class. Figures will be provided to illustrate examples of graphical feedback as well as an observation report that participants receive. A figure will illustrate the equity index that has been developed for this project. This feedback can reveal aspects of a teacher’s implicit biases for gender and/or ethnicity.

Table 1. Modules Completed by Classroom Teachers
Elementary School Modules
Tutorial: Teaching in simSchool
Module 1: Cultural Intelligence and Inclusion 2.0
Module 2: Bullying and Bias the First Coconut Tree
Module 3: Gender and Identity: Supermom Saves the Day
Middle School Modules
Tutorial: Teaching in simSchool
Module 1: Gender and Identity: The Misfits
Module 2: History Empowering Learners to Change the world
Module 3: Race, Ethnicity, Class, Immigration: A Tale of Two Schools
High School Modules
Tutorial: Teaching in simSchool
Module 1: Showing Empathy
Module 2: Sounds of Change
Module 3: Why Local Elections Matter

Data Sources
Data were collected within the simulator to measure pre-post changes during the modules for academic, emotional and equity performances by project participants. In addition, teacher self-report survey measures focused on self-efficacy, culturally responsive teaching, and self-awareness of bias. Student survey data were also collected following their teachers’ participation in the project, but not included in this paper. Below are descriptions of the teacher surveys. Each of the surveys is rated on a 6-point Likert scale from Strongly Disagree (1) to Strongly Agree (6).

The teacher surveys included:
1. The Teachers’ Sense of Efficacy Scale (TSES) (Tschannen-Moran & Hoy, 2001) was used to measure self-efficacy related to three subscales: instructional strategies, classroom management, and student engagement.
2. The Culturally Responsive Self-Efficacy Survey (Siwatu, 2007) was included to determine the level of competency in the skills and knowledge needed to engage in culturally responsive teaching that includes curriculum, assessment, classroom management and cultural enrichment.
3. Three scales from the Educator Bias Inventory (Collum et al., 2020) were included. These scales include: Self-Awareness, Pedagogical environment, and Relationships with families and community. This survey was recently used with simSchool research and is based on Chen et al. (2009).

Results

Both self-report and simulation collected data were downloaded from simSchool and analyzed using paired t-tests to measure pre-post changes. Results are reported for the 39 teachers who completed all required components.
As will be shown in tabular form, there were significant (p <.05) positive changes from pre to post for Culturally Responsive Teaching Self-efficacy and two scales on the Educator Bias Inventory: Pedagogical environment and Relationship with families and communities. Effect size is a meaningful indicator for looking at educational significance. The effect sizes were educationally meaningful at .30 and above for three of the seven measures. Fourteen of the 25 individual items on the Culturally Responsive Self-Efficacy Survey and five of the Educator Bias Inventory were significant. An additional single item related to reflection of participants’ own identity was also included in the survey items. The item," I reflect on how my own identity influences my interactions with students" was not significant at the p <.05 level, but the effect size was .21 (Cohen’s d) regarding the magnitude of the gain for this item, meaning the teachers became more reflective on their identity following the simEquity module completions.

Results from the Simulation Data
Teaching behaviors are captured within the simSchool system that allow computation of academic gains, emotional gains, and equality gains while teaching within a module. In addition, ratings for “How likely to succeed in future lessons” were also recorded in the system by each teacher for each of the 12 simStudents in a teacher’s class. These were presented to the teachers following the first 15-minute simulation, and the last (fifth)15-minute simulation, for each of the three modules. Separate ratings were completed by each teacher, first based on just reflecting on the image of each simStudent (Avatar featuring different skin tones) and then based on name of the student with no image.
  Across the three modules completed by each teacher, from the first time of capturing data at the end of completing the first 15-minute simulation (of 5) for Module 1, to the end of the last 15-minute simulation (of 5) for Module 3, the overall gains will be shown in tabular format and graphically displayed. They were highly significant (p <.005) for Academic Index (ES = .42), Avatar Rating (ES = .37), and Name Rating (ES = .29). The magnitudes of these gains are at or beyond levels that would be widely accepted as educationally meaningful in the research literature (Bialo & Sivin-Kachala,1996). The simulator-produced Emotional Index and Equality Index showed non-significant (at the p < .05 level) positive gains overall with effect sizes of ES = .08 and ES = .07 respectively. These would be considered small positive effects by Cohen (1988) and would be considered “Developmental Effects” (0 - .1) by Hattie (2009).
 Within each module there were differences that are beginning to emerge and will be studied in more detail. Specifically, for Module 1, First to Last Academic, Emotional, Equality and Avatar plus Name ratings all become more positive from the first of five simulations to the last of five simulations, and all but the Equality Index gains were significant (p < .05). Within Module 2, Teachers appear to have concentrated on Academic Gains (p < .0005, ES = .41). They began on the first simulation in M2 near the same level of proficiency as where they began (with a different lesson to teach) in M1, but advanced more. None of the other indices changed significantly (p < .05) and magnitude/ES was typically very small in M2. In Module 3, with a third new lesson to teach, educators held steady at what we can now call a “high” level of First and Last in Academic Index (.85 vs. .72 in the beginning), Avatar Ratings and Name ratings (3.7+ vs. <=3.4 in the beginning). Mean ratings in these three areas changed little first to last simulation. Teachers advanced first to last significantly (p < .05) in the areas of Emotional Index and Equality Index but the magnitude/ES was ES = .12 and ES = .09, small per Cohen (1988). Note that these latter two areas were the only ones without significant gains in Module 1.

Importance

Simulations offer many possibilities as a pedagogical approach for teacher professional development related to equity-based teaching practices and are increasingly being used to approximate various teaching scenarios and support the transfer of learning into classroom situations (Dalinger, Thomas, Stansberry, & Xiu, 2020). Most research on simulations for teacher education that focus on equitable teaching practices include human actors (Cohen, Wong, Krishnamachari, & Berlin, 2020), a type of simulation that is not affordable or sustainable for large groups of educators. SimSchool provides a fully digital environment for supporting the improvement of teacher practices related to equity.

References

Badiee, F. (2012). From the digital to the authentic classroom: A study using an online simulation for teacher education (Unpublished master’s thesis). Simon Frasier University, Burnaby, BC, Canada.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W.H. Freeman and Company.
Bialo, E. & Sivin-Kachala, J. (1996). The effectiveness of technology in schools: A summary of recent research. School Library Media Quarterly, 25(1), 51-57.
Callaway, R.F. (2016). A correlational study of teacher efficacy and culturally responsive teaching techniques in a Southeastern urban school district. Education Dissertation Projects, 188. https://digitalcommons.gardner-webb.edu/education_etd?188
Case, R. (1993) Theories of learning and theories of development, Educational Psychologist, 28(3), 219-233, DOI: 10.1207/s15326985ep2803_3
Chen, D.W., Nimmo, J., & Fraser, H. (2009). Becoming a culturally responsive early childhood educator: A tool to support reflection by teachers embarking on the anti-bias journey. Multicultural Perspectives, 11(2), 101-106. doi: 10.1080/15210960903028784
Christensen, R., Knezek, G., & Collum, D. (2020). Using simulation based learning to increase multicultural awareness in educators. In Proceedings of SITE Interactive Online 2020 Conference (pp. 102-106). Association for the Advancement of Computing in Education (AACE).
Christensen, R., Knezek, G., Patterson, L., Wickstrom, C., Overall, T. & Hettler, L. (2007). Early experiences with SimMentoring: From virtual to real teaching. In R. Carlsen et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2007 (pp. 1186-1188). Chesapeake, VA: AACE. http://www.editlib.org/p/24719.
Christensen, R., Knezek, G., Tyler-Wood, T., & Gibson, D. (2011). SimSchool: An online dynamic simulator for enhancing teacher preparation. International Journal of Learning Technologies. 6(2), 201-220.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Cohen, J., Wong, V., Krishnamachari, A., & Berlin, R. (2020). Teacher coaching in a simulated environment. Educational Evaluation and Policy Analysis, 42(2), 208-231. Doi.org/10.3102/0162373720906217
Collum, D., Christensen, R., Delicath, T., & Knezek, G. (2020). Measuring changes in educator bias in a simulated learning environment. In G. H. Marks & D. Schmidt-Crawford (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 410-416). Online: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/215788/
Dalinger, T., Thomas, K.B., Stansberry, S., & Xiu, Y. (2020). A mixed reality simulation offers strategic practice for pre-service teachers. Computers & Education, 144. https://doi.org/10.1016/j.compedu.2019.103696
Derman-Sparks, L., & Ramsey, P. (2000). A framework for relevant ‘ multicultural’ and antibias education in 21st century. In J. Roopnarine & J. Johnson (Eds), Approaches to Early Childhood Education. Upper Saddle River, NJ: Merrill Prentice Hall.
Dweck, C. (1999). Self-theories: their role in motivation, personality and development, Philadelphia: Psychology Press.
Fischler, R. (2006). SimTeacher: Simulation-based learning in teacher education (Doctoral dissertation). Available from ProQuest Dissertation and Theses database. (UMI No.3210046)
Girod, G., & Schalock, M. (2002). Does TWSM work? In G. Girod (Ed.), Connecting Teaching and Learning: A Handbook for Teacher Educators on Teacher Work Sample Methodology. Washington DC: American Association of Colleges of Teacher Education.
Kitsantas, A. (2012). Teacher efficacy scale for classroom diversity (TESCD): A validation study. Vol. 16 no.1, Profesorado, Revista de curriculum y formacion del profesorado. Retrieved June 1, 2015, from http://www.ugr.es/local/recfpro/rev161ART3en.pdf
Knezek, G., & Christensen, R. (2009). Preservice educator learning in a simulated teaching environment. In Research Highlights in Technology and Teacher Education (Vol. 1, pp. 161–170).
Knezek, G., Hopper, S., Christensen, R., Tyler-Wood, T., & Gibson, D. (2015). Assessing pedagogical balance in a simulated classroom environment. Journal Digital Learning in Teacher Education, 31(4), 148-159.
Malone, J.C. (2016). E.R. Guthrie: A behaviorism for everyone. In D. Zilio & K. Carrara (eds.). Behaviorisms: Historical and conceptual issues (Vol. 1). Sao Paulo: Nucleo Paradigma Press.
McCrae, R., & Costa, P. (1996). Toward a new generation of personality theories: Theoretical contexts for the five-factor model. In J. S. Wiggins (Ed.), The five-factor model of personality: Theoretical perspectives (pp. 51-87). New York: Guilford.
McGinnis, C.M. (2017). Effects of implicit bias on teachers’ expectations of student relationships. Public Access Theses and Dissertations from the College of Education and Human Sciences. University of Nebraska-Lincoln. http://digitalcommons.unl.edu/cehsdiss/294.
Oyerinde, S.A. (2008). A correlational study of teacher efficacy and culturally responsive teaching techniques in four public middle schools. Dissertation University of Missouri-Kansas City.
Phakiti, A., Hirsh, D., & Woodrow, L. (2013). It’s not only English: Effects of other individual factors on English language learning and academic learning of ESL international students in Australia. Journal of Research in International Education, 12(3), 239-258.
Protheroe, N. (2008). Teacher efficacy: What is it and does it matter? Principal, 87(5), 42-45.
Schneider, W. J., & McGrew, K. S. (2012). The Cattell-Horn-Carroll model of intelligence. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (p. 99–144). The Guilford Press.
Sianjina, R.R. (2000). Educational technology and the diverse classroom. Kappa Delta Pi Record.
Siwatu, K.O. (2007). Preservice teachers’ culturally responsive teaching self-efficacy and outcome expectancy beliefs. Teaching and Teacher Education, 23, 1086-1101.
Smith T.W. (2013). Interpersonal Circumplex. In M.D. Gellman & J.R. Turner (eds). Encyclopedia of Behavioral Medicine. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-1005-9_1584
Tschannen-Moran, M., & Hoy, A.W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783-805.
Tsachannen-Moran, M., Hoy, A., & Hoy, K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68(2), 202-248.
Tucker, C.M., Porter, T., Reinke, W.M., Herman, K.C., Ivery, P.D., Mack, C.E., & Jackson, E.S. (2005). Promoting teacher efficacy for working with culturally diverse students. Preventing School Failure, 50(1), 29 -34.
Vygotsky, L.S. (1962). Thought and language. Cambridge, MA: MIT Press.