Implementing Computer Science Education Policy: Insights from Computer Science Education State Supervisors

Framework

We apply the sociocultural lens to the Capacity for, Access to, Participation in, and Experience of (CAPE) Framework (Fletcher & Warner, 2021) in our research. The CAPE Framework conceptualizes equitable and accessible CS education across four essential domains: the capacity to offer CS, providing access to CS to every student, ensuring equitable participation in CS across demographic groups, and ensuring students' experiences of computing have comparable outcomes. The CAPE Framework has been used to assess equity through analyzing and interpreting data to inform policy decisions (Fletcher & Warner, 2021). We apply the CAPE Framework to the data collected from responses to the CSEdSS survey and focus groups, positing that the perspectives of CSEdSS are based on the dynamic interaction of individual and situated cognition within unique organizational structures.

Methods

This examination of the CSEdSS landscape constitutes the initial phase of a five-year research initiative. This study aims to explore the role, perspectives, mediators, drivers, and outcomes of CSEdSS across states as CS education policies gain traction across the nation. Both qualitative and quantitative data was collected. Qualitative data included survey responses and documents concerning CS education policy including reports, organizational data, and CSEdSS job descriptions. Open-ended survey responses, focus group transcripts, and documents were examined using a thematic coding approach to analyze the sphere of CSEdSS work (e.g. resources they use, activities they engage in, challenges they face, among others). We used recursive coding (Bingham, 2023) to move among the themes grounded in the conceptual framework and themes that emerged from the data. We developed summaries that captured thematic trends across states and identified patterns regarding how the policy of CSEdSS is appropriated by local actors across states and enacted as policy (Miles et al., 2014). Quantitative data consisted of questions such as the number of years CSEdSS had been in their position, and demographic information including gender, age, and race. We used descriptive statistics to analyze these responses.
Data collection included a survey and focus group administered to CSEdSS participants, and a variety of materials that outline, guide, and influence policy related to CS education. The survey consisted of 21-questions that collected qualitative and quantitative data. The survey and focus group questions were adapted from the following sources: 1) Science Coordinator Role Survey and Science Coordinator Interview (Whitworth et al., 2017), and 2) Science Education Supervisor Interview Protocol (Haverly et al., 2023). The above sources contain questions used with science education supervisors which we adapted for CSEdSS. For the purpose of this study, we also created questions related to the CS education landscape more specifically to capture who CSEdSS collaborate with (item #18) and how they provide inclusive support for students throughout their respective states (item #20). While 41 states have CSEdSS, our sample included some states that have more than one individual in the role. We used Qualtrics to design the survey and record the CSEdSS' responses for subsequent analysis, and distributed it through a listserv of CSEdSS. To protect participants' individual identities, we present the data from this survey anonymously. The main objective of the survey was to gain insights into who CSEdSS are and their backgrounds, how CSEdSS perceive and operationalize their role, and which aspects of their work align with the CAPE Framework. We also collected a variety of publicly available materials, including state legislation, learning materials, and state CS education-related documentation to inform our findings and deepen our understanding of the role CSEdSS have in improving CS education. We member-checked our assumptions with the survey participants to ensure accuracy and reliability.

To examine our data, we focused on the research questions we had delineated and open-coded the participants' responses emphasizing patterns around CSEdSS' job responsibilities and their involvement in championing equitable access to CS education in their states. This process entailed examining recent literature around CS education and equity, writing research journal entries and analytic memos to identify and highlight relevant issues arising from the data and coding process.
Our codebook consisted of five general themes aligned to items in our CSEdSS survey, along with a set of sample codes for each theme. The five themes are 1) drivers as the elements that initiate or sustain the work of CSEdSS. Drivers were measured in survey items #13-14, #17-18, 2) Mediators of CSEdSS include elements that either facilitate or pose challenges to implementing K-12 CS education. CSEdSS leverage mediators to navigate their responsibilities (item #19), 3) Activities encompass a variety of endeavors that enable CSEdSS to implement CS initiatives, 4) Outputs (item #21) pertain to both perceived and actual impacts resulting from CSEdSS activities, and 5) Promising equity practices (item #20) increase access and opportunity to CS education. A sample of our codebook can be seen in Table 3.
After coding the survey and focus group responses, we examined each code for their alignment to the dimensions of the CAPE Framework. Situating our work within CAPE provided a sustained focus on equitable CS within the sphere of state policy implementation from the perspective of CSEdSS.

Results

RQ1. How do CSEdSS enact their role across and within their respective states?
We found that CSEdSS work across all dimensions of the CAPE Framework.
CSEdSS build capacity for CS education by developing resources, supporting teacher professional development, and facilitating networks of support among stakeholders. Two important statewide resources that CSEdSS develop are the CS education statewide implementation plan and CS education academic standards. According to a CSEdSS, "My primary responsibility currently is developing the Computer Science Education Statewide Implementation Plan.” This plan is critical to set strategic mission, vision, goals, and metrics for implementing K-12 CS education within their respective states. As of 2023, 30 states have developed and adopted state CS plans. Another CSEdSS stated, “I am responsible for our CS standards, as well as the writing of and the implementation of the 5 year CS plan.”
CSEdSS stated that they build capacity for CS by developing and integrating CS into courses and course pathways. Several CSEdSS highlighted that developing CS education courses often involves integrating CS with Career Technical Education (CTE) and other disciplines. Some “oversee the implementation of… K-12 Computer Science Standards by providing professional development” and others “develop and coordinate professional learning; evaluating and monitoring professional learning effectiveness.” Others support CS teachers by “compiling and distributing CS education resources to teachers across the state” and “[creating] support and materials for CS teachers.” CSEdSS also build capacity of the larger community in their respective context by “developing and fostering effective relationships with local school systems and other education entities.” They recognize that “communication with external partners is a key facet of [their] work as there are many organizations in the state who contribute to CS education.”

Through supporting access to CS education in their respective states, CSEdSS produce outputs such as creating and disseminating resources to teachers and providing policy guidance. To increase access to professional learning for teachers and CS courses for students across their respective states, CSEdSS have also developed online courses which allows access to CS learning to take place any time, anywhere. One CSEdSS shared about her participation in a national working group whose goal is to increase the accessibility of CS resources and platforms to students with disabilities. A part of CSEdSS’ role may also include in-person support where they travel to schools “to help teachers, counselors, and administrators broaden access to computer science.” By traveling to sites, CSEdSS mitigate the geographical barrier to provide access to valuable resources and CS education support.

Based on the data collected and analyzed from the surveys, there was evidence that CSEdSS support participation in CS education. For example, to ensure that educational institutions and policies support equitable enrollment of students in CS courses, CSEdSS shared that they collect and analyze CS participation data. By contributing to data analysis and reporting efforts, CSEdSS also “[provide] information for policymakers” and “assist schools with policy updates around computer science and computer literacy.” One CSEdSS even shared that it was their role to provide guidance on CS education to policymakers. While less common among CSEdSS, there was one CSEdSS who contributed to research efforts within their state due to the grant-funded nature of her position where they “collect and evaluate data and support research in computer science education.”

As part of their role and responsibility as CSEdSS, they also support the experience of CS education in their role through curriculum efforts and work on CS pathways. Depending on whether the curriculum is determined at a state or local level, some CSEdSS state that they “collaborate to evaluate CS curricula, facilitate collaboration of [SEA], school districts." CSEdSS work to support the experience and quality of CS learning experiences and course content by participating in the process to identify High Quality Instructional Materials. Related to providing support that meets the needs of diverse contexts, one CSEdSSS shared, “My priority will always be to get students access to high quality CS material that they can engage with and enjoy. That said, each district faces its own set of challenges when trying to attain that goal. So while priority may not be different, the scaffolds and approaches may.” Another CSEdSS stated, “There are, of course, many contexts that we must consider for groups, especially our historically underserved student populations.” Whether it is supporting teacher training, relevant curriculum or ensuring there are diverse CS pathways, CSEdSS work to scaffold CS education across diverse settings within their respective states.

RQ2. How does equity impact and inform the work of CSEdSS around teacher training, resource allocation, and providing statewide support?
As one CSEdSS shared, “Broadening participation, access, and engagement in CS learning is the overarching goal of our state plan. Race, gender, and geography absolutely impact how we approach developing this.” While some CSEdSS acknowledged that the word “equity” is not “allowed” due to anti-diversity, equity, and inclusion agendas in their states, they referenced equity issues through responses such as highlighting the rural vs. urban or opportunity gaps based on gender or race that exists in their states. The CSEdSS responses indicate that equity impacted areas primarily related to building capacity in and ensuring access to CS education opportunities for all students.
When considering equity issues in building capacity for CS education, CSEdSS identify resource allocation, teacher training, and statewide support as key domains. Equity priorities and challenges around resource allocation included funding, internet/broadband connectivity, infrastructure, and facilities with larger, urban schools having the capacity to train teachers and leverage “upgraded equipment.” Another key challenge identified by CSEdSS is related to funding. Specifically, they highlight the strict rules pertaining to how grant funding can be used, which in turn impact what schools and districts can request. Moreover, funding connects to another equity issue related to resource allocation with CSEdSS pointing out that smaller districts may not even have dedicated grant writing staff to access grant funding.
Having the capacity to support CS education was closely linked with ensuring access to CS education for every student in the state. As one CSEdSS noted, “It is crucial to provide access to populations who are under-represented in CS. While this is recognized, the state does not make any distinction on how it serves one location from another.” In particular, CSEdSS highlighted that they face challenges around ensuring equitable access to CS courses and pathways with tribal nations and Bureau of Indian Education schools. They also indicate that disparities in access to CS may be the result of different policies. For example, CSEdSS share that when CS is required, districts place greater value on it. Moreover, they note that the “disparities in participation” among different student populations is “one of the main forces driving this work” because they believe that “it is not okay for these opportunities to be available to a privileged few.”
When goals to broaden participation in CS are enacted at the level of considering students’ experience of CS education, CSEdSS state that they aim to adjust their support to local context and culture. At the same time, CSEdSS attend to the ways that they can address potential disparities across the system even when there are policies in place to ensure equitable outcomes. For example, a CSEdSS shared that while their state made CS a graduation requirement that has ensured access to and participation in CS for all their students, there is still the need to address the systemic inequities across schools and districts that result in variation in the quality and range of CS courses and experiences available to their students.

Importance

This work lies at the intersection of policy, practice, and research to investigate the root causes of systemic inequities while working in partnership with non-profit organizations, state departments of education, and policymakers to illuminate and advance solutions to dismantle structures that perpetuate inequities. To ensure diverse, inclusive, and accessible CS educational pathways across schools and districts within states, it is critical to explore how CSEdSS operationalize and enact their roles. With the rapid adoption of CS education policies, there is a need to understand the structural barriers in access and exposure to CS and how education stakeholders can dismantle these obstacles to ensure equitable learning of CS within their states, as well as identify opportunities and key levers that help advance equity in K-12 CS education.

References

References contained in the proposal:
Bingham, A. J. (2023). From data management to actionable findings: A five-phase process of qualitative data analysis. International journal of qualitative methods, 22, 16094069231183620.

Fletcher, C. L., & Warner, J. R. (2021). CAPE: A framework for assessing equity throughout the computer science education ecosystem. Communications of the ACM, 64(2), 23-25. https://doi.org/​​10.1145/3442373

Haverly, C., Seeber, E. R., Lyle, A. M., Davis, E. A., & Ovies-Bocanegra, M. A. (2023). District science coordinators’ conceptions of and levers for advancing equity agendas. In National Association for Research in Science Teaching (NARST) Annual International Conference.

Miles, M., Huberman, M., & Saldaña, J. (2014). Qualitative Data Analysis: A Methods Sourcebook. Washington, DC: Sage.

Whitworth, B. A., Maeng, J. L., Wheeler, L. B., & Chiu, J. L. (2017). Investigating the role of a district science coordinator. Journal of Research in Science Teaching, 54(7), 914-936. https://doi.org/10.1002/tea.21391

References planned in the full paper:
Alexander, T., Baldwin Clark, L., Reinhard, K., & Zatz, N. (2023). Tracking the Attack on Critical Race Theory. CRT Forward initiative of the Critical Race Studies (CRS) Program at UCLA School of Law. https://crtforward.law.ucla.edu/wp-content/uploads/2023/04/UCLA-Law_CRT-Report_Final.pdf

Anderson, L. (2002). Curricular alignment: A re-examination. Theory into Practice, 41(4), 255–260. https://doi.org/10.1207/s15430421tip4104_9
Barnes, C., Goertz, M., & Massell, D. (2014). How state education agencies acquire and use research knowledge for school improvement. In K. Finnigan & A. Daly (Eds.), Using research evidence in education: From the schoolhouse door to Capitol Hill . New York: Springer.

Brown, A. L., & Camptioone, J. C. (2013). Interactive learning environments and the teaching of science and mathematics. In Toward a scientific practice of science education (pp. 111-139). Routledge.

Coburn, C. E. (2001). “Collective Sensemaking about Reading: How Teachers Mediate Reading
Policy in Their Professional Communities.” Educational Evaluation & Policy Analysis 23(2): 145– 70. https://doi.org/10.3102/01623737023002145

Coburn, C. E. (2006). Framing the Problem of Reading Instruction: Using Frame Analysis to Uncover the Microprocesses of Policy Implementation. American Educational Research Journal, 43(3), 343-349. https://doi.org/10.3102/00028312043003343

Cohen, D. K., & Spillane, J. P. (1991). Policy and practice: The relations between governance and instruction. Review of Research in Education, 18(1), 3-49.

Cohen, M. D. (1991). Individual learning and organizational routine: Emerging connections. Organization science, 2(1), 135-139. https://doi.org/10.1287/orsc.2.1.135

Code.org, CSTA, ECEP Alliance (2023). 2023 State of Computer Science Education. https://advocacy.code.org/stateofcs

Cunningham, D. H., & Wyckoff, J. (2013). Policy makers and researchers schooling each other: Lessons in educational policy from New York. Education Finance and Policy, 8(3), 275-286. https://doi.org/10.1162/EDFP_a_00095

Expanding Computing Education Pathways Alliance. (n.d.). Broadening Participation In Computing State by State. Retrieved September 4, 2024, from https://ecepalliance.org/

Flapan, J., Hadad, R., Shorall, C., & Twarek, B. (2020). CS Equity Guide: A K-12 education leader’s guide to designing, scaling, and sustaining equitable computer science in California. CSforCA. https://csforca.org/csequityguide

Fullan, M. (2015). The new meaning of educational change. Teachers college press.

Garcia, D. (2022). Teach truth to power: how to engage in education policy. The MIT Press.

Gross, B., & Hill, P. T. (2016). The state role in K-12 education: From issuing mandates to experimentation. Harv. L. & Pol'y Rev., 10, 299.

Hargreaves, A., & Fullan, M. (1998). What’s worth fighting for out there? Teachers College Press.

Honig, M., & Hatch, T. (2004). Crafting coherence: How schools strategically manage multiple, external demands. Educational Researcher, 33(8), 16 – 30. https://doi.org/10.3102/0013189X033008016

Hopkins, D. (2015). School improvement and system reform. The Routledge international handbook of educational effectiveness and improvement, 124-148.

Koshy, S., Martin, A., Hinton, L., Scott, A., Twarek, B., & Davis, K. (2021). The Computer Science Teacher Landscape: Results of a Nationwide Teacher Survey. https://www.kaporcenter.org/the-computer-science-teacher-landscape-results-of-a-nationwide teacher-survey

Layland, A. (2019). Connecting Commitments, Principles, and Practices to Strategically Address Equity and Improvement. Council of Chief State School Officers.

López-Quiñones, A., Martinez-Lopez, M., Moreno Sandoval, C. D., Carroll-Miranda, J., Lindala, A. E., Chatman, M. C., ... & Flores-Reyes, E. (2023). Ancestral Computing for Sustainability: Centering Indigenous Epistemologies in Researching Computer Science Education. TechTrends, 67(3), 435-445.

Madkins, T. C., Martin, A., Ryoo, J., Scott, K. A., Goode, J., Scott, A., & McAlear, F. (2019). Culturally relevant computer science pedagogy: From theory to practice. In 2019 research on equity and sustained participation in engineering, computing, and technology (RESPECT) (pp. 1-4). IEEE. https://doi.org/10.1109/RESPECT46404.2019.8985773

Authors. (2024). State-Level Perspectives of High School Computer Science Education Policy: A Complex Systems Analysis. In Proceedings of the 2024 on RESPECT Annual Conference (pp. 245-253). https://doi.org/10.1145/3653666.3656102

Margolis, J., Estrella, R., Goode, J., Jellison-Holme, J., & Nao, K. (2017). Stuck in the Shallow End: Education, Race, & Computing (2nd ed.). MIT Press: Cambridge, MA

Master, A., Cheryan, S., & Meltzoff, A. N. (2016). Computing whether she belongs: Stereotypes undermine girls’ interest and sense of belonging in computer science. Journal of educational psychology, 108(3), 424.

National Association of State Boards of Education. (2022). Education governance dashboard. https://www.ecs.org/education-governance-dashboard/

Nowosad, I. (2021). Educational policy in an integrated model of intervention at the macro, meso and micro levels. Designing and Implementing Public Policy of Contemporary Polish Society: Selected Problems, 115-126.

Pennsylvania Training and Technical Assistance Network. (n.d.). CS Equity Guide. https://www.pattan.net/Supports/STEM/computer-science/CS-Administrator-Toolkit/Equity-Inclusion

Railey, H. (2017). State Education Governance Structures: 2017 Update. 50-State Review. Education Commission of the States.

Sachs, J. (1997). Revisioning teacher education. Unicorn, 23(2), 46-56.

Scott, A., Kapor Klein, F., McAlear, F., Martin, A., & Koshy, S. (2018). The leaky tech pipeline: A comprehensive framework for understanding and addressing the lack of diversity across the tech ecosystem. Kapor Center. https://leakytechpipeline.com/wp-content/themes/kapor/pdf/KC18001_report_v6.pdf

Scott, J., & Jabbar, H. (2014). The hub and the spokes: Foundations, intermediary organizations, incentivist reforms, and the politics of research evidence. Educational Policy, 28(2), 233-257. https://doi.org/10.1177/0895904813515327

Scott, J., Lubienski, C., DeBray, E., & Jabbar, H. (2014). The intermediary function in evidence production, promotion, and utilization: The case of educational incentives. Using research evidence in education: From the schoolhouse door to Capitol Hill, 69-89.

Scott, W. R., & Meyer, J. W. (1994). Structural complexity and individualism. Thousand Oaks, CA: Sage.

Scott, S.L., Khanani, J., and Scott, K.R. (2022). Building a State Equity Strategy. Rockville, MD: Region 5 Comprehensive Center.

Smarick, A., & Squire, J. (2014). The State Education Agency: At the Helm, Not the Oar. Bellwether Education Partners.

Snader, D. W. (1962). The Leadership Role of State Supervisors of Mathematics: Report of a Conference under the Auspices of the U.S. Department of Health, Education, and Welfare, Washington, June 19-23, 1961. Bulletin, 1962, No. 2. OE-29032. Distributed by ERIC Clearinghouse.

Spillane, J. P. (2000). Cognition and policy implementation: District policy makers and the reform of mathematics education. Cognition and Instruction, 18, 141-179. https://doi.org/10.1207/S1532690XCI1802_01
Spillane, J. P. (2004). Standards Deviation: How Schools Misunderstand Education Policy. Cambridge, MA: Harvard University Press.

Spillane, J. P. (2005). Distributed leadership. In The educational forum (Vol. 69, No. 2, pp. 143-150). Taylor & Francis Group. https://doi.org/10.1080/00131720508984678
Spillane, J. P., & Jennings, N. E. (1997). Aligned instructional policy and ambitious pedagogy: Exploring instructional reform from the classroom perspective. Teachers college record, 98(3), 449-481. https://doi.org/10.1177/016146819709800303

Spillane, J. P., & Zeuli, J. S. (1999). Reform and Teaching: Exploring Patterns of Practice in the Context of National and State Mathematics Reforms. Educational Evaluation and Policy Analysis, 21(1), 1-27. https://doi.org/10.3102/01623737021001001

Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72, 387 – 431. https://doi.org/10.3102/00346543072003387

Timperley, H. S. (2005). Distributed leadership: Developing theory from practice. Journal of curriculum studies, 37(4), 395-420.
https://doi.org/10.1080/00220270500038545

Tseng, V., & Nutley, S. (2014). Building the infrastructure to improve the use and usefulness of research in education. In Using research evidence in education: From the schoolhouse door to Capitol Hill (pp. 163-175). Cham: Springer International Publishing.

Wedel, J. R., Shore, C., Feldman, G., & Lathrop, S. (2005). Toward an anthropology of public policy. The ANNALS of the American Academy of Political and Social Science, 600(1), 30-51.
https://doi.org/10.1177/0002716205276734