Links Between Social-Emotional Learning and Academic Outcomes Across Three School Districts

Framework

This is a quantitative study that embodies a post-positivist theoretical framework with a modified objectivist epistemology. A major point of emphasis of the study is generalizability across populations.

Methods

Sample
Data was collected through a web-based platform designed to support the implementation of a successful MTSS practice. Reading, Math, and SEL assessment scores were collected during the fall 2021 and spring 2022 semester. The total sample size includes 4,478 students from three school districts located in East Texas, South-East Illinois, and North-East Illinois. Ethnicity of the students collapsed across districts is 22.6% White, 15.9% Hispanic, 59.4% Black, 2.1 Other.

Measure(s)
Reading and Math: The Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) was used to assess students’ Reading and Math abilities. The computer adaptive screening tool provides grade-level nationally normed percentile scores and takes approximately 45 minutes to administer for each student (NWEA MAP, 2004). In accordance with the He and Meyer (2021) review of recommended universal percentile cut scores to maximize accuracy for the NWEA MAP Reading and Math assessments, cut scores of the 30th percentile were chosen to identify students in need of “intensive intervention”.
Social Emotional Learning: The Devereux Student Strengths Assessment Mini (DESSA-Mini) was used to capture the social emotional skill level of each student. This increasingly popular tool was designed for the universal screening of students’ SEL skills. Typically relying on teacher’s assessments, the survey consists of 8 strength-based items that provide a summary score capturing the SEL construct without sacrificing convenience (Naglieri, LeBuffe & Shapiro, 2011; Shapiro et al. 2017). In accordance with both the guidelines of Naglieri et. al (2011) and practices of Shapiro et al. 2017 of the DESSA-Mini scores in the 16th percentile and below on the DESSA-Mini were categorized as need of intervention.

Methods
Statistical relationships among scores for the Reading NWEA MAP, Math NWEA MAP, and DESSA-Mini were examined through three methods: 1) correlations were calculated to characterize the linear relation between fall 2021 measures for student SEL skill and fall 2021 student reading and math performance, 2) Stepwise regression models were used to calculate unique variance of spring academic percentile scores accounted for by fall SEL assessments after controlling for fall academic percentile scores, and 3) the percentage overlap of students screened for SEL was compared to those screened for Reading and Math interventions. The correlations between each of the measures provides an indication of the direct relationship that Reading, Math, and SEL have with one another. The R2 change test in the stepwise regression model will provide an indication of the unique variance of academic progress accounted for in social-emotional skill. The final analysis will focus on the percent overlap between students who are identified as needing additional support within the three domains.

Results

Fall Assessment Correlations: When collapsing across all districts a significant moderate correlation was found between fall 2021 student DESSA-mini assessment scores and both fall 2021 NWEA MAP Reading (r(4476) = .37, p < .001) and Math (r(4504) = .39, p < .001) assessment scores. These moderate correlations were additionally observed at all three districts individually with slightly higher correlations observed between measures for SEL and student performance on the fall Math assessment scores than measures for SEL and student performance on the fall Reading assessment.

Stepwise Regression Spring Academics: Aggregate data from all three districts was used to assess whether DESSA-Mini percentile scores account for unique variance in spring NWEA MAP academic percentile scores after controlling for fall NWEA MAP academic performance. These models were assessed for both Reading and Math NWEA MAP assessments.

In Reading, the first model regressed spring Reading percentile scores on fall Reading percentile scores while the second model regressed spring Reading percentile scores on both fall Reading and fall SEL percentile scores. As intuition may suggest, spring Reading percentile scores and fall Reading percentile scores were highly correlated (r(4476) = .78, p < .001). The first regression model similarly showed a high degree of variance in student spring Reading percentile scores being accounted for by their fall Reading percentile scores (R2 = .61, F(1,4476) = 6981.99, p < .001). The second model including both fall Reading percentile scores and fall SEL percentile scores as predictors showed the addition of the SEL measure accounting for a slightly larger portion of variance in spring Reading percentile scores (R2 = .62, F(2,4475) =7081.56, p < .001). With a significant change of .01 (F(1,4475) = 99.56, p < .001) and significant regression coefficient for student DESSA-Mini percentile scores, the DESSA-Mini is shown to account for a small but significant portion of the variance seen in student spring NWEA Reading performance even after controlling for NWEA Reading percentile scores.

A similar method was used in Math with a R2 change test being run between two regression models predicting student spring Math percentile scores from fall Math percentile scores first then by fall Math percentile scores and fall SEL percentile scores. Similar to Reading, a high correlation was observed between student spring and fall Math percentile scores (r(4504) = .83, p < .001). The first regression model therefore showed the large amount of variance in student spring semester performance in Math by their performance in the fall (R2 = .68, F(1,4504) = 9631.94). The second model including both fall Math percentile scores and fall SEL percentile scores showed a very small increase in variance explained (R2 = .69, F(2,4503) = 9712.00, p < .001). With a significant change of .006 (F(1,4503) = 80.06, p < .001) and significant regression coefficient for student DESSA-Mini percentile scores, the DESSA-Mini was shown to account for a very small but significant difference in student spring Math percentile scores not accounted for in student fall Math percentile scores.

Overlap in Need of Support: Analyses on the percentage overlap between students being recommended for additional support in SEL and those being recommended for additional support in both Reading and Math is in progress.

Importance

This study provides quantitative support for the impact of student social-emotional competence on academic outcomes and shows how technology can be utilized to more effectively deliver those practices. This intricate breakdown of the relationship these constructs share could be of great use to ISTE attendees who may be interested in incorporating either SEL or MTSS teaching practices into their curriculum.

References

Clarke, A., Sorgenfrei, M., Mulcahy, J., Davie, P., Friedrich, C. & McBride, T. (2021).
Adolescent mental health: A systematic review on the effectiveness of school-based interventions. Early Intervention Foundation. Retrieved from https://www.eif.org.uk/report/adolescent-mental-health-a-systematic-review-on-the-effectiveness-of-school-based-interventions

Fletcher, J. M., & Vaughn, S. (2009). Response to Intervention: Preventing and Remediating Academic Difficulties. Child development perspectives, 3(1), 30–37. https://doi.org/10.1111/j.1750-8606.2008.00072.x

He, W., & Meyer, P., (2021). MAP Growth Universal Benchmarks: Establishing MAP Growth as an Effective Universal Screener. NWEA Psychometric Solutions. March 12, 2021. Retrieved from https://www.nwea.org/content/uploads/2021/05/MAP-Growth-Universal-Screening-Benchmarks-2021-03-12_NWEA_report.pdf

Goldstein, D., & Saul, S. (2022). A Look Inside the Textbooks That Florida Rejected. The New York Times. Published April 22, 2022. Retrieved from https://www.nytimes.com/2022/04/22/us/florida-rejected-textbooks.html

Mahoney, J.L., Durlak, J.A., & Weissberg, R.P. (2018). An update on social and emotional learning outcome research. Phi Delta Kappan, 100 (4), 18-23. https://doi.org/10.1177/0031721718815668

Naglieri, J. A., LeBuffe, P., & Shapiro, V. B. (2011). Universal screening for social-emotional competencies: A study of the reliability and validity of the DESSA-mini. Psychology in the Schools, 48(7), 660–671. https://doi.org/10.1002/pits.20586

National Center for Educational Statistics (NCES) (2022). Education Demographic and Geographic Estimates. East St Louis School District 189, IL (Retrieved, 7/26/22) https://nces.ed.gov/Programs/Edge/ACSDashboard/1713320

Northwest Evaluation Association (2004). NWEA achievement level tests and measures of academic progress. Northwest Evaluation Association: Oregon.

O’Boyle, E. H., Jr, Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P. A. (2011). The relation between emotional intelligence and job performance: A meta-analysis. Journal of Organizational Behavior, 32(5), 788–818. https://doi.org/10.1002/job.714

Shapiro, V. B., Kim, B. K. E., Robitaille, J. L., & LeBuffe, P. A. (2017). Protective factor screening for prevention practice: Sensitivity and specificity of the DESSA-Mini. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 32(4), 449–464. https://doi.org/10.1037/spq0000181

Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: The promise and potential problems. Learning Disabilities Research & Practice: A Publication of the Division for Learning Disabilities, Council for Exceptional Children, 18(3), 137–146. https://doi.org/10.1111/1540-5826.00070