Listen and learn: Research paper Research papers are a pairing of two 18 minute presentations followed by 18 minutes of Discussion led by a Discussant, with remaining time for Q & A.
This is presentation 1 of 2, scroll down to see more details.
Other presentations in this group:
With the dual approach of using the Likert-type scale along with the reflection on best assessments, the self-evaluation should provide a more robust and accurate depiction of the student’s learning from the course. This study was designed to investigate two research questions: 1) does the semester length have an effect on the self-evaluation scores by students in online courses, and 2) does the self-evaluation score vary by gender in semester length?
METHODOLOGY
Participants for this study were students (N=223) enrolled in a fully online general education course for 16-weeks during the fall of 2021 or spring of 2022 (N=160) or 10-weeks during the summer of 2021 or summer of 2022 (N=63). The 16-week gender breakdown was N=92 females and N=68 males. The 10-week gender breakdown was N=43 females and N=20 males. Fall 2021 participation in this study was N=87 and spring 2022 was N=73. Summer 2021 participation was N=27 and summer 2022 was N=36.
The university was a regional master’s granting institution in a rural area with an annual enrollment of 6,500 students and a ratio of students to faculty of 22:1. Students at the university were over 70% first generation college students and predominantly Hispanic descent. The university utilized Canvas as its learning management system (LMS).
The course was a fully online, freshman-level Music Appreciation type course with content focused on the development of blues, jazz, and popular music.
Students worked in small groups of up to four during the first eight weeks of the course to establish a peer network within the learning community and support system for the non-musicians in the course. There were group projects and individual assignments due throughout this early stage of the course. The group projects suggested the usage of software such as PowerPoint, Word, or creation of a webpage. These software were then recommended for individual assignments in the last half of the course after usage by the group.
During finals week, students completed a self-evaluation of participation and learning worth 5% of their final course grade using a Canvas assignment tool. The tool stated each of the ten course objectives then provided students with a Likert-type scale of 1 to 5: 1=Needs improvement, 2=Fair, 3=Average, 4=Good, and 5=Excellent. Question eleven asked students to suggest a numeric percentage from 1-100 for their grade in the course. Question twelve then asked students to provide 2-3 paragraphs citing specific assignments to justify why they deserved that grade.
The score of the self-evaluation of participation and learning assignment was five points for questions 1 through 10, totaling 50 points. Whatever score the student entered into the answer was their score for that question. There was no correct answer. Question eleven was worth 40 points. The student’s answer (1-100) was multiplied by .4 to convert the percentage into a value of 40 points. Question twelve was worth 10 points and was subjectively evaluated by the quality of the justification provided by the student. No justification provided was no points. Minimal effort received half credit or five points. Full credit was awarded for answers with abundant reflection and apparent justification for the suggested grade in question eleven.
RESULTS
The self-evaluation scores for short semesters (10-weeks) compared to long semesters (16-weeks) were the first research question for this study. The total short semester (N=63) mean was 87.79 with a median of 95, whereas the total for long semester (N=160) mean was 87.46 with a mean of 90. A student t-Test of two tails resulted in a p value of .7797 with a critical value of 1.9707. The two populations are statistically equal. The variance was 60.60 and 61.58 yielding an Eta (effect size) of .000355. Eta with 0 is a small effect to 1 is large effect. A .06 is a medium effect with a .14 or higher generally understood as a large effect size.
The second research question dealt with the difference of gender in scores on self-evaluation. Students in the long semesters of 16-weeks had an overall mean of 87.46/100 and a median of 90. Females scored an 88.01/100 with a median of 90. Males, however, scored a mean of 86.73/100 and a median of 85. The ANOVA p value = .3070 meaning there is no statistical difference between the genders in the long semester self-evaluation scores.
Students in the short semesters of 10-weeks had an overall of 87.79/100 and a median of 95. Females scored an 87.95/100 with a median of 90. Males scored a mean of 87.45 with a median of 87.5. The ANOVA p value = .8148 suggesting, like the long semesters, no statistical difference between the genders in the short semester self-evaluation scores.
To search for the effect of gender on self-evaluation scores, an ANOVA compared the scores by gender of short and long terms of both males and females. The p value was .790 meaning there were no statistical difference between genders in the either lengths of the semester on self-evaluation scores. The Eta for this was .005339.
DISCUSSION
The hypothesis was that students were generally more positive in their self-evaluation during the short terms of summer in contrast to the longer terms of the fall and spring. That assumption was based on teaching the course over fifty times in the researcher’s career. However, the data suggest that is not the case and the populations are similar. It is critical to note the eta, effect size/power of this study, of the statistical inference is small effect with a score of .000355. Additional data need to confirm or refute this finding. A note of interest is the overall mean for short semester of 87.79/100 while the median was 95 which seems to indicate a high degree of variance in the data.
A shorter term of four or six weeks would be interesting to explore for the effect on self-evaluation scores, yet the researcher does not offer the course in that format length. The long semester of 16-weeks had a mean of 87.4588/100 and median of 90. The shorter semester of 10-weeks mean was 87.7937/100 and a median of 88. While the mean for the shorter semester is higher, the median is lower, yet closer to the mean. Perhaps a shorter term of four to six weeks would further clarify the unity of the median to the mean, or the distance between the mean of the shorter term and longer term.
A similar hypothesis was that female students were more positive than their male counterparts in their self-evaluation scores. The scores in long semesters, short semesters, and combined all confirm that no statistical difference exists between genders in their self-evaluation scores. Despite interesting variances such as mean for females in short semesters as 87.95 whereas the median was 90, there were no statistical differences found in the data. No further investigation of gender variance is warranted in self-evaluation.
Allen, D. & Flippo, R. (2002). Alternative assessment in the preparation of literacy educators: Responses from students. Reading Psychology, 23(1), 15-26.
Andrade, H., Wang, X., Du, Y., & Akawi, R. (2009). Rubric-referenced self-assessment and self-efficacy for writing. The Journal of Educational Research, 102(4), 287-302. https://doi.org/10.3200/JOER.102.4.287-302
Bloom, B. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. David McKay Co, Inc.: New York.
Brown, G. & Harris, L. (2013). Student self-assessment. In J. McMillan (Ed.). Sage Handbook of Research on Classroom Assessment (p. 367-393). Sage: Thousand Oaks, CA.
Boud, D., Lawson, R., & Thompson, D. (2015). The calibration of student judgement through self-assessment: Disruptive effects of assessment patterns. Higher Education Research & Development, 34(1), 49-59. https://www.doi.org/10.1080/07294360.2014.934328
Dungan, A. & Mundhenk, L. (2006). Student self-assessment: A tool for engaging management students in their learning. Organization Management Journal, 3(1), 54-73. https://doi.org/10.1057/omj.2006.7
Geist, E. & King, M. (2008). Different, not better: Gender differences in mathematics learning and achievement. Journal of Instructional Psychology, 35(1), 43-53.
Gopinath, C. (1999). Alternatives to instructor assessment of class participation. Journal of Education for Business, 75(1), 10-14.
[author] & Tapper, L. (2016). The ability of non-music majors to self-evaluate at the end of a music course. Journal of Educators Online, 13(1), 39-57.
Orsmond, P., Merry, S., & Reiling, K. (2002). The use of exemplars and formative feedback when using student derived marking criteria in peer and self-assessment. Assessment & Evaluation in Higher Education, 27(4), 309-323.
Sadler, D. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144.
Severiens, S. & Ten Dam, G. (1994). Gender differences in learning styles: A narrative review and quantitative meta-analysis. Higher Education, 27, 487-501.
Soldner, L. (1997). Self-assessment and the reflective reader. Journal of College Reading and Learning, 28(1), 5-11.
Taras, M. (2016). Situating power potentials and dynamics of learners and tutors within self-assessment models. Journal of Further and Higher Education, 40(6), 846-863. https://www.doi.org/10.1080/0309877X.2014.1000283
Return
Trips and Tours