Bridging the Digital Divide Through Blended Learning for Freshmen Engineering Students

Framework

In a physical classroom, where students are able to socially connect with each other and the instructor, the process of learning becomes very seamless. Eye contact allows the instructor to understand whether his/her words are being understood, and the student
can respond without the fear of being graded. Hence, the concept of creative interactive video lectures using the Edpuzzle platform was utilized in this course to replicate the exact in-class learning experience.

The primary elements of the adopted pedagogy are:
1) Video assignments with interleaved questions
2) Live sessions for discussions and problem-solving
3) Hands-on practical sessions on simulator
4) Continuous evaluation through quizzes; end-term evaluation through lab exam, viva, and written exam
5) Student Feedback

The course is a common first-year course for all engineering students, and on an average has 200+ students enrolled per semester.

Study material such as lecture notes, chapter excerpts, electronic resources, etc. were uploaded as study material on Google Classroom. However, there is no way (in conventional LMSs) for the instructor to know if a student has actually reviewed the reading material before coming for a discussion session. Here, the Edpuzzle platform was used to assign video lectures as ‘video assignments’ to the students since the Edpuzzle scores can be directly imported to Google Classroom.

The primary framework of this research involves analyzing the response and performance of students with interactive videos, in comparison with a control group of their peers. Further, 3D immersive platforms were used for informal interactions with the students of the treatment group but its empirical impact is yet to be analysed. This paper therefore focusses on analysis of 'engagement' through interactive video lectures.

Methods

The steps followed in this methodology are as follows:

Step 1: Preparing the Video Lectures
Each video contained audio-visual elements, demonstrations and animations, and both typed and handwritten elements (using digital pen tablet) to cater to preferences of all types of learners. While each video may be longer duration, for the purpose of assigning them to students, each video was broken into shorter modules of less than 30 minutes in order to retain the attention of the student. Ideally the video duration should be 15 minutes, but owing to difficulty in maintaining continuity of engineering-level topics, a duration of 20-29 minutes was adopted for most of the videos. Simple questions were designed (4-5 per video module of 25 minutes) to be interleaved in the videos. Questions are inserted at appropriate times in the video to replicate in-person classroom interaction.

Step 2: Uploading as Video Assignments
Prepared video modules are posted on the Edpuzzle platform as ’Assignments’. Students are not allowed to skip questions (if the instructor chooses to do so), and must answer each question in order to continue the assignment. A student may re-watch certain portions as many times as they desire. A student cannot fast-forward the video, nor can they change the playback speed. The correctness of the response (for MCQs) is available instantly and the student cannot change his/her response after that. The instant response can be supplemented with feedback from the instructor, or some detailed explanation of correct/incorrect answers. The video also cannot be played in the background and would be played only in the active browser tab.

The following analytics are recorded for each student per assignment.
• Percentage of video completed
• Response to each question and total score
• Watch-time
• Status of turning in the assignment (On-time or Late) and turn-in time

These statistics are then compared in terms of their mean and standard deviation. We also use k-means clustering to investigate if the analytics support the student classification into four groups:

Group 1: Low Watch-time, High Score
These students watched the video for a minimum time very attentively, and therefore were able to answer all questions correctly.

Group 2: High Watch-time, High Score
These students re-watched certain portions of the video for better understanding, and only then were able to answer all questions. The time spent in re-watching certain portions denote efforts on the part of the student, and is a positive attribute.

Group 3: High Watch-time, Low Score
These students re-watched portions of the video but still could not achieve the correct level of understanding.

Group 4: Low Watch-time, Low Score
These students watched the video just once (since it was mandatory), but did not pay attention to the content. This shows both low effort and motivation in the student.

Among the above categorized groups, Groups 1 and 2 do not need special attention from the instructor. However, Groups 3 and 4 require different types of treatments. This work is still in progress and k-means clustering algorithm has been used to determine the four clusters. The decision boundaries of these clusters are fuzzy and need more work.

Results

The student responses in terms of the parameters mentioned earlier were carefully analyzed over two semesters. These parameters contain valuable information regarding the quality of the video, understanding and engagement, and the overall experience with the provided learning material. In this paper, we will be looking at the watch-time–score observations.

During a semester of Academic Year 2020-21, 230 students were enrolled in this course. This set of students would be referred to as Set 2 (Treatment Group). Let us discuss the ob- servations from the example of one such video assignment on ’closed-loop operation amplifiers’ of duration 27:48 minutes. The fundamentals of diodes and transistors had already been discussed in prior modules. The introduction to opamps had been discussed in the preceding video. Of the 230 students, 226 students completed this 27-minute video assignment with an average of 81.1% score (SD = 21.5), and an average watch-time of 37 minutes. Note that this video assignments is the first time the students are introduced to the specific topic. The questions interleaved in these videos are extremely simple, and the score in these video assignment do not reflect overall performance, but just a measure of attentiveness to the video material.

The Edpuzzle ‘engagement’ is a partial component in the continuous evaluation, in terms of completion and involvement, and not scores obtained. However, in order to validate the efficacy as well as reception of flipped classroom in this course, we may look at both the increase in engagement as well as the performance in quizzes held after completion of opamp-related topics. The first set of 129 students (Control Group, Set 1) were provided this video module only as an unlisted YouTube video. According to the YouTube analytics data for the first ten days of posting and sharing with students, the video got 300 views (indicating approximately two views per student), but only 7:02 minutes of Average View Duration (in comparison with its actual 27-minute duration, i.e. 25.3% Average percentage view. This showed that while the students did have access to the video, in the absence of a monitoring mechanism and an interactive element, they chose to watch less than 25% on an average. There was no mechanism to monitor their consumption of the study material, and hence an overall low engagement was observed.

On the other hand, with a detailed monitoring system in place along with interactive video assignments, students are mandated to go through the material, though at a time of their own convenience. As mentioned in the previous section, for this particular video module itself, 226 out of 230 students (Treatment Group) who received the interactive video assignment, completed the full assignment. For the course with 60 such video assignments, spanning about 24 hours, the Treatment Group (Set 2 of 230 students) spent an average watch-time of 27.4 hours on the videos, with a standard deviation of 11.2 hours. Considering the specific example of the 27-minute video on closed-loop opamps, the Treatment Group (Set 2), on an average, spent 37.6 minutes on this video, with a standard deviation of 18.5 minutes. By providing eight more similar video modules on the topics related to more topics on operational amplifiers, a post-quiz was conducted on these topics. The level of difficulty was the same for both sets of students. The class average for Set 2 for this particular quiz was found to be higher than Set 1 by 14%. It must be noted that both the quizzes was held with utmost care with three different sets of question papers and relatively less time to prevent cheating as far as possible. While the average for Set 1 was 11/20, that observed for Set 2 was 13.8/20 under the same difficulty level, duration and guidelines. While this increase in class average may not be a direct result of interactive video lectures alone, but they indicate a strong performance of flipped teaching, in amalgamation with other factors like more watch-time, extra tutorials and interactions.

The self-paced flipped classroom model has been very popular among the students during its implementation in the first-year course, especially in view of distance learning over the past 1.5 years. For Set 1, the course was only completed through partial flipped teaching, while for Set 2, it was almost entirely through flipped teaching. After completion of the course, surveys were taken from the two sets of students. In a survey from 89 student respondents (from Set 1), video lectures were rated as the most useful tool for students during remote learning, followed by lecture notes and assignments. The students for whom the entire course started completely online in the Fall of 2020 (Set 2), a survey from 92 respondents showed that the self-paced flipped classroom through video lectures were considered widely successful and very useful by the students with an average scores of 4.2/5 in terms of usefulness.

3D immersive platforms including Virbela and Frame VR were used for informal interaction with the students of treatment by creating a virtual 3D classroom and breakout rooms where students could come together and discuss. A 3D walkthrough of the basic electronics lab was created where students could walkthrough the actual lab and identify equipment. This was particularly appreciated by these students who had never yet visited the college campus as it helped create a sense of belonging. While these VR interactions were not used for assessment, they helped create a strong and collaborative learning environment during these challenging times. The use of 3D platforms were widely appreciated in the end-term student feedback comments.

Importance

The use of flipped classroom blended learning with interactive video assignments has proven to be a highly useful and indispensible approach during distance learning, especially since it gives students the flexibility to view/study the given material at a time of their convenience (within a certain deadline, of course), but at their own pace. The format of providing interactive video lectures as a flipped classroom study material has garnered the support of more than 80% of the students in the author’s courses. About 10-15% of students feel the need for more time and spacing between these video assignments so that they are not overloaded with offline assignments. About 5% of students prefer learning a topic in-class for the first time, and feel the need of a live instructor in understanding topics.

This study has been focused from the point of view of student experience, but it should also be noted that creating video content for every topic and editing to remove unnecessary gaps itself is a monumental task for any instructor. However, once a pool of video modules is ready, the analytics derived from such video assignments can provide a significant advantage to the instructor by helping understanding student learning patterns, shortcomings of the video content, motivation and engagement of the student in the material, etc. All these attributes, when collated and analyzed with reference to student performance in exams/quizzes, student feedback, and statistical data, can provide great insights to a teacher in shaping meaningful and enjoyable learning experiences for the learners.

References

[1] “The Great Digital Divide: Why bringing the digitally excluded online should be a global priority,” Capgemini Research Institute, Report, May 2020. [Online]. Available: https://www.capgemini.com/research/the- great-digital-divide/. Accessed on 10 January 2020
[2] W. Dick and L. Carey, The Systematic Design of Instruction, 6th ed. Allyn & Bacon, 2004.
[3] G. S. Mason, T. R. Shuman, and K. E. Cook, “Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course,” IEEE Transactions on Education, vol. 56, no. 4, pp. 430–435, 2013.
[4] R. Gullayanon, “Flipping an engineering mathematics classroom for a large undergraduate class,” in 2014 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), 2014, pp. 409–412.
[5] L. Gren, “A flipped classroom approach to teaching empirical software engineering,” IEEE Transactions on Education, vol. 63, no. 3, pp. 155– 163, 2020.
[6] J. Urquiza-Fuentes, “Increasing students’ responsibility and learning outcomes using partial flipped classroom in a language processors course,” IEEE Access, vol. 8, pp. 211 211–211 223, 2020.
[7] J. Ranalli and J. Moore, “Targeted flipped classroom technique applied to a challenging topic,” in 2016 IEEE Frontiers in Education Conference (FIE), 2016, pp. 1–4.
[8] C. Tang, H. Lin, L. Zhang, T. Bao, Y. Zhang, and Z. Liu, “Electrical specialty experiment teaching reform method with flipped classroom,” in 2020 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), 2020, pp. 1–5.
[9] B. Kerr, “The flipped classroom in engineering education: A survey of the research,” in 2015 International Conference on Interactive Collaborative Learning (ICL), 2015, pp. 815–818.
[10] Z. Liang, M. G. da Costa Junior, and I. Piumarta, “Opportunities for improving the learning/teaching experience in a virtual online environment,” in 2020 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), 2020, pp. 243–250.
[11] H. Al-Samarraie, A. Shamsuddin, and A. I. Alzahrani, “A flipped classroom model in higher education: a review of the evidence across disciplines,” Educational Technology Research and Development, vol. 68, 2020.
[12] Y.-C. Jiang and S.-Y. Jong, “Learner preparedness in flipped classroom: A case study of a flipped postgraduate course,” in 2020 International Symposium on Educational Technology (ISET), 2020, pp. 57–61.