Address Individual Differences in Problem-Solving Instruction: An Alternative Design Model

Framework

1. Problem-Solving skills
No matter which type of problems have been solved and what steps people used to solve problems, problem-solving is a personal cognitive process that aims to complete the task. Two sets of skills are involved in this process: recurrent skills and non-recurrent skills (Lee, 2010; Van Merriënboer, 2013). Recurrent skills refer to the ability to understand of the problem and proficiency in performing related procedures and rules. Non-recurrent skills refer to the ability to correct and improve performance in problem-solving process, such as reflecting, comparing, reasoning, monitoring, and decision-making.
2. Learning Readiness
Individual difference is an important issue that should be taken into consideration in problem-solving instruction. According to Jonasson and Ggrabowski (1993), students come to the classroom with different learning abilities and willingness. Ability refers to the demonstrated recurrent skills and non-recurrent skills while willingness refers to learner’s motivation or desire to complete the required learning task. Individual differences are reflected as various learning readiness statuses, which are eight combinations of ability and willingness. Each learning readiness status is defined as below.
• Learning Readiness Status 1 (R1). R1 represents student who lacks all recurrent skills, non-recurrent skills, and willingness.
• Learning Readiness Status 2 (R2). R2 represents student who has recurrent skills but lacks non-recurrent skills and willingness.
• Learning Readiness Status 3 (R3). R3 represents student who has non-recurrent skills but lacks recurrent skills and willingness.
• Learning Readiness Status 4 (R4). R4 represents student who has recurrent skills and non-recurrent skills but lacks willingness.
• Learning Readiness Status 5 (R5). R5 represents student who lacks recurrent skills and non-recurrent skills but willing to learn.
• Learning Readiness Status 6 (R6). R6 represents student who has recurrent skills and willingness but lacks non-recurrent skills.
• Learning Readiness Status 7 (R7). R7 represents student who has non-recurrent skills and willingness but lacks recurrent skills.
• Learning Readiness Status 8 (R8). R1 represents student who has recurrent skills and non-recurrent skills and also is willing to participate in learning.

3. Situational Design Model
Situational design model has defined procedural learning activities to describe learning activities that aim to develop recurrent skills. Supportive learning activities are defined to describe learning activities that aim to develop non-recurrent skills. Relationship activities are defined to describe instructional activities that focus on managing learners’ willingness. The situational design model (Figure 2) emphasizes that instruction should match students’ differences in learning ability and willingness. Instructional treatments for learning readiness statuses are summarizes in Table 1.

Table 1
Instructional treatment on each learning readiness status
Learning readiness status Instructional treatment
R1 HP-LS/LR(S1): high procedural learning activities, low supportive activities, low relationship activities
R2 LP-HS/HR(S2): low procedural learning activities, high supportive activities, high relationship activities
R3 HP-HS/HR(S3): high procedural learning activities, high supportive activities, high relationship activities
R4 LP-LS/HR(S4): low procedural learning activities, low supportive activities, high relationship activities
R5 HP-LS/HR(S5): high procedural learning activities, low supportive activities, high relationship activities
R6 LP-HS/LR(S6): low procedural learning activities, high supportive activities, low relationship activities
R7 HP-HS/LR(S7): high procedural learning activities, high supportive activities, low relationship activities
R8 LP-LS/LR(S8): low procedural learning activities, low supportive activities, low relationship activities

For learners who possess neither recurrent skills nor non-recurrent skills, such as R1 and R5 students, instructional treatments with high procedural learning activities (HP) but low supportive learning activities (LS) are more appropriate for R1 and R5 learners to ensure completion of required learning tasks. For learners who possess part of required skills, such as R2, R3, R6, and R7 learners, instructional treatments with both high procedural learning activities and supportive learning activities (HS) are more suitable for R3 and R7 learners. R2 and R6 students need low procedural learning activities (LP) and high supportive learning activities (HS). For learners who already possess required skills, such as R4 and R8 learners, decreasing both procedural learning activities and supportive learning activities is more appropriate as prescribed by remediation match model.
In terms of willingness issue, there is no doubt that high relationship activities (HR) are necessary for less motivated learners while low relationship activities (LR) are more beneficial for highly motivated learners. However, R1 and R5 learners are exceptions of this rule according to cognitive load theories. Since R1 and R5 learners do not have both recurrent and non-recurrent skills, cognitive load requirement is much higher for R1 and R5 learners. Instructional treatment should avoid adding extra cognitive load for R1 and R5 students. Thus, low relationship activities are more suitable for R1 students while R5 students need high relationship activities.

Methods

Situational design model was piloted in an introductory technology course in workforce education program at a large southern research university in the United States. This 12-week course consists of two learning modules, each of which is six-week long with different topics. Participants were required complete three project assignments in each module and take an exam at the end of each module. The following questions were examined in this pilot study:
1. Is situational design model effective in improving students’ real-life problem-solving skills?
2. Is situational design model effective in improving students’ academic performance?
3. Is situational design model effective in improving students’ learning readiness status?

Research Design
Single group repeated measurement design was used in this study. Situational design model was applied on the second module. Based on students’ learning readiness statuses, they were divided into five groups in the 8th week and five different instructional treatments were applied in the second module.

Participants
Participants were 11 students (3 female and 8 male) enrolled in an introductory technology course in a large south research university in the United States.

Data collection
Grades of the two exams were collected to answer the first research question. The two exams had the same format that consists of two parts: 20 multiple choice questions (part I) and a scenario-based project (part II). Multiple choice questions aim to test students’ recurrent skills while scenario-based project is to examine students’ non-recurrent skills.
Sum of the three project assignments’ grades were used to compare students’ academic performance. Participants could earn a maximum of 300 points on all the three project assignments in each learning module.
A 6-point Likert learning readiness survey developed by the researcher was used to answer the third research question. The survey contains 6 items that ask participants to rate their recurrent skills, non-recurrent skills, and willingness. The survey was delivered to participants three times: the beginning of the semester, in the middle of the semester after midterm exam, and the end of the semester after final exam.

Data analysis
Rubrics developed by the researcher were used to calculate the grades of the two exams. Two trained raters (doctoral students) scored the exams independently and average scores between the two raters were used for further data analysis.
Grades of the three project assignments in two modules were added together to determine students’ academic performance. Participants could earn a maximum of 300 points on all the three project assignments in each learning module.
An interrater reliability analysis using the Cohen’s Kappa statistic was performed to determine internal consistency among the raters in terms of the two exams. Reliability coefficient examination using Cronbach’s Alpha was performed to determine reliability of the learning readiness survey. One-way repeated measures MANOVA was used to examine the effect of situational design on learners’ academic performance, problem-solving skills, and learning readiness status.

Results

The interrater reliability for the raters was found to be Kappa = 0.80 > .75 (p<.001), 95% CI (0.55- 1.00). The learning readiness survey was found to be reliable (6 items, Cronbach’s = .73 > 0.7). Using spehericity assumed values, MANOVA results showed significant effect on problem-solving skills (F (1, 10) = 19.17, p < .05) and academic performance (F (1, 10) = 7.39, p < .05). See Table 2.

Table 2
Means, standard deviation for problem-solving skills and academic performance and results of MANOVA

  MANOVA
 Module 1 Module 2 Situational Design
 Mean SD Mean SD F (1, 10) p
Problem-Solving 74.45 18.19 96.09 4.75 19.17 .001
Academic Performance 237.93 24.55 254.82 26.01 7.39 .022
Full score of problem-solving exam is 100. Full score of academic performance in each module is 300.

Mauchly’s test indicated that the assumption of sphericity had been met, χ^2 (2)=.65,p= .145>.05. Therefore, spehericity assumed values were used for further analysis of learning readiness. The spehericity assumed values showed significant effect on learners’ learning readiness status (F (2, 20) = 7.20, p = .004 < .05). See Figure 4.

Figure 4. Learning readiness status changing

Within-subject analysis indicated significant interaction between academic performance and learning readiness statuses (V = .38, F (1,10) = 6.15, p = .033 < .05) and interaction between problem-solving skills and learning readiness statuses (V = .65, F (1,10) = 18.67, p = .002 < .05). However, interaction of academic performance and problem-solving skills was not significant (V = .27, F (1,10) = .28, p = .61 > .05).

In this study, situational design model was found to be effective in improving students’ problem-solving skills, academic performance, and learning readiness status. This finding indicated that customizing instruction according to students’ learning ability and motivation as indicated by learning readiness status is a good approach to improve students’ academic performance and develop problem-solving skills. This finding is in line with the work done by Salden, Aleven, Schwonke, and Renkl (2010) and Kayyuga and Sweller (2004, 2005), who demonstrated that adaptive instruction improves students’ learning performance. This finding also is consiste with Salden, Aleven, Schwonke, and Renkl’s recommendation (2010) that adaptive instruction should consider students’ prior knowledge to determine the initial level or baseline of instructional support.

This study also found that student’s academic performance and problem-solving skills were changing with learning readiness status but direct relationship between academic performance and problem-solving skills was not found. This finding indicated that academic performance improvement would not guarantee acquisition of problem-solving skills. The finding is contradicted with Danner et al. (2011) and Wüstenberg, Greiff, and Funke’s (2012) studies, which indicated problem-solving skills are predictors of academic performance. The possible reason is the low participant number in this study.

Importance

The implication of this study is that the proposed situational design model has accommodated individual differences in instructional design, which fills a prominent gap in problem-solving skills instruction. Although students’ differences in learning have been informed in many instructional design models, feasible ways of accommodating those differences in instructional design virtually not exist. This situational design model has provided a practical approach to accommodate students’ differences in instruction by conceptualizing individual differences in different learning readiness status and by matching those statuses with appropriate instructional styles. This model will help practitioners, especially instructors, to develop personalized instruction that can better satisfy students’ individual learning needs.

References

Brophy, J. (1985). Teacher influences on student achievement. American Psychologist, 41, 1069-1177.
Hall, T., Vue, G., Strangman, N., & Meyer, A. (2003). Differentiated instruction and implications for UDL implementation. Wakefield, MA: National Center on Accessing the General Curriculum. (Links updated 2014). Retrieved from http://aem.cast.org/about/publications/2003/ncac-differentiated-instruction-udl.html.
Hwang, G., Sung, H., Hung, C., Huang, I., & Tsai, C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60(4), 623-638.
Jaciw, A., Toby, M., Ma, B., Lai, G., & Lin, L. (2012). Measuring the average impact of an iPad algebra program. Palo Alto, California: Empirical Education.
Lee, C. B. (2010). The interactions between problem solving and conceptual change: System dynamic modeling as a platform for learning. Computers and Education, 55(3), 1145-1158.
Van Merriënboer, J. J. G. (2013). Perspectives on problem solving and instruction. Computers & Education, 64(1):153-160.