Effective Use of Case Studies in the MIS Capstone Course through Semi-Formal Collaborative Teaching
Jeffrey S. Harper, Steven W. Lamb, James R. Buffington
This paper provides the theoretical grounding for the development of an instrument to assess the effectiveness of the case study approach in teaching the higher-level concepts associated with the MIS capstone course. The instrument was administered to students in the capstone class over three semesters. The resulting findings from the survey, combined with an understanding of each instructor's approach to teaching case studies are then used collaboratively by the instructors as a means to identify strengths and areas for improvement. Items in the survey were developed from applicable prior research on teaching and learning and collaborative teaching. Responses to the survey items were subjected to an exploratory factor analysis, resulting in two dimensions of interest-teaching substance and teaching technique. The results of the survey indicate widespread satisfaction by students with the case study approach.
Keywords: Instrument Development, MIS Capstone, Case Studies, Collaborative Teaching, Teaching Substance, Teaching Technique
How can the quality of teaching in the classroom be improved? This question is of central concern to dedicated educators everywhere, resulting in a large body of literature in response. Many techniques for improving the quality of teaching have been proposed, developed, validated, and implemented through the years. Of particular interest in this paper is the technique of using case studies as a primary pedagogical instrument for teaching higher-level concepts in the MIS capstone course.
The authors of this paper are faculty members in the Information Systems and Decision Sciences area. In our discipline, it is not unusual for two or more faculty members to routinely teach the same course, although in different sections, over a number of semesters. Two of us routinely teach the MIS capstone course, alternating semesters. Although we both adhere to a general master course outline, each of us brings our own style, emphasis, techniques, and pedagogy to the class.
Because of our own interest in answering the opening question of this paper, we have been engaged in discussion for many years about what each of us does in the classroom that is particularly effective. As a result, both instructors have placed a greater emphasis on the case study approach to teaching and learning. Each of us maintains an independent approach to how we teach the case studies in our respective classes. However, we continue to search for support that using case studies enhances learning in the course.
This paper provides the theoretical grounding for the development of an instrument that assists us in determining the effectiveness of the case study approach in teaching the higher-level concepts associated with the capstone course. We believe that this research has provided us with substantive results that show the effectiveness of using case studies. In turn, we have used the results of the research to improve our instructional delivery and the quality of instruction in the classroom.
1.1 Collaborative Teaching: A Basis for Improvement
Because two of the authors routinely teach our capstone class, usually rotating semesters, we typically share experiences from the class in an effort to find what works best. We consider this an informal form of collaborative teaching. The structured approach we employ is presented in full later in this section.
Collaborative teaching should be distinguished from collaborative learning. Collaborative learning occurs when two or more students work together, often on a team project. Collaborative learning has long been promoted as an effective classroom methodology by theorists (Vygotsky, 1978). Among the virtues of collaborative learning is that students learn to take advantage of each team member's expertise and to experience first-hand the problems of coordinating a team effort (Goyal, 1995/1996). Studies have shown that collaborative learning leads to a higher degree of satisfaction with the learning process, to a greater motivation to learn, and to better performance (Flynn, 1992). Our experience with collaborative teaching indicates that these virtues also apply to our informal approach to collaborative teaching.
Collaborative teaching is often equated with team teaching, and educators have developed a significant body of research on team teaching. Brabston (1999) offers a typical paradigm for collaborative team teaching. This paradigm proposes three different models for collaborative teaching:
1. The interactive model. Two or more instructors in front of the class at any one time.
2. The rotational model. Each member of the teaching team teaches in only that part of the course related to his or her area of expertise.
3. The participant-observer model. Each team member alternately takes the lead in teaching. The other team member primarily observes but also actively participates when appropriate.
All three models have shown positive results, but all suffer from a serious disadvantage: all require intensive time and energy commitments on the part of the instructors. At first blush, the rotational model may seem to be far less time intensive, but if the various portions of the course are to be seamlessly integrated, time commitments are only slightly less intensive than the other two models.
Buffington and Harper (2002) have presented a fourth model for collaborative teaching. This fourth model, semiformal collaborative teaching, presupposes two or more instructors who teach the same course. To implement semiformal collaborative teaching with regard to case studies, the model was executed by implementing the following steps:
1. Compile a list of generally agreed upon desired outcomes from teaching the cases.
2. Construct a questionnaire designed to evaluate the processes used to achieve the outcomes.
3. Administer the questionnaire at the conclusion of each semester.
4. Meet to discuss questionnaire results; identify methodologies that best meet desired outcomes.
5. Incorporate appropriate methodologies in future classes.
This five-step procedure has several roots. Mentkowski, et al (2000) advocate the building of "collaborative inquiry" as a sustained program. Collaborative inquiry is envisioned as a multidisciplinary inquiry into and reflection upon a vast array of curricular and pedagogical issues, focusing on problems arising from the current state. The semi-formal model is more limited in scope, but provides some of the same benefits afforded by collaborative inquiry.
The work of Paulsen and Feldman (1995) also offers theoretical grounding for the semi-formal model. This report advocates developing a supportive teaching culture. To develop this culture effectively, is it necessary for faculty to be widely involved "in every aspect of planning and implementing improved teaching," [see EbIe and McKeachie, 1985; from p. 31]. The nurturing of these cultures also depends on "opportunities for frequent interaction among faculty regarding teaching related issues" (Massey, Wilger, and Colbeck, 1994; Wright and O'Neil, 1994 [p. 33]. Finally, establishing this culture requires classroom assessment, used to "obtain useful feedback on what, how much, and how well their students are learning [, which they] can use... to refocus their teaching to help students make their learning more efficient and more effective" (Angelo and Cross, 1993 [p. 62]. The semi-formal collaborative teaching model incorporates each of these features.
We believe the goals of collaborative teaching are to improve the content of the material presented to students and to improve the techniques used in conveying that content. One method to accomplish these goals is to produce results consistent with Bloom's (1965) higher levels of learning. Of particular interest in this research into the usefulness of case studies are the higher levels of Bloom's taxonomy, which are referred to in the Instrument Development portion of this paper.
This study was conducted in phases. First, the authors reviewed the literature to determine what is known about collaborative teaching and the relationship between student learning and teaching case studies. Based upon our understanding of the literature in combination with our own teaching experiences, a questionnaire was developed and administered to test the effectiveness of our approach to teaching topics in the field of information systems with case studies. We then performed an analysis of the results of the questionnaire. Finally, the conclusions, implications, and limitations of the study were developed and presented.
2.1 Instrument Development and Administration
Based upon our discussions about how we teach cases, we developed an eight item questionnaire. The questionnaire is intended to assist us in determining the effectiveness of each of our approaches and as a valid and justifiable basis for modification of how we teach.
Students were asked to rate their agreement with each of the eight items using a five-point Lykert scale for responses (1: strongly agree, 2: agree, 3: neutral, 4: disagree, and 5: strongly disagree). The questionnaire was administered to students in the Management of Information Systems (Capstone) class at our university for three consecutive semesters.
The instrument itself (see Appendix A) was designed as an exploratory tool to assess the effectiveness of our approach to teaching case studies. We were particularly interested in students' perceptions of whether the use of case studies facilitates learning and whether the classroom techniques we employed were appropriate.
Both instructors feel that cases are an important tool for teaching key issues, resulting in the statement (Item 1 on the questionnaire):
The cases brought out important points about managing information systems, such as the role of MIS in a global economy, the potential of e-commerce, the role of IT in strategic planning, IT ethics, etc.
Romm and Pliskin (2000) have reported that a basic value of the case approach is in the teaching of soft skills, which led to the development of the statement (Item 2): The cases are a good way of teaching "soft skills"; for le, interpersonal skills and management skills. We also developed an item (Item 3) to assess students' perceptions of cases as agents of active learning, as suggested by Horgan (1 999):
Cases increase the likelihood of student participation in class discussion.
In addition, we believe that the case approach is an excellent vehicle for teaching Bloom's (1965) higher levels of learning, resulting in the development of four statements:
Writing the case report aided in understanding the case principles;
The cases provide students with a good means of applying information systems principles to real world situations;
The cases provide with a good opportunity to synthesize; that is, i potential solutions to a case problem and the most solution; and
The cases provide students with a good opportunity to exercise evaluation skills, (i. e., appraising the extent to which particulars are accurate, effective, economic, or satisfying.
These items were included in our instrument as items 4, 5, 7 and 8, respectively.
Finally, we wanted to learn whether students preferred our new approach to the more traditional case approach, which accounts for Item 6:
I prefer the approach to cases used this semester to the traditional case approach, i.e., when student teams are assigned the responsibility of presenting a particular case.
Each instructor administered the survey to his class(s) at the end of the semester. All students in each class were asked to complete the questionnaire, resulting in a total of 68 responses (22 in the first spring class, 20 in the fall class, and 26 in second spring class).
In order to draw meaningful conclusions from this study, the results of the questionnaire were analyzed in the following ways:
1. An analysis of the questionnaire results was performed to determine whether the items in the exploratory questionnaire identify meaningful constructs that are of value to our continuous improvement goals.
2. The mean response and frequency distribution to each item was examined to determine the extent of student agreement with each statement.
3. Comparison of the results by instructor was conducted as a vehicle for collaborative teaching and continuous improvement.
3.1 Component Analysis
A factor analysis of the survey responses was conducted to determine whether the items in the questionnaire measure meaningful constructs. If instructors are to modify their teaching approach based upon the results of a survey, they should have confidence that the instrument itself is reflecting compatibility with intended competencies. Also, such testing allows for successful redesign of questions when the instrument is used in successive rounds.
As Hair, et al. state, 'The general purpose of factor analytical techniques is to find a way of condensing (summarizing) the information contained in a number of original variables into a smaller set of new composite dimensions (factors) with a minimum loss of information; that is, to search for and define the fundamental constructs or dimensions assumed to underlie the original variables".
In order to ascertain the underlying dimensionality of the 8 statements, factor analysis procedures were used. First, principal component analysis was conducted using the correlation matrix of the 8 variables (items) as input. Principal component analysis yielded the following set of eigenvalues:
Forty percent of the variance to the responses to the eight items in the survey was associated with the first component. Component two was associated with about 13 percent of the variance with an eigenvalue of slightly more than one. Thereafter, the eigenvalues were less than one, indicating that each successive component had less explanatory power than the original item. Therefore, there is no justification for examining the factor loadings associated with the first two components.
Table 2 presents the principal component analysis factor matrix for the first two components. The items are listed in order of the magnitude of their correlation value with component one. Items 8, 5, 1, 2, 6 and 7 are more highly correlated with component one than they are with component two. Each of these six items is positively correlated with the first component.
Items 3 and 4 are more highly correlated with component two than with component one. Item 3 is positively correlated with component two, while item 5 is negatively correlated with that component. Item 2 has an absolute correlation value of slightly more than 0.40 with the second component, but it has a greater correlation with component one, and therefore should be identified primarily with component one. All other items have absolute correlation values of less than 0.30 with component two and thus coefficient of determination values (r2) of 0.09 or less with a resulting minor variance associated with component two. As Hair, et al. indicate, loadings less than 0.30 are not important when interpreting a factor.
Thus, the analysis of the responses to the eight items in our exploratory questionnaire has identified two components. We shall call the first component teaching substance. This construct represents the perception of effective student learning consistent with our approach to teaching with case studies. The second component, we call teaching technique. This construct represents the perception of effective classroom methodology relating to the teaching of case studies.
An investigation of the output of orthogonal rotation procedures employed on selected factor matrices was used to determine whether or not additional insight into the dimensionality of this instrument could be gleaned. The rotation procedures employed on both a two-factor matrix and a three-factor matrix did not prove to yield additional insight.
3.2 Mean Agreement With Questionnaire Statements
The mean agreement to the closed-ended items in the survey is shown in Table 3. Breakdown of the survey results by individual class/instructor is included in the Appendix. Questions have been rank-ordered by strength of agreement.
As can be seen from the table, the student respondents generally perceived each item positively, with all items having a mean below 2.0. In addition, those items with which the students most strongly agreed also had the lowest standard deviation, indicating greater consensus. There is slightly less consensus among the student responses (standard deviation increases) as agreement weakens.
3.3 Inter-instructor Rankings
Table 4 presents the rankings derived from the item means for each of the three classes to which the questionnaire was administered. The means for each item in each class administration can be obtained from the Appendix. One instructor (A) taught the two spring classes, another (B) taught the fall class. A lower mean (and, thus, a higher ranking) indicates that the students responded more positively to the item.
Items 5, 1, and 7 ranked highest for Instructor A in both semesters. Item 3 is ranked highest with Instructor B, followed by items 5, 1, and 7. The average of responses in each class was positive (less than 3.0), with 3.0 indicating neutral.
4. CONCLUSIONS AND IMPLICATIONS
This study reports on the development of an assessment instrument that is specific to the material being taught and grounded in pedagogical theory. The results of the survey indicate a widespread satisfaction with the use of case studies in the course. Students seem to be particularly satisfied with cases as tools for making abstract MIS principles concrete. The mean responses on items 5, 1, and 7 (all of which address teaching substance) indicate that students strongly believe that case studies provide students with a good means of applying information systems principles to real world situations, bring out important points about managing information systems, and provide students with a good opportunity to synthesize. Items 5 and 1 were two of the three highest ranked items in the teaching substance component. We were gratified that these items were also the most positively viewed by the students.
As a result of our interpretation of the results, collaborative teaching has taken place. In a discussion of the results after each instructor had administered the survey once, we noted that Item 6 (preference over the traditional team-based approach to case assignments) did not rank very high for either Instructor A (1st semester) or Instructor B. Accordingly, during the second spring semester, Instructor A made each individual responsible for completing case assignments, as opposed to the team assignments he had made in the past, as the survey results indicate that this practice is preferred by the students. The results from the subsequent survey of Instructor A' s second class indicated a positive change for this area, as Item 6 moved from eight in the rankings to fourth.
Along the same lines, Instructor B now also makes each individual responsible for completing case assignments. We believe that these adjustments in teaching technique using case studies will improve the quality of the learning experience in our respective classrooms.
The results of this study must be interpreted with caution. Apparent student satisfaction with the case study approach to teaching capstone MIS topics does not ensure that the methodology was more effective than other teaching methods. Also, there could have been contaminating factors in our study - the students may have found that pioneering a new methodology was a positive experience. Or, the instructor may have done a more effective job for all aspects of the course - pulling the case study results up by the bootstraps, as it were. Also of concern was the relatively low comparative ranking of Item 2 ("soft skills"). While the students agreed that the case study methods employed by both instructors were a good way of teaching soft skills, this agreement was only slight. Perhaps, case studies may not, by themselves, be the best approach for emphasizing soft skills.
Nevertheless, the evidence does point to the use of case studies as being effective in applying the theoretical linkages to teaching described in this paper. Indications are that the semi-formal model for collaborative teaching using case studies is a realistic and practical method of collaboration between instructors teaching the same course in different sections. Instructors who choose to collaborate in this way can discuss, compare, and contrast their individual approaches and styles in the classroom to identify teaching content and teaching techniques to improve their classroom deliveries.