11. Evaluation

To assess the Consortium's progress in meeting its goals of systemic change in quantitative collegiate instruction, a comprehensive evaluation of the proposed work is needed to observe changes in both student and faculty performance and attitudes. There will be one-page surveys completed by all faculty trying reformed instruction, student course evaluations of reformed courses, and progress reports from project personnel.

More formal evaluation studies will be undertaken to measure the amount of change occurring. A sample of faculty (including participants and non-participants) from each institution will be interviewed annually for the duration of the project. Faculty interviews will gather qualitative information concerning a) attitudes toward modes of instruction, b) attitudes toward student learning, c) use of technology, and d) curriculum design. The evaluation will be run by co-PI Anne Preston, with assistance from a social sciences graduate student trained in assessment methods.

Because participation in the project is voluntary rather than random, unbiased evaluations of change in behavior are hard to develop. However, to address this problem, we intend to compare a) participating faculty attitudes before and after program participation and b) participating faculty attitudes to nonparticipating faculty attitudes. Comparing participating faculty attitudes before and after participation may understate the impact of the participation since faculty who choose to participate in the program may already have attitudes in line with those we are attempting to establish. On the other hand, comparing participating faculty attitudes to attitudes of a control group may overestimate the effects of the program because of opposing attitudes in nonparticipants. As a result, the two comparisons together should give a realistic estimate of the range of impacts of this program on faculty attitudes and instruction. Feedback from interviewed faculty will be crucial in fine-tuning the project activities.

To measure the project's impact on student performance and attitudes, a random sample of students will be selected from courses implementing the project's major goals (changes in modes of instruction, use of technology, coordination with other disciplines) and compared to a matched control group with similar GPAs, majors, etc. taking an unreformed version of the same course, a previous (unreformed) offering of the course, or a course judged to have similar general goals and clientele. Immediate and longer term performance will be compared for the two groups. In particular, through surveys and college records, we will document and compare the following measures: 1) grades in the quantitative courses; 2) grades in courses whose connections to quantitative disciplines have been illustrated in the quantitative courses; 3) continuation in courses with quantitative content; and 4) continuation in a career that requires quantitative skill. These students will also be questioned about specific reformed courses and coordination among courses. All survey results will be used for feedback to make continual improvements in individual courses and the overall educational environment that the project seeks to create.

Student surveys will collect information on demographics, family background, ability, and other external factors that could also affect the measures we will be comparing. Regression analysis will be used to determine whether participation in the innovative instruction has a positive effect on the four measures and whether the effects of participation differ for students of different gender, different races, and different abilities.

Previous | Contents | Next

The Long Island Consortium is sponsored by the NSF Initiative: Mathematical Sciences and Their Application Throughout the Curriculum, DUE #9555142. The original NSF proposal can be accessed by clicking here.