Direct Measures for Assessing Student Learning
Developing Ratings and Criteria for Capstones and Other Advanced Projects
Students in many Northwestern majors complete at least one advanced course in which they write a research paper, complete a project, prepare a performance, or engage in some other activity that builds on knowledge, abilities, and ways of thinking gained through their earlier coursework. Examples include the senior seminar in Asian Languages and Cultures, the research seminar in History, the senior project in Musicology, the research seminar in Communication Studies, and the Capstone Design course required for majors in the McCormick School of Engineering and Applied Science.
Evaluating student work in these courses is a good way to assess the extent to which students have achieved the learning objectives for the major. The same is true for minors or certificates for which such courses are required.
Steps in the Process
- Decide Which Learning Objectives Are Relevant
Perhaps all of the department’s learning objectives are relevant for these projects, but perhaps not. For example, many departments share the goal of having students gain breadth of knowledge within the field, but breadth may not be evident in a research paper focused on a narrow question.
- Develop a Scoring Plan
The idea is to articulate detailed guidelines for evaluating each student’s standing on each relevant learning goal. It may be helpful to break some objectives down into more specific, measurable components.
- Decide Which Projects To Evaluate
In a small department where all students take the same capstone course, all of their final projects can be evaluated. If a department has more students and they take different courses, one approach would be to look at a sample of projects. For example, each senior in the department could submit the research paper he or she sees as the best, or a subset of projects could be randomly selected from each relevant course.
- Decide Who Will Evaluate the Projects
This could be done by course instructors, but a better approach may be to have a committee of department faculty evaluate student work anonymously. Self-ratings and peer-evaluations can also be useful. For example, if one goal is to demonstrate the ability to work well with a team, then feedback from project teammates can be part of the evidence compiled.
- Carry Out the Plan
Evaluate the projects, and then summarize findings. What strengths and weaknesses do these projects reveal? Do students, on average, excel in some domains, but perform less in others? Are there dimensions on which students’ performance is especially variable—and is the variability related to which courses they took in the department or other experiences they’ve had?
- Think About—and Implement—Changes
If some students’ performance falls short of the objectives, why might that be? What changes might make a difference? This is an opportunity to reexamine the curriculum. Think about possible modifications to requirements and/or to the content of key courses, about ways to enhance opportunities for achieving the department’s learning objectives. Then, institute changes to help students develop the knowledge, skills, and ways of thinking identified by department faculty as signature qualities of strong graduates in that field.
Assessing Learning in Graduate Programs
The principles of assessment shared throughout this site apply to graduate programs as well as undergraduate programs. Assessment at the graduate level does also have some unique features. For example, graduate-level assessment benefits from clarity and transparency in the following:
- Criteria used for examining a Master's or Doctoral Thesis
- Learning objectives for PhD exams and oral defense
- Learning objectives and assessments for independent study
Faculty on thesis and oral committees can benefit greatly from a shared rubric, for example, which brings additional objectivity to that assessment process. The criteria specified in such a rubric can be shared with defending students and serve as a guide for their work and preparation.
Comparing students’ pre-program skills and knowledge with their post-program skills and knowledge can help faculty understand and communicate to others the value of the program. It can also help faculty in continuing to develop the program (e.g., bolstering courses or strategies that foster particular learning outcomes).