 |
|
| |
|
|
|
A
DISCUSSION PAPER ON TEACHING EVALUATION |
| |
| Background
|
| Several
recent initiatives at Mount Allison have helped to foster a climate committed
to teaching excellence and continuous improvement, for example the creation
of a new Strategic Statement in 2007 and the Academic Renewal Process begun
in 2008. As our efforts in these areas mature, so does our need to measure
and document our progress and our success in achieving our shared and individual
goals; external agencies (such as the Maritime Provinces Higher Education
Commission) require such quality assurance and individual professors and
students benefit from it. |
| |
| While
Mount Allison does not yet have a systematic mechanism for teaching evaluation,
there have been significant landmarks towards its creation. In 2002, the
Committee on Teaching Evaluation made several recommendations and coordinated
a pilot project using the QUEST teaching evaluation form from Queen’s
University; beginning in 2005 and continuing for almost four years, the
Senate Committee on Teaching and Learning has conducted an extended pilot
project using the Student Evaluation of Educational Quality (SEEQ) form
online within WebCT; and many departments have long-standing evaluation
forms that many professors use. |
| |
In
September 2006, the University Senate passed a resolution mandating that
evaluations be conducted at the end of every course offered through Mount
Allison University; however, despite broad support for this mandate, its
implementation has been complicated by the variety of forms and approaches
for collecting student evaluations described above and the low student
response rates to the online form.
To bring clarity and consistency to the implementation of its September
2006 mandate, Senate approved the creation of an Ad Hoc Committee on Student
Evaluation of Teaching in October 2008. That Committee is charged with
drafting a Policy on Evaluation of Teaching.
|
| |
| The
purpose of this discussion paper is to provide some context for that policy
document by presenting information on the sources, types, purposes, and
administration of teaching evaluation. In its research, the Ad Hoc Committee
on Student Evaluation of Teaching was especially impressed by and relied
on the 2008 report Student Course Evaluations: Research Models & Trends.
Authored by Pamela Gravestock & Emily Gregor-Greenleaf (University of
Toronto) for the Higher Education Quality Council of Ontario, the report
is a comprehensive meta-analysis of the literature and best practices surrounding
teaching evaluation. |
|
| SOURCES
OF TEACHING EVALUATION |
| |
The
literature on evaluation of teaching supports the view that teaching evaluation
should be based on evidence gathered from a variety of sources. Furthermore,
the literature agrees that student evaluation of teaching (SET) is one
useful measure of teaching effectiveness and a valuable source of evidence.
|
| |
| There
is also general agreement in the literature that SET is reliable and the
forms used are consistent and valid tools for evaluation provided they ask
about things students are qualified to critique. These items, which are
common to almost every inventory of what constitutes effective teaching,
include such qualities as preparation, rapport, enthusiasm, accessibility,
fairness in assessment. |
| |
| However,
just as teaching is a multi-dimensional activity, so are multiple sources
necessary for its evaluation. Best practice in teaching evaluation points
to using various measures contextualized over time. In addition to student
evaluations collected at the end of a course, these other measures include
students’ responses to mid-term teaching evaluations, peer review
(formal and informal), and self-reflection and self-assessment by the individual
professor. |
|
| TYPES
OF EVALUATIONS |
Many
institutions assign the definition “formative” to the open-ended
questions which occur on most evaluation instruments and “summative”
to the questions which require a numerical response. In fact, the terms
formative and summative refer not to the type of question asked but to
the end use of the data.
The primary purpose of formative evaluation is teaching improvement; summative
evaluation is evaluation that informs personnel decisions, such as hiring,
promotion, or tenure.
|
| |
| At
Mount Allison, teaching evaluation data are disclosed only to the individual
professor who then decides how, whether, and to whom they are revealed.
For this reason, because it is primarily confidential and used by individual
professors to develop their teaching, Mount Allison’s current practice
can be correctly described as “formative.” Moreover, the literature
finds that students tend to focus on the formative aspects of teaching evaluation,
and are not always aware of how data are used in general or by administrators
in particular to make personnel decisions. |
|
| EVALUATION
FORMS |
| It
is common practice in most universities to have a single teaching evaluation
form that contains both numerically scored and open-ended questions. Many
universities mandate a common form, or at least a number of core questions
common to all forms. All forms have at least one global question that asks
students to give an overall rating of the course and/or the professor. These
global questions are important because, when administrators and selection
committees look at teaching evaluation data in order to make personnel decisions
such as hiring or promotion, they tend to focus on the general ratings or
“overall” questions rather than specific indicators or items.
|
| Most
evaluation forms focus on five common measures of effective teaching: |
| 1) |
course content |
| 2) |
communication
skills |
| 3) |
student-teacher
interaction |
| 4) |
difficulty
and workload |
| 5) |
assessment
practices. |
|
| |
| Another
common element in evaluation forms is the inclusion of “demographic”
questions, e.g., GPA, expected grade in course, major/minor, etc. |
|
| PURPOSES
& INTERPRETATION OF TEACHING EVALUATIONS |
| Information
gathered from teaching evaluation is used mainly for the two following purposes: |
| |
| Teaching
Improvement (Formative) |
| For
this purpose, student evaluation data alone are insufficient. Student evaluations
need to be complemented by consultation and by data from other sources.
For example, professors should discuss SET results with colleagues or the
Teaching Centre; contextualize results in terms of their own goals; compare
end-of-term results to mid-term evaluations; and look at results from various
courses over time. This kind of consultation and contextualization increases
the likelihood that professors will benefit from formative evaluations and
use them to modify their teaching. |
| |
| Personnel
Decisions (Summative) |
| Evidence-based
evaluation of teaching is essential to inform hiring, annual reviews, promotion
and tenure, and other career landmarks. Personnel decisions should be based
on sound, corroborative evidence from a variety of sources: for example,
student evaluation data (both numerically scored and narrative); peer reviews;
and materials provided by the professor, such as syllabi, sample course
handouts, and reflective statements about teaching strategies and outcomes.
|
| |
| Evaluations
are also used for two other purposes: |
| |
| Student
Course Selection |
| The
literature shows that where evaluation data are disseminated to students,
fewer than one third use SET to choose courses based on workload. Many more
students (about two thirds) use SET data to choose courses based on the
instructor. Since multi-sectioned courses are not the norm at Mount Allison,
this use is less frequent at the university. |
| |
| Teaching
Awards |
| Nomination
dossiers for both internal and external teaching awards commonly contain
student evaluation of teaching data. |
|
| EVALUATION,
ADMINISTRATION & DISSEMINATION |
| Just
as the validity and utility of teaching evaluation forms depend to a large
extent on choosing questions that are aligned with and reflect the goals
and practices of the institution, their integrity depends on how they are
administered. |
| |
| Most
universities have a teaching evaluation policy that addresses administrative
details, such as the frequency and timing of evaluations and the dissemination
of their results. For example, most institutions evaluate every course each
year. The process is centrally coordinated—printing and distribution
of forms, computer analysis, calculation of means, typing written comments,
and so on. |
| |
| Similarly,
there are certain universal aspects about the administration of in-class
evaluation forms: they are given at the end of the course; they are anonymous;
instructors are ‘removed’ from the process to the extent that
they are not present in the room when the evaluations are being conducted;
results are not disclosed until final grades have been submitted. |
|
| CONCLUSION |
| The
above discussion points to the need for Mount Allison University to develop
an evidence-based teaching evaluation policy that both respects and reflects
institutional values and the importance of high quality teaching at the
university. To ensure widespread adoption, support, and benefit, other corollary
points must be considered in the development and implementation of the policy.
These include |
| 1) |
That while student feedback is an important component of teaching
evaluation, both for formative and summative purposes, it is but one
source of evidence. |
| 2) |
That
personnel decisions should be based on sound evidence from a number
of sources. Professors are strongly encouraged to provide a wide variety
of evidence of teaching achievement, complementing and contextualizing
SET data with peer review and self-assessment, preferably within a
teaching portfolio. Similarly, those who review evaluation data for
summative purposes are expected to do so carefully, with attention
to the professor’s performance over time and in variety of teaching
contexts. Reviewers are encouraged to look for the alignment of evidence
over several years from a number of sources. |
| 3) |
That
the form used to collect student evaluations must be aligned with
the university’s priorities and values. Therefore, to allow
for consistency, the form should comprise some standard questions,
including global or “overall” questions; to reflect disciplinary
differences or individual approaches to teaching, it should also permit
the individual professor to add customized questions. |
| 4) |
That,
since the maintenance of teaching quality is the central aim of teaching
evaluation, |
| |
| a) |
the evaluation data should be presented in a way that makes
sense to the professor so that he or she can act upon it appropriately;
and |
| b) |
there should be clear guidelines regarding who has access to
the evaluation data and how that access is granted; for example,
it is common at other universities for department chairs, deans,
and promotion and tenure committees to receive the SET results
(in some cases, they see only the core/standard questions or
the overall questions). |
|
| 5) |
That
teaching evaluations should not be used to compare individual professors
to each other because the data cannot reflect the differing contexts
of their teaching; the best comparison is to look at the same professor’s
performance over a number of years/terms and teaching a variety of
courses (at least five courses, even more if the class size is small).
|
| 6) |
That
a robust teaching evaluation system relies on full, thoughtful, and
honest participation from students. |
| 7) |
That,
given the importance of teaching evaluation both in maintaining the
quality of instruction at the institution and in selecting and promoting
individual professors, a Policy on Teaching Evaluation will be created
and guidelines for its implementation widely disseminated to students,
professors, and administrators. |
|
|
|
| REFERENCES |
Gravestock, P. & Gregor-Greenleaf, E. (2008). Student Course Evaluations:
Research, Models and Trends. Toronto: Higher Education Quality Council
of Ontario.
Herteis,
E. (Chair). (2007). Report on Student Evaluation of Teaching. Mount Allison
University: Senate Committee on Teaching and Learning.
MacMillan, C. (Chair). (2002). Final Report. Mount Allison University:
Teaching Evaluation Committee.
|
| |
|
| ©
Mount Allison University |
| |
|
January 27, 2010
|