Monday, December 31, 2007

First Question: What if students learn better in a course they don't like?

What if the following are BOTH true for a particular course at the same time
A. Some kind of learning outcomes assessment conducted on all students who just completed the course clearly demonstrates superior student learning when compared with similar assessments of similar courses
AND
B. Some kind of student ratings collected for the same course clearly demonstrate that the students are more dissatisfied with this course than with similar courses.

Or vice versa.

Thursday, December 20, 2007

What kinds of evidence counts?

How does anyone know, really know, that a particular course is good or bad? Better or worse than a similar course? That a particular teacher is good or bad? That a particular course taught by a particular teacher at a particular time for a particular student is good or bad? For a particular group of students?

What kinds of evidence would be useful to resolve one of these apparent conflicts?
What kinds of evidence would be compelling?

Monday, December 10, 2007

Bus Test, Embarrassment Test, Pride Test, Student Engagement Test, ...

Bus Test: If a teacher gets hit by a bus, can another smart and well-prepared teacher replace the first one and achieve similar learning outcomes with the students?

Embarrassment Test: Do colleagues and/or students prefer to hide a course from the public because they are embarrassed by the teacher or some aspect of that course?

Pride Test: Do colleagues and/or students want to tell the world about a course because they are proud of the teacher or some aspect of that course?

Student Engagement Test: Do the course design and the actions of the teacher facilitate and support student engagement in significant learning?

Learning Outcome Tests: ......

Other Tests: ....????

Saturday, December 01, 2007

Modest Proposal: "7 Principles for Fair and Effective Student Evaluation of Teaching" - Tom Angelo 4-28-2006

"... proposal... that the key leaders in the field ... issue a short, simply worded, research-based manifesto on student evaluation of teaching -- supplemented by a few very clear suggested guidelines. ...'7 Principles for Fair and Effective Student Evaluation of Teaching', "
From: Tom Angelo
[Also see "9 Principles of Good Practice for Assessing Student Learning" and "7 Principles of Good Practice for Undergraduate Education"

Date: Apr 28 2006 - 8:16pm

I agree, in general, with all the statements made by my expert colleagues. And I particularly agree that course design and teaching are means to an end -- efficient, effective, deep and meaningful learning. For good or ill, I'm on record repeatedly restating these articles of faith. I also agree completely with Mike that infrequent exceptions don't (necessarily) disprove general findings in social science.

I will note three other obvious points. Sometimes teachers who significantly change their teaching approaches subsequently get poorer ratings for the same reasons that tennis or golf players do more poorly, for a time, after being coached to change their swings. The second reason, which has already been mentioned, is that students -- even the most fair-minded ones -- are making comparisons within a context. If I'm the only teacher in their program or institution using challenging and unfamiliar methods, students may understandably think I'm being unfair and perhaps am not very professional.

Despite the assessment "movement" and the "learning paradigm shift", student evaluations alone are all that most colleges and universities use (in any real way) and all, I have come to believe, they ever will use in the foreseeable future to evaluate the quality and effectiveness of teaching and learning. My experience is that, despite the all the good research, writing and consulting of Mike and his colleagues, there is as much mis-, mal- and non-feasance around evaluation of teaching and courses as ever -- and that the systems in many institutions probably inhibit change and improvement more than they support them.

Thus, I'd like to move on from defending student evaluations of teaching, in general, to exploring much more specific, politically palatable and practically applicable ways to work around or manage the biases, manias, paranoias (justified and unjustified) and various other self-serving bogeys and beliefs that hinder progress.

I say we forget the flat Earthers and move on. They have never been convinced by research or reasoned argument. More of the same will be a similar waste of time.

I have a proposal, perhaps a bit naive, that the key leaders in the field follow the successful precedent of Gamson and Chickering and issue a short, simply worded, research-based manifesto on student evaluation of teaching -- supplemented by a few very clear suggested guidelines. If the right associations could be convinced to sponsor and disseminate such a "7 Principles for Fair and Effective Student Evaluation of Teaching", then it might actually have an impact on some of the benighted committees and individuals struggling endlessly with this morass.

One page containing seven or so crisply worded principles. Two or three more pages of suggestions, including the seven or so questions that are most valid and reliable and seven or so guidelines for good administration, followed by seven or so key references.

Thanks for your good work and consideration!

Tom