You know those “please give feedback on the unit” surveys we ask you to fill in at the end of the unit? Please fill them in.

Please also pretend my name is Karl Greenhill, presume that I am cis-gendered.

The data is about to be misused. If there are low numbers of responses, it is about to be used very badly indeed.

The surveys were originally designed so that we could get feedback on the unit to improve what needs improving and to keep doing what we are doing well. Great, if only that was all they were actually used for.

In many universities these surveys are also used:

1. In annual staff appraisals for academic staff who are unit coordinators.
   • I have been through several where the interviewer was not really interested in any of the qualitative information, or most of the numeric totals – just in one figure. The final “Overall, I am satisfied with this unit” figure. That was then compared with the total for other units in the faculty and in the university.
   • Just think about that for a moment. If any one of these scenarios exist, the figure is likely to be low, but there to be nothing in the survey that indicates why.. and then it will be used to determine my personal job performance:
     • The university schedules the lecture at 6pm on a Friday
     • There is an external marker whose mother dies and marks are returned late
     • The unit covers rather difficult material with rigour that is demanding but very pedagogically effective
     • The people responsible for ensuring the textbook is listed on the website do not do this
     • One student has a run-in with the tutor and then shares their displeasure on a private Facebook group for students in the unit
   • We can ALSO ask students to fill in a “teaching appraisal” survey. This is optional, but I always ask for this. I want to know how I can improve. Several times I offered to bring the results to my staff appraisal, only to be told that it was not necessary.

1.   As a measure of “performance” of units, regardless of how many students respond to the survey
   • I sat through a meeting this morning where course coordinators were presented with sheets of summaries of student surveys for various units. Those with very high scores for a heading had a very green cell, those with lower, an amber, and very low, a red.
   • Some units, full of dark green cells, were singled out as performing well, those coloured in the red zone, as performing poorly. Generally fewer than 20% of students had responded for each unit. This often translated to 3-4 students.
   • Troublingly we were told that “the Government” is linking funding to performance, and that these measures were being taken seriously and we needed to ensure that any units with orange/red were “fixed”.

Personally, I have big problems with these uses and the impact on my employment.

Our job as academics is to teach students about valid measurement and research. I have a big problem with this demonstrably not being observed internally.

Just as worrying are studies that show that if the students *think* I am a woman (NOT if I am actually a woman) then I am likely to have lower scores than if they *think* I am a man. For example, in one study where assessments were returned at exactly the same time, students rated tutors who they *thought* were female 16% lower on “promptness” than if they thought the tutor was male. MacNeil(2015) found that:

“the male identity received significantly higher scores on professionalism, promptness, fairness, respectfulness, enthusiasm, giving praise and the student ratings index. … Students in the two groups that perceived their assistant instructor to be male rated their instructor significantly higher than did the students in the two groups that perceived their assistant instructor to be female.”

I remember reading somewhere that it could be to do with expectations of women actually being more prompt, efficient and organised, therefore behaviour that students will forgive and rate higher for a male tutor (because they expect a lower standard), they penalise and rate lower for a female instructor.

I don’t really care what the reason is. Just please fill in the surveys, and presume I am actually Karl.

For more, see:

MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching. Innovative Higher Education, 40(4), 291–303. https://doi.org/10.1007/s10755-014-9313-4

Boring, A., Ottoboni, K., & Stark, P. (2016). Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. ScienceOpen Research. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1