Secret confession time

I actually enjoy writing exams.

To me, writing exams is like putting together a puzzle. You have all the pieces there: all of the topics that you want to cover on the exam (the ones you've covered in class since the last exam, along with the ones you want to reinforce/retest from previous exams. In CS, it's hard to have an exam that is not at least partially cumulative). And you have the ways in which the pieces can fit together: the concepts within the topics that you want to test. Putting these together in such a way that (a) the exam assesses the students' learning in an appropriate way and (b) the exam is do-able within the allotted time is, to me, an interesting challenge.

My strategy for writing exams has evolved over the years, but I've finally settled on something that seems to work well for me. First, I put off writing the exam until the day before. This is probably not the most effective working strategy, but I find that I just cannot write an exam until the day before. However, I probably start thinking about the exam about a week before I start writing it: what have we covered? what would be a good way to test that particular concept? would a problem like X be too weird/too esoteric for this crop of students? So by the time I sit down to write it, I already have some good ideas for questions.

Second, I sit down and list all of the topics that we've covered since the last exam, along with any other topics I want to include on this exam. For each topic, I list two categories: "base knowledge" and "master knowledge". The "base knowledge" category lists the basic things I want students to know about a topic, the things that I think demonstrate the most basic understanding of a topic. The "master knowledge" category lists things that demonstrate what I think constitute "mastery" of a topic: things that show that students understand the topic and can reason about it in unfamiliar contexts. (Or, to put it bluntly, the level of understanding needed to earn an A for that particular topic.)

Third, I take each topic and the knowledge lists, along with the scattered ideas for questions that have been floating around in my head all week, and start to structure each question. The knowledge lists help shape each question, and also help me tailor the exam to the exam context. For instance, an in-class exam will focus more on base knowledge than on mastery (maybe 80%/20%, sometimes 70%/30% depending on the level of the class), while on a take-home exam the mix may be closer to 60%/40%. I also think about the context when structuring the exam: on a take-home, for instance, I'm more comfortable writing a problem that is instructive as well as assessive (is that a word?)---a question in which the students learn something even as they are demonstrating their understanding of a topic.

Fourth, I treat writing an exam like writing a paper: I write a first draft, print it out, then leave it alone for a couple of hours before going back to edit it. This helps me disconnect from the exam a bit and go back to it with a somewhat fresh perspective. Often while editing, I will actually do the problem, to make sure that my assessment of the difficulty level and time required matches up with the actual problem I've written. I will sometimes bug my colleagues to help me with the wording on questions, too.

This process is a bit time-consuming, but as a result I am rarely unhappy with an exam that I've written. The process forces me to be thoughtful about how I'm assessing the learning in my classes. It forces me to really think about *what* I want my students to know and *how* I want them to demonstrate that to me. And I think ultimately, it's made me a more thoughtful teacher as well: while planning out a new unit, I'll often think "what do I want my students to demonstrate to me at the end of this?" And I hope that ultimately, that makes me a more effective teacher as well.