Before my current job, I was a graduate student at the University at Buffalo. During the school year, I was employed as a teaching assistant, but during the summers I taught my own classes. I almost always made my own tests, so I have had a decent amount of experience designing my own questions. After the seventh iteration of a question involving the electric field produced by point charges, I began to wonder how else I could possibly ask the question. So in the interest of mining out unused design space, I began to concoct bizarre and/or complicated scenarios. Eventually, I reached a point where I asked myself, “Is this question really extracting the information I need about the student’s understanding, or am I just trying to prove how clever of a question I can make?”
Let’s take the electric field produced by point charges as an example. An easy first question to design is to pick two locations, place a charge at each location, then ask for the electric field at a third location. It is pretty clear what this question is assessing, and this is usually where I start. When it comes time to design a second question, I often look for a different take. How about a question with two point charges that asks for the location at which the net electric field is zero? Students usually do much better on the first kind of question than the second kind even though the fundamental ideas are the same. Why? Usually, the problem is either with the algebra, with discarding the unphysical solution, or with the fact that in order to set up the problem you need to arbitrarily pick a location and know to set that as your variable. In all cases, the issue is not one of having an incomplete understanding of electric field and the superposition principle. I am totally fine with this kind of question, but I also have to be honest with myself that this is not entirely a question that tests the students’ understanding of electric fields.
Making sure that your question is assessing what you want it to assess is even more important when designing questions for SBG. I want to know exactly what I am assessing, which is why I have decided that I am not going to worry about being creative and asking something that looks entirely different from before. Every single iteration of a question about the electric field produced by multiple point charges is going to be same: two point charges at given locations, and the student calculates the value of the electric field at a third location. All I need to do is change the values for each version.
This does not mean that I am going to avoid making questions that test things like math skills or the ability to translate a word problem; I am just going to relegate those things primarily to the higher level “A” objectives (whereas the example question I was referring to would be a “B” objective). This way, a student can’t achieve the highest possible course grade unless he or she can demonstrate the ability to use the learning objectives in a bigger context.