Standards Based Grading: Designing Questions

Before my current job, I was a graduate student at the University at Buffalo.  During the school year, I was employed as a teaching assistant, but during the summers I taught my own classes.  I almost always made my own tests, so I have had a decent amount of experience designing my own questions.  After the seventh iteration of a question involving the electric field produced by point charges, I began to wonder how else  I could  possibly ask the question.  So in the interest of mining out unused design space, I began to concoct bizarre and/or complicated scenarios.  Eventually, I reached a point where I asked myself, “Is this question really extracting the information I need about the student’s understanding, or am I just trying to prove how clever of a question I can make?”

Let’s take the electric field produced by point charges as an example.  An easy first question to design is to pick two locations, place a charge at each location, then ask for the electric field at a third location.  It is pretty clear what this question is assessing, and this is usually where I start.  When it comes time to design a second question, I often look for a different take.  How about a question with two point charges that asks for the location at which the net electric field is zero?  Students usually do much better on the first kind of question than the second kind even though the fundamental ideas are the same.  Why?  Usually, the problem is either with the algebra, with discarding the unphysical solution, or with the fact that in order to set up the problem you need to arbitrarily pick a location and know to set that as your variable.  In all cases, the issue is not one of having an incomplete understanding of electric field and the superposition principle.  I am totally fine with this kind of question, but I also have to be honest with myself that this is not entirely a question that tests the students’ understanding of electric fields.

Making sure that your question is assessing what you want it to assess is even more important when designing questions for SBG.  I want to know exactly what I am assessing, which is why I have decided that I am not going to worry about being creative and asking something that looks entirely different from before.  Every single iteration of a question about the electric field produced by multiple point charges is going to be same: two point charges at given locations, and the student calculates the value of the electric field at a third location.  All I need to do is change the values for each version.

This does not mean that I am going to avoid making questions that test things like math skills or the ability to translate a word problem; I am just going to relegate those things primarily to the higher level “A” objectives (whereas the example question I was referring to would be a “B” objective).  This way, a student can’t achieve the highest possible course grade unless he or she can demonstrate the ability to use the learning objectives in a bigger context.


Standards Based Grading: Planning Assessments

When I first started reading about Standards Based Grading and considered trying it myself, I realized that the biggest obstacle to implementing it would be my time constraints.  We work on 10-week trimesters here at Union College, and my total class time with students is 50 hours including labs.  SBG typically requires lots and lots of assessments, which are usually done during class time.  We would be covering about 5-7 objectives per week, which means that each assessment would need roughly that many questions.  Given the difficulty of some of the objectives, this would just take too much time out of class.  The other option, then, is to have out-of-class assessments.  The problem with this, of course, is that students could seek outside help. Thus I would not have an accurate picture of what they really understand.  I was seriously beginning to doubt that I could pull this off.

Not long ago, I had a conversation over email with Kelly O’Shea, and she pointed out something that hadn’t occurred to me at all.  In SBG, you typically only count the student’s most recent mark on a particular objective.  If the student knows that they are going to be assessed on a particular objective again at some point in the future, then the student has a rather strong incentive not to cheat.  Contrast this to a typical points-based system of grading where grades get “locked in”.  A student who is having difficulty or is short on time may rationalize cheating on that assignment because they will make up for it later when they study for the exam.  We all know how that usually goes.

So here is the plan that I am going with: at the end of each chapter, I will give the students a take-home test with at least one question from each of the new objectives as well as some questions covering old objectives.  I have a little speech planned for the first time so that students understand this new way of doing things.

“This is your first take-home assessment, and I would like you to finish it by our next class.  The purpose of this assessment is so that you can receive feedback about your current level of understanding of the material, and for this reason you should not use any outside resources.  Let’s suppose, though, that you are extremely busy tonight and don’t have time to complete this, so you find a friend, copy his or her answers, and receive high marks on all of the objectives.  Remember, though, that I only count your latest score on any objective.  Given that you will be tested again, there is absolutely no reason for you to cheat now.  In fact, doing so will only rob you of valuable feedback from me.  So if you find yourself in the situation that I described above, take an extra day.  It’s fine.”

I also plan to have in-class assessments on weeks 4 and 8 as well as a final exam.  This is partly because that is the standard practice in our department.  In addition, I am hoping that this will get the students to take the out-of-class assessments more seriously.

Standards Based Grading

The school year is starting in just under two weeks, and I am preparing to make a major change in the way I grade.  I have always used a typical point system for grading.  However, I have never been completely satisfied with this system because:

  1. students often don’t get the grade that I feel they deserve
  2. students fixate on getting a certain grade, which can lead to cheating
  3. students have very little incentive to try to fix their mistakes since grades are locked in
  4. even if a student demonstrates to me later in the term that they understand something they previously received a poor grade on, I can’t give them any credit for it
  5. determining cutoffs for letter grades at the end of the term feels very arbitrary

So this Fall I am going to try Standards Based Grading in my calculus-based introductory Electricity & Magnetism class.  The basic idea of Standards Based Grading (SBG) is that your gradebook is organized by standards, or learning objectives (i.e. The student can determine the range of a projectile), rather than by assessment (i.e. Quiz 1).  My choice to use SBG was inspired by Kelly O’Shea, Frank Noschese, Shawn Cornally, Andy Rundquist, Jason Buell, Dan Meyer, and others that I’m sure I’m forgetting at the moment.   These blogs combined contain a huge archive of excellent information on SBG, so rather than rehash all of that, I will just walk through my process and the challenges I encountered.

The first thing that I did was come up with a list of standards (objectives), which turned out to be easy because the course’s textbook (Matter & Interactions) has a list of key ideas that translates very well to learning objectives (about 5-7 per chapter).  When I tried to come up with questions, though, I encountered a problem.  Let’s say I want to see if students can use the electric field concept in combination with Newton’s 2nd Law and kinematics to describe the motion of a charged particle.  How does that work when my learning objectives only include material from electricity and magnetism?  My idea, again inspired by many other blogs, was to create different levels of objectives.

C objectives: These are the simplest ideas, often involving identification, simple sketches, or basic arithmetic.

B objectives: These are slightly more involved ideas, often requiring calculus, vector algebra, or a process that involves multiple steps.

A objectives: These are not separate ideas but rather require the students to identify and use multiple objectives together.  This is where I can spiral back to mechanics, for example.

A student’s letter grade is calculated by doing 2*(normalized C score)+(normalized B score)+(normalized A score) and then converting that to a letter grade using the typical 4.0 scale.

The next challenge is to figure out how to perform assessments and design questions.