Job Hunt

Being a Visiting Assistant Professor means that I am perpetually searching for new jobs, and I am discovering that applying to teaching jobs is a tricky process.  I decided some time ago that my strategy for setting myself apart from the crowd would be to emphasize my use of non-traditional teaching methods and my desire to constantly improve the way that I teach.  By all accounts, this strategy has been very successful in getting me phone interviews for visiting positions.  If the phone interview goes well, I end up having an exchange with the search committee that usually goes something like this:

Search Committee: Congratulations!  We were impressed with your phone interview, and we would like to invite you to visit our campus and give a talk in front of our faculty and students that demonstrates your teaching style.  Please provide us with a name for your talk so that we can advertise it.

Me: Excellent!  The name of my talk is Implementing Non-Traditional Teaching Methods in the Physics Classroom.

Search Committee:  That talk is not really appropriate for what we are looking for.  We are looking for a talk that demonstrates your ability to make a complex topic understandable to undergraduates.

Me: Okay, I understand.  How about Black Holes and their Formation?

Search Committee:  Capital! (Okay, nobody ever said that, but how cool would it be if they did?)

Did you catch the subtle shift there?  First, they ask for a talk that demonstrates how I teach, then they ask for a talk that demonstrates my ability to make a complex topic understandable to undergraduates.  In my mind, those are not the same thing.  I’m not the one offering the job, though, so I go along with it.

So I go and give my black hole talk, and it goes gangbusters.  I get to show my enthusiasm for the subject and my personality, and the audience is reeling from all the crazy things about black holes that they just heard.

On two occasions, the search committee accepted my first talk, so I talked to them about some of the non-traditional teaching methods that other people/institutions have used and the evidence that shows the benefits of non-traditional teaching.  Both times I was ridiculed by people in the audience.  They attacked the validity of the supporting evidence and called the methods childish.  That’s not to say that there weren’t people in the audience that supported me, but being ridiculed like that in front of a group of people tends to stick in your mind.

Many teachers seem to get offended when you talk about different ways of teaching.  My intent is to make suggestions and inspire people to try something different, but some people take it as criticism.  I’m probably partly to blame; walking the line between suggestion and criticism is difficult, and my excitement invariably causes me to stumble.

It’s not all bad news, though.  Much of my success in getting people to try new things has been the result of people walking by my classroom and briefly watching.  For example, some people were intrigued by what I was doing with whiteboards, and now there are whiteboards in every physics classroom in the building.  My experiment with Standards-Based Grading has piqued the interest of some of my colleagues as well.

So I guess the lesson I learned is that I need to stop telling people what they should be doing.  I should just do what I do, and if it is successful, people will notice.

Review of My First Year of Standards-Based Grading

I just finished my first full year of using Standards Based Grading, and the results were mixed.  I still can’t imagine going back to a completely traditional grading system, but the details of the implementation of SBG are difficult to get right.  I already talked about what I liked about my first term of SBG, so here I am going to focus on what went wrong so that I can prepare a solution.

It seems to me that the biggest benefit of SBG is that students are able to incorporate feedback and use it to improve.  To take advantage of this, though, you need to assess each skill multiple times.  This presented the biggest challenge to me because I had a lot of content to teach and only 10 weeks (with 50 hours of class time) to do it.  My solution to this was twofold: reduce the number of standards, and offload assessments to homework rather than in-class quizzes.  The difficulty with the first solution was that it required a good amount of prescience.  There were times when I designed a really good question only to find that there was no way of grading it since it wasn’t on the list of standards. The second solution didn’t work as well as I had hoped, either.  Scores on tests were always much worse than scores on homework, which means that the homework was not as helpful to the students as I wanted it to be.

Another major issue was the amount of time that was spent recording grades.  In each trimester, I used a different system of keeping track of grades.  Each new bookkeeping system was created to solve problems of the previous iteration, but each presented its own new problems.  The end result of all of this extra time spent bookkeeping was that we ended up shortening the assignments, which made it harder to revisit old objectives, which eliminated the main benefit of using SBG in the first place.

One more thing that is difficult to do with SBG is to have problems that require synthesizing skills.  The student may have no problem using Skill X and Skill Y individually, but combining them together is a different issue entirely.  Do I give two scores for that problem, one for Skill X and one for Skill Y?  Do I create a separate standard that says, “I can synthesize Skill X and Skill Y”?  That seems reasonable at first glance, but it quickly gets out of hand, as there are many possible combinations of skills.  Then we are back to the problem of having too many standards to address, which therefore makes it harder to revisit standards, which eliminates the main benefit of using SBG in the first place.

Proposed Solution

It seems to me that the best way of solving all of these problems is to have one standard for each model rather than having one standard for each skill required to implement that model correctly.  For example, the ability to properly implement the Forces Model requires that you be able to identify forces, draw a force diagram, break down forces into components, write down a force equation, and solve the equation for unknown variables.  Rather than score students on those individual abilities, I want to try assigning scores based on their ability to apply all of them together.  I was glad to see recently that someone else came to the same basic conclusion and even came up with a very nice list of big-picture standards.  One thing I like about the setup of that list is that each model is accompanied by a list of the individual skills needed to successfully implement that model.

The one-standard-per-model system also means that synthesis problems will be the norm rather than a challenge that is given from time to time.  While it is true that students often have trouble with synthesis, this is where the beauty of SBG shines through; they will improve with repeated assessment and feedback.

Pre-Mortem

When coming up with plan, I find it useful to imagine myself in the future recounting all the ways in which the plan failed (If I remember correctly, I first heard of this idea from Thinking, Fast and Slow by Daniel Kahneman).  So here goes:

  • focusing only on the models meant that students didn’t get enough practice on the more basic skills

  • synthesis is difficult, so grades were low

  • the fact that there were so few “grade columns” meant that a poor score on one particular standard dragged that student’s grade down too much

Are there any other reasons you can think of that my plan failed?  How would you solve these problems?

As a teacher, I have always struggled to strike a balance between “teacher knows best” and “you are old enough to take care of yourselves”.  For example, do I require attendance or not?  Do I require homework or not?  I have experimented with varying degrees of each, but I have tended to favor the hands-off approach because I think that students should have some measure of control over their learning experience.  On the other hand, I have also seen the negative effects of the hands-off approach; it usually ends up that the students who most need to attend class or do the homework are the ones who choose not to do it.

I recently read a book called Thinking, Fast and Slow by Nobel-winning psychologist Daniel Kahneman.  One part that I found particularly interesting was a comparison of organ donation rates in two particular demographically similar, neighboring countries.  One country had a very high donation rate, while the other had a very low donation rate.  The difference seemed to stem from the fact that organ donation was “opt-in” in one country and “opt-out” in the other country.  In both cases, people had a freedom of choice, but the default option was a strong determining factor for that choice.  Richard Thaler and Cass Sunstein of the University of Chicago have been studying this effect to see how organizations can help people make better decisions (they even wrote a book about it called Nudge: Improving Decisions About Health, Wealth, and Happiness), and they have given the name Libertarian Paternalism to the practice of taking advantage of this effect.

I’m wondering whether this could be leveraged in the classroom.  For example, telling students that attendance is not mandatory is likely to yield different results from telling students that they should email you if they aren’t going to show up to class.  Or suppose I want students to take more advantage of office hours.  Would I get better results if I scheduled specific times for each student and gave them all the option of opting-out by sending me an email?

What other sorts of practices could benefit from this approach?  Does it sound like a bad idea?  I’m interested to hear anyone’s thoughts on this.

 

Thoughts on my First Experience with Standards-Based Grading

I just completed my first term using Standards-Based Grading, and I learned a lot from it.

What I liked about it:

  • The classroom atmosphere was no longer filled with stress and fear.  I think this was because assessments weren’t “final”.  No score was ever locked in, which meant that students paid more attention to the feedback than to the score.
  • Academic dishonesty was virtually non-existent.  This was probably due to the fact that there was no incentive to cheat on homework because only the student’s last grade on a particular standard would count.  When I used online homework in the past, the average homework score was typically at least 95% and there was almost no correlation between homework grades and exam grades.  This past term, I was fairly confident that there was no cheating because perfect scores were extremely rare.
  • Re-assessment seemed to work.  I kept track of the five lowest-scoring standards each week, and those lowest scores were almost always from what we had covered most recently.  Scores slowly crept up over time, and there were very few times when scores went down.
  • Students seemed more concerned with understanding the process than with getting the right answer.


Mistakes that I made:

  • I underestimated the amount of procrastination I would see from students.  Deadlines seemed to go against the spirit of Standards-Based Grading, so I didn’t really give any.  Interestingly, this was not much of a problem for the homework, but it was a major problem with the labs.  I assumed that most students would do a rational cost-benefit analysis and realize that they could maximize their grade by handing lab reports in as early as possible.  Instead, most of the lab reports were handed in for the first time within an hour of the final deadline.
  • My policy that re-assessments could be done at any time made for an unpleasant and chaotic final week of office hours.  There were a few students who were diligent about re-assessing during the term, but most students waited until the end.
  • I made my list of standards before preparing questions for some of them.  It turned out that some of the standards I had on the list weren’t very conducive to assessing multiple times.  The biggest culprit here was the standard “I know the ideal internal resistances of ammeters and voltmeters.”  Even the textbook only had one question on that topic.
  • My standards didn’t address the fact that not all ways of representing an answer have the same difficulty.  The ability to calculate something and the ability to answer a question using words, graphs, or proportional reasoning are very different things and should have been treated separately in my list of standards.  For example, scores on the topic of the Voltage Loop Rule fluctuated wildly depending on whether or not I gave actual numbers for the battery voltage and resistances.
  • The “A”, “B”, and “C” categories for my standards were largely irrelevant.  I felt like the different categories were clever because each category represented a different level of difficulty as well as the letter grade someone would get for mastery of all of those objectives.  In the end, though, the net effect of each standard on the overall grade was almost exactly the same regardless of whether it was a “C”, “B”, or “A” standard.


Overall, I can’t imagine going back to a points-based grading system.  That being said, there are some things that I need to change.  Here are changes that I plan on implementing next time:

  1. I need to set deadlines.  I put too much control in the hands of my students, and in the end it hurt them.  The deadlines will also apply to Student-Initiated Assessments.  If they want to fix their score on a standard, they will have a limited time to do so.  This way, re-assessments won’t all happen in the last week of the course.
  2. I will create a set of questions that I want to ask, then structure the list of standards around those questions.  This ensures that I have enough questions for a particular standard and that I am actually testing things that I care about.
  3. I will separate quantitative and qualitative skills on my list of standards.  This should reduce the amount of grade fluctuation.


I am glad I was able to figure out all the kinks on my own because next week I start teaching with two other professors.  I somehow managed to convince them to try SBG, and they have been great so far in accommodating me and the new system.  I’m looking forward to hearing their feedback in the coming weeks.

Why is Mechanics Taught Separately from Electricity & Magnetism?

Almost all undergraduate physics curricula start with one term studying mechanics.  In the first term, students learn about motion and the forces that cause motion.  Then they learn a (different) way of explaining motion in terms of energy conservation.  The second physics term is spent studying electricity and magnetism.  Students learn about the motion of charged particles and the forces that cause that motion.  Then they learn a (different) way of explaining motion of charges in terms of energy conservation.  There seems to be a nice sort of symmetry between the two courses, but I encounter a problem each year, namely, my Electricity and Magnetism students have a hard time applying the skills they learned in Mechanics to Electricity and Magnetism.  I’m sure that one culprit is time; the students have probably forgotten what they had learned before.  But I’m wondering if part of the problem is the lack of a unified approach to representing physical situations in their first physics course.  The students spend a little time learning how to use Newton’s Laws, then a little time with Conservation of Momentum (which is sometimes presented as a separate topic when in fact it easily combines with Newton’s Laws), then a little time learning about Conservation of Energy.  From a student’s perspective, I wouldn’t be surprised if his or her Mechanics class felt like this:

  1. There are quantities called position, velocity, acceleration, and time — sometimes you have to use kinematic equations to answer questions
  2. There are these things called forces  — sometimes you have to use force equations to answer questions
  3. There is this thing called momentum — sometimes you have to use momentum equations to answer questions
  4. There is this thing called energy — sometimes you have to use energy equations to answer questions

The students don’t have to think too much about which physical principles (i.e. Newton’s Laws, Conservation of Momentum, or Conservation of Energy) to apply in order to answer a question because the questions that are asked are pretty much always aligned with the current chapter.  It should be no surprise, then, when students have difficulty knowing which physical principle to apply to a generic situation.  In fact, when presented with a generic situation, the solutions that I see from students are often “Equation Soup”.

Equation Soup

noun

1.  A collection of partially-worked formulas from the Equation Sheet that have letters in them that correspond to the quantities that were given in the problem statement


The problem is perpetuated in the Electricity and Magnetism course because the curriculum goes through exactly the same process, except now it gets applied to the property of charge instead of the property of mass.

What if, instead, the courses were organized as follows:

First Course – Interactions as Forces

  1. Objects and systems have properties such as mass and charge.
  2. Interactions between objects can be described by forces.
    1. the gravitational force is an interaction between two objects with mass
    2. the electric force is an interaction between two objects with charge
    3. the magnetic force is an interaction between two objects with charge that are moving
  3. Knowledge of the forces acting on a system can be used to describe the motion of the system.

Second Course – Interactions as Energy

  1. Objects and systems have properties such as mass and charge.
  2. Interactions between objects can be described using energy.
    1. gravitational potential energy is the energy of interaction between two objects with mass
    2. electric potential energy is the energy of interaction between two objects with charge
  3. Knowledge of the energy transfers within a system can be used to describe the motion of the system.


In this course curriculum, the different subtopics in each course are the types of interactions (i.e. gravitational, electric, or magnetic), not how you choose to represent those interactions (i.e. forces or energy).  For example, in the Interactions as Forces course, you learn that the Earth interacts with all other massive objects (i.e. moon, other planets, the sun), you represent all of those interactions with an arrow called a “force”, then you apply the rules of forces to determine the resulting motion of the Earth.  Next you learn that a charged particle interacts with all other charged particles, you represent all of those interactions with an arrow called a “force”, then you apply the rules of forces to determine the resulting motion of the charged particle.  The same goes for the magnetic force and every situation that is a combination of the gravitational force, the electric force, and the magnetic force.  In the Interactions as Energy course, you take exactly the same situations as before, except now you represent those interactions with a scalar quantity called “potential energy”, then you apply the rules of energy transfer to determine the resulting motion of the object.  The process of how to represent and analyze a situation is always going to be the same for every type of interaction.  In terms of teaching, the advantage of this is that the students will get practice in applying the same process over and over again, and they should eventually start to see that any interaction can be represented either with forces or energy.  The choice of which representation to use comes down to ease and efficiency.  The other advantage here is that rather than promoting the idea that there is the “mechanical world” and the “electromagnetic world”, there is only one world, and we choose to represent it in whatever way is most convenient.

I’m sure this has been considered before; in fact, I remember reading somewhere about some schools that actually do separate their courses this way.  Is there a compelling reason that I am missing for why the vast majority of curricula are divided into Mechanics and E&M instead?

Why I Decided to Change the Way I Teach

When I was hired as a teaching assistant in graduate school, I taught in the same way in which I had been taught…lecture.  I had never received any formal education in how to teach.  In fact, the extent of my teacher training was someone telling me, “You’ll be fine”.  (That’s not completely fair, actually, because there was a two-day workshop for teaching assistants that I missed for reasons I can’t remember.)  I was in charge of four recitation sections in which I was supposed to do practice problems from the material that was covered in lecture.  When the students had difficulty with some piece of material from lecture, I tried to explain it to them in my own way. The students seemed to think I was doing a pretty good job.  I had a problem, though: while  the students understood me as I explained something, they often couldn’t replicate what I did when they were on their own.  I couldn’t quite figure out what else I could do.  After all, I explained everything the best I could.  Wasn’t the rest of it in the students’ hands?

In retrospect, I shouldn’t have been too surprised at the fact that the students had trouble communicating ideas and making connections on their own because similar issues arise in other aspects of life.  As an example, many people who have been immersed in a foreign language find that they can understand the language but they can’t speak it themselves.  This suggests that the ability to comprehend doesn’t necessarily translate to the ability to speak.

It was around this time that a friend of mine suggested group work.  To be honest, I thought it was just a waste of time.  I had heard about group work before, but I never really trusted the theories in favor of group work because they were based on words that I didn’t feel had a precise meaning, and they didn’t have measurable outcomes.  Proponents of those theories reminded me of those New Age treatment peddlers that are able to make wild claims because of how nebulous the terminology is.

“Use these crystals to protect against negativity and attune to your higher self.”

I needed to hear something that was more scientific.

Last summer, I took two physics pedagogy courses at Buffalo State College, and that is when I was finally exposed to a lot of the hard data:

In order to understand this graph, you need to know something about the Force Concept Inventory, or FCI.  It is a conceptual assessment test that was designed to gauge a student’s understanding of Newtonian concepts.  It has been around for about 20 years, and it has been extremely thoroughly vetted.  Typically, the FCI is given on the first and last day of class in order to see how much students learned.  The normalized gain is calculated by: (post-test % – pre-test %)/(100% – pre-test %).  This is a measure of how much a student learned relative to how much they could have learned.  Anyway, what Richard Hake did in his 1998 paper was to compare the FCI scores of 6,542 students from 62 classrooms.  He divided the classrooms into two categories: Traditional Instruction — defined as relying on passive-student lectures, recipe labs, and algorithmic-problem exams — and Interactive Engagement — defined as classes designed at least in part to promote conceptual understanding through interactive engagement of students in heads-on (i.e. minds engaged) and hands-on activities which yield immediate feedback through discussion with peers and/or instructors.  This graph shows the results, with normalized gain on the horizontal axis and the fraction of courses of that style that achieved that particular gain on the vertical axis.  Traditional Instruction courses are in red and Interactive Engagement courses are in green.  As you can see, there is a huge difference; in fact, the average gain of an Interactive Engagement course is roughly double that of a Traditional Instruction course.

I also watched Derek Muller’s Veritasium video in which he describes his doctoral research on how students understand and interpret physics material from video explanations.  He gave his own pre-test and post-test that was much like the FCI, and in between he showed a video that gave the answers to the majority of the questions on the test.  The students were quite confident that they did well on the post-test, yet the average score only went from a 6/26 to a 6.3/26.  So not only was the direct instruction not helpful, but it actually made the students more confident about their misconceptions.

Lastly, I was informed that Interactive Engagement physics classes produce nearly double the number of STEM (Science, Technology, Engineering, Math) majors as Traditional Instruction classes do.

That was enough data to get me to at least try teaching using Interactive Engagement methods, and actually experiencing Interactive Engagement for myself fully convinced me that this was the way to go.

Saving Time by Creating Models in Lab

In the summer of 2011, I attended a Modeling Workshop.  It was an intense two week (~90 hours) course designed to introduce us to an interactive method of teaching known as Modeling.  The basic idea of Modeling is this:

  1. Each unit starts with an experiment.  Students collect data and learn how to properly represent and interpret that data.
  2. Students use the experimental data to construct a physical or mathematical model.  For example, a student studying springs might realize that there is a linear relationship between the force applied to a spring and the amount it stretches by, which leads the student to develop an equation as a mathematical model of the relationship.  A student studying atomic bonds might realize that the properties of bonds are similar to the properties of springs, so he or she develops a physical model of an atomic bond as balls connected by a spring.  This is typically followed up by a presentation of results on portable whiteboards (a topic for another post), and the students are asked to come to a consensus on a model that explains their observations.
  3. Students apply the model they created to new situations and evaluate its applicability.

It was an awesome workshop, and it radically changed the way that I teach (by the way, there are Modeling Workshops for other subjects as well).  We were even given a set of materials and a course plan that we could use in our classes.  The only problem for me was that those materials were designed for high school physics, not calculus-based college physics.

In the following year of teaching, I used whiteboards and interactive methods of teaching, but I didn’t exactly follow the Modeling learning cycle as described above.

This summer I decided to spend some time redesigning the course so that I could incorporate more of the Modeling method.  The example lab that I described in a previous post was my first attempt at this, and I recently did this lab with my students.  The lab consisted of taking electric field strength and distance data from a simulation, then plotting that data and using it to create a mathematical model for electric field strength.  After the lab was over, I went back to my office to plan the next day’s lesson.  At first, I just copy-and-pasted the lesson that I used when I last taught the course; the lesson involved using a gravitational analogy as a means of deriving the relationship between electric field strength and distance.  I was pretty happy with how that lesson went last time, as the students got to start with something that was comfortable to them and create something new from it.  Having the students derive it also meant that they didn’t just have to watch me do it.  Lesson done…time for lunch!

Later that day I realized something: we had already derived the relationship between electric field and distance in the lab!  Sure, the gravitational analogy is cool and all, but why waste time doing a derivation when the empirical results had already done that for us in the lab?  Creating the model in the lab had actually saved us a bunch of time…time that I will gladly use on more interesting things in the future.

It makes so much sense when I think about it now.  We generally use regular class time to develop our physical and/or mathematical models, then use lab time to “prove” that those ideas were correct.  In addition to other reasons I dislike those kinds of labs, they are not a valuable use of time. Using lab time to develop those very same models, though, means that we don’t need to spend regular class time on them.  Instead, we can use our regular class time to practice using the concepts that they have learned.

The trick now is to see if I can come up with modeling-style labs for the other chapters we need to cover.