Multiple choice tests for dummies

How can we expose more people to critical thinking?
Jeff
Posts: 405
Joined: Sat Jun 05, 2004 6:01 pm

Multiple choice tests for dummies

Post by Jeff »

I had some students complain that my tests were unfair because they only consisted of short essay and fill-in questions.
"Everyone uses multiple choice questions!"
Yes, especially those teachers who look at a test bank file, check off which questions should be included, have it produced by computer or a person, and then have it graded by an assistant or machine.
This is why, when people get a question such as,
"This course is titled "Introduction to ____________, which is defined as:

", they freak, because they have never had to write a letter past d, or maybe e, on a test, once they signed their name.

User avatar
gnome
Posts: 23753
Joined: Tue Jun 29, 2004 12:40 am
Location: New Port Richey, FL

Re: Multiple choice tests for dummies

Post by gnome »

Jeff wrote:I had some students complain that my tests were unfair because they only consisted of short essay and fill-in questions.
"Everyone uses multiple choice questions!"
Yes, especially those teachers who look at a test bank file, check off which questions should be included, have it produced by computer or a person, and then have it graded by an assistant or machine.
This is why, when people get a question such as,
"This course is titled "Introduction to ____________, which is defined as:

", they freak, because they have never had to write a letter past d, or maybe e, on a test, once they signed their name.
It was generally recognized in my college classes that the most difficult test questions were fill in the blank. For multiple choice/true-false you could guess. For essay you can bring out the shovel. But for fill-in-the-blank you have to know the answer. Period. No amount of BS will get you through that.

In my toughest class we had seven-page fill-in-the-blank/short-answer/essay combinations. They were nightmares. When we reached a unit on Dante, someone hung on the door the day of the test, "Abandon hope all ye who enter here."

But I must say I got a hell of an education from it.

Jeff
Posts: 405
Joined: Sat Jun 05, 2004 6:01 pm

Post by Jeff »

I hope the instructor corrected that quote to " Abandon all hope, ye who enter here", which I think is what "Lasciate ogne speranza, voi ch'entrate" means.
Not to be pedantic or anything.

User avatar
iain
Posts: 187
Joined: Mon Jun 14, 2004 8:55 am
Location: Cheshire, England

Post by iain »

Well designed multiple-choice tests can be excellent and in IT are pretty much the standard. Some of the ones I've taken have been very tough indeed. I've done some essay exams where I've basically bluffed my way through and scraped a decent pass, not something that's possible with multiple choice.

There's a place for both. Both are fine for testing knowledge. Essay questions additionally test language/grammar skills which may be somewhat lacking. Probably, though, most students would feel very unfair that they should be marked down just for not being able to write a coherent sentence.

I did essay exams exclusively at university and it never did me any harm. *twitch*

Jeff
Posts: 405
Joined: Sat Jun 05, 2004 6:01 pm

Post by Jeff »

I don't know about IT, but in psychology they can be quite confusing and cause the students to recall information that is incorrect because they saw it on an exam.
Example:
"When previous memories disrupt the recall of more recent experiences, this is termed:
a. Retroactive interference.
b. Proactive interference.
c. Retrograde amnesia.
d. Anterograde amnesia."

That test item could well cause retoactive interference in the recall of the definition of retroactive interference.

Also, I have seen test banks for instructors where the wrong answer was starred as correct.

User avatar
TruthSeeker
Posts: 420
Joined: Wed Jun 09, 2004 9:53 pm
Location: On a comfy chair with a sleeping cat

Post by TruthSeeker »

I use both types of exams.

I write my own multiple choice questions and only re-use them if the psychometrics look good (i.e. % who got it correct, distribution of incorrect responses). I use them because I teach a class of over 800 students. It would be impossible to have the grading resources for any other type of format.

In my higher year course, I have 50 students. There I use short answer and essay type exams. I also make them give presentations. It is more manageable at that size.

User avatar
Brown
Posts: 297
Joined: Wed Jun 09, 2004 3:15 pm

Post by Brown »

If you think students complain about essay questions, you ought to see them gripe when they have to answer questions that require artistic skill.

In some of my classes, the exams obligated us to draw two- or three-dimensional representations of objects. In an advanced Chemistry class, for example, we had to draw a three-dimensional representation of a particular molecule. Even though there had been specific instruction in the lectures about how to do it, many students screamed bloody murder when the question appeared on the exam. In Engineering and Physics classes, we had to draw free-body diagrams, as well as particular "views" of objects. Students were told they would be tested on these subjects, but they complained mightily anyway.

As for me, I have a bit of artistic skill (and I received an excellent grade school education in basic drawing techniques; thanks, Mr. Hiatt!), so the questions were a snap for me. (I always got 100% on any question that involved artistic ability.) But many smart people felt that such questions were unfairly skewed against them, because they did not know how to draw well.

Jeff
Posts: 405
Joined: Sat Jun 05, 2004 6:01 pm

Post by Jeff »

Brown,
You have a point about huge classes. My first year teaching, I had an intro class of 200. It was horrible, even with help of a grad TA. It prompted me to adopt the Keller Plan, or the Personalized System of Instruction (PSI) for the next year.
Now my largest intro class is 43, and fill-in tests can be administered and graded in one class period to provided immediate feedback.
Another advantage of fill-in items is that they allow you to recognize novel correct responses or partially correct ones.

User avatar
Quester_X
Posts: 12926
Joined: Fri Jun 04, 2004 9:26 pm

Post by Quester_X »

I had some students complain that my tests were unfair because they only consisted of short essay and fill-in questions.
You teach? Cool. What subject?
You'll all be mine, in the end.

Jeff
Posts: 405
Joined: Sat Jun 05, 2004 6:01 pm

Post by Jeff »

Psychology. This semester, Intro I and Experimental I and II. Last year, those and Critical Thinking and Applied Behavior Analysis.

User avatar
gnome
Posts: 23753
Joined: Tue Jun 29, 2004 12:40 am
Location: New Port Richey, FL

Post by gnome »

Jeff wrote:I hope the instructor corrected that quote to " Abandon all hope, ye who enter here", which I think is what "Lasciate ogne speranza, voi ch'entrate" means.
Not to be pedantic or anything.
Actually I'm sure the sign had the correct quote (the one you used) and I misremembered it--as the student that hung the sign was rather pedantic himself :) It's been 15 years or so.

Nyarlathotep
Posts: 49191
Joined: Fri Jun 04, 2004 2:50 pm

Post by Nyarlathotep »

I am not a teacher, but I feel the classes I learned the most in were the ones where multiple choice questions were not the norm. This is especially true for my biggest interest which is history. Multiple choice questions can only test knowledge of basic facts, they don't demonstrate reasoning ability at all. The problem with that in subjects like history is that memorizing facts isn't the point. If I really need to know,say, what date the Magna Carta was signed on, I can easily look it up. Knowing why it was signed and what factors led to the signing is more to the point of taking such a class and a multiple choice question can't really show whether you have learned that or not.

Paul
Posts: 142
Joined: Tue Sep 28, 2004 1:03 am

Post by Paul »

Nyarlathotep wrote:If I really need to know,say, what date the Magna Carta was signed on, I can easily look it up. Knowing why it was signed and what factors led to the signing is more to the point of taking such a class and a multiple choice question can't really show whether you have learned that or not.
What about this:

The Magna Carta was signed because
A. The King had no choice
B. It was in the King's own interest
C. He was feeling a little frisky that day
D. None of the above

A multiple-choice exam doesn't have to be limited to basic or fundamental facts.

Paul

User avatar
Doctor X
Posts: 72256
Joined: Fri Jun 04, 2004 8:09 pm
Title: Collective Messiah
Location: Your Mom

Post by Doctor X »

Bah!

Whiners!

Take a Dan Test--perform numerous katas, pre-arranged kumite [Two fools hit one another in a pre-arranged sequence.--Ed], conditioning [Two fools hit one another.--Ed.], bunkai [One fool attacks the other fool who must demonstrate the application of kata.--Ed.], questions [Fool must know in what province Grand Poohbah pulled his groin muscles, which split the boulder.--Ed.], then jiyu-kumite [Two fools attack one another to the amusement of the testing board.--Ed.]

Injures neck trying to look up whilst "looking down" on the mere peasants. . . .

--J.D.
Mob of the Mean: Free beanie, cattle-prod and Charley Fan Club!
"Doctor X is just treating you the way he treats everyone--as subhuman crap too dumb to breathe in after you breathe out." – Don
DocX: FTW. – sparks
"Doctor X wins again." – Pyrrho
"Never sorry to make a racist Fucktard cry." – His Humble MagNIfIcence
"It was the criticisms of Doc X, actually, that let me see more clearly how far the hypocrisy had gone." – clarsct
"I'd leave it up to Doctor X who has been a benevolent tyrant so far." – Grammatron
"Indeed you are a river to your people.
Shit. That's going to end up in your sig." – Pyrrho
"Try a twelve step program and accept Doctor X as your High Power." – asthmatic camel
"just like Doc X said." – gnome

ImageWS CHAMPIONS X4!!!! ImageNBA CHAMPIONS!! Stanley Cup!Image SB CHAMPIONS X6!!!!!! Image

User avatar
Luke T.
Posts: 27092
Joined: Mon Jun 07, 2004 1:40 pm
Location: Nowhere near Pakistan

Post by Luke T. »

TruthSeeker wrote:I use both types of exams.

I write my own multiple choice questions and only re-use them if the psychometrics look good (i.e. % who got it correct, distribution of incorrect responses). I use them because I teach a class of over 800 students.
800 students? How in blazes can you teach a class of 800 students? It would take a couple hours just to take attendance!

The potential for fraud practically begs for it to occur.
[size=75]"it seems you don't believe how your enviroment of smells affects for a young ones.how many young girl are are in just involved in porn industry just because of lack of natural smells" - pillory (7/13/02)[/size]

I [size=167]♣[/size]69dodge

User avatar
Luke T.
Posts: 27092
Joined: Mon Jun 07, 2004 1:40 pm
Location: Nowhere near Pakistan

Post by Luke T. »

The hardest class I ever took was on Soviet history. In the first five minutes of the class, the instructor (a JAG officer) explained it was impossible to understand Soviet history without understanding Russian history, and he proceeded to start at the beginning. Literally. He started with a mythological man first setting foot on a grassy plain in what is modern day Russia, and went from there.

His tests were 100 percent essay. And we had to write term papers. And even after he explained how he graded them, no one understood what the hell he was talking about. I never did figure it out, but I still got an A.

Here's how tough this bastard was. A week before the mid-term, my wife was found to have cancer. We were stationed in Guantanamo Bay and she had to be flown to Portsmouth Virginia's naval hospital for surgery right away. So I go to my instructor to let him know I'll be out of town due to my wife's cancer. He gives me the assignments for the next week. A massive amount of work, and I do them in the hospital waiting room. When I return, I take the mid-term. But since I was two days late (!!!), the instructor penalized me 10 points on my test grade!

I still ended up making the highest grade in the class when it was all over.
[size=75]"it seems you don't believe how your enviroment of smells affects for a young ones.how many young girl are are in just involved in porn industry just because of lack of natural smells" - pillory (7/13/02)[/size]

I [size=167]♣[/size]69dodge

User avatar
TruthSeeker
Posts: 420
Joined: Wed Jun 09, 2004 9:53 pm
Location: On a comfy chair with a sleeping cat

Post by TruthSeeker »

Luke T. wrote:
TruthSeeker wrote:I use both types of exams.

I write my own multiple choice questions and only re-use them if the psychometrics look good (i.e. % who got it correct, distribution of incorrect responses). I use them because I teach a class of over 800 students.
800 students? How in blazes can you teach a class of 800 students? It would take a couple hours just to take attendance!

The potential for fraud practically begs for it to occur.
Hi Luke
Teaching a large class is like giving a performance really. You have to use really effective slides and tune into the general pace of the class. I do encourage discussion but you can imagine this is usually confined to just the first few rows of students.

We don't take attendance for lectures. Students write their exams in smaller groups in smaller rooms so that they can be monitored more closely. Then we do take attendance and check student id. Also, I make up several different exams to discourage cheating.

It is not an ideal situation, that's for sure.

User avatar
Luke T.
Posts: 27092
Joined: Mon Jun 07, 2004 1:40 pm
Location: Nowhere near Pakistan

Post by Luke T. »

I was an instructor in the Navy. We had multiple choice question tests I wrote myself, but since it was a technical course, we also had troubleshooting practical tests which, to me, are what really determined whether the student was going to graduate my class.

I'm a big picture kind of person, and I believe a strong bedrock of theory will assist a person in intuitively being able to troubleshoot equipment. But I've seen some really book smart people who couldn't troubleshoot a flat tire.

Practical application. Can you fix the damn thing? That is what it was all about for me.
[size=75]"it seems you don't believe how your enviroment of smells affects for a young ones.how many young girl are are in just involved in porn industry just because of lack of natural smells" - pillory (7/13/02)[/size]

I [size=167]♣[/size]69dodge

User avatar
Denise
Posts: 805
Joined: Tue Jun 08, 2004 4:13 pm

Post by Denise »

Nyarlathotep wrote:I am not a teacher, but I feel the classes I learned the most in were the ones where multiple choice questions were not the norm. This is especially true for my biggest interest which is history. Multiple choice questions can only test knowledge of basic facts, they don't demonstrate reasoning ability at all. The problem with that in subjects like history is that memorizing facts isn't the point. If I really need to know,say, what date the Magna Carta was signed on, I can easily look it up. Knowing why it was signed and what factors led to the signing is more to the point of taking such a class and a multiple choice question can't really show whether you have learned that or not.
So true!!! I love history too and I did take a couple of classes at community college ( I have no degree) and loved the essay tests. It's the understanding more than memorizing.

Nyarlathotep
Posts: 49191
Joined: Fri Jun 04, 2004 2:50 pm

Post by Nyarlathotep »

Denise wrote:
Nyarlathotep wrote:I am not a teacher, but I feel the classes I learned the most in were the ones where multiple choice questions were not the norm. This is especially true for my biggest interest which is history. Multiple choice questions can only test knowledge of basic facts, they don't demonstrate reasoning ability at all. The problem with that in subjects like history is that memorizing facts isn't the point. If I really need to know,say, what date the Magna Carta was signed on, I can easily look it up. Knowing why it was signed and what factors led to the signing is more to the point of taking such a class and a multiple choice question can't really show whether you have learned that or not.
So true!!! I love history too and I did take a couple of classes at community college ( I have no degree) and loved the essay tests. It's the understanding more than memorizing.
I am glad to see that I am not the only weirdo that ENJOYS essay questions on history tests. :D I enjoy writing real eassays even more. I always thought the people who whined about them were missing out on the best part of the subject.