Is crowdsourcing a better choice for grading?
Summary: According to an article I stumbled across tonight, one Duke University professor thinks so. In a blog post, the professor, Dr.
According to an article I stumbled across tonight, one Duke University professor thinks so. In a blog post, the professor, Dr. Cathy Davidson, writes,
I loved returning to teaching last year after several years in administration . . . except for the grading. I can't think of a more meaningless, superficial, cynical way to evaluate learning than by assigning a grade. It turns learning (which should be a deep pleasure, setting up for a lifetime of curiosity) into a crass competition: how do I snag the highest grade for the least amount of work? how do I give the prof what she wants so I can get the A that I need for med school? That's the opposite of learning and curiosity, the opposite of everything I believe as a teacher, and is, quite frankly, a waste of my time and the students' time. There has to be a better way . . .
The "better way" involves allowing rotating pairs of students (who also lead classroom seminars) to evaluate whether student writing is acceptable or not. Dr. Davidson takes a very simple approach to the entire grading process:
Do all the work, you get an A. Don't need an A? Don't have time to do all the work? No problem. You can aim for and earn a B. There will be a chart. You do the assignment satisfactorily, you get the points.
The satisfactory piece is judged by those teams of students; it's similarly straight-forward.
Thumbs up, thumbs down. If not, any student who wishes can revise. If you revise, you get the credit. End of story. Or, if you are too busy and want to skip it, no problem. It just means you'll have fewer ticks on the chart and will probably get the lower grade. No whining.
It's an interesting approach, no doubt. Whether it has academic value remains to be seen, but it is abundantly clear that students operate in the real world much differently than they do in a college classroom. Their creative work is immediately evaluated by peers and they receive constant feedback about many aspects of their lives. In the same way, employers want to know if they met deadlines and created deliverables. An employer has little interest in whether the deliverable was "what they wanted to see;" rather, the deliverable needs to meet requirements.
I'm not 100% convinced yet; we'll need to check back on that blog at the end of the semester to see how things went.
Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.
Talkback
Today, tomorrow and self interest
Their self interest is at stake: today they are the reviewers, but tomorrow their work will be reviewed by the same person whose work they are reviewing now.
In other words: a classical (pun intended) case of conflict of interest.
That's why an impartial judge is needed: a teacher, whose self interest doesn't play a role and who therefore can be trusted to judge more fairly.
Perhaps the grade system isn't so bad after all...
RE: Is crowdsourcing a better choice for grading?
Reality is, grades are not so much for students as for others beyond the student. They are an efficient summary mechanism. Parents want to quickly know if students are succeeding, employers want to know if students have requisite knowledge, grant givers want to rank applications...and so on. These would be awfully hard to do with no summary evaluations or conversly, piles and piles of written formative feedback - ugh - who would review that?
Well said.
World view
Basically, the Western world view is based on world view concepts that arose in Greece several centuries BC. It basically looks at everything as information that can be analyzed, categorized, put into a series of fact statements, and passed on. Other world views, especially oriental, are more "holistic".
I teach in a church's children's program and I like to use this example to show the contrast:
Many children's programs have kids memorize the names of books of the Bible in order. That is the Greek, Hellenistic world view. Tommy got them all right but Joey missed two. We can [b][i]grade[/i][/b] that. In an adult scenario, Harry has an MBA from a top university with a 3.8 GPA. Therefore, he knows how to run a business. He got an A- in a Business Ethics course. Therefore he knows all about business ethics and will act ethically.
On the other hand, the non-Hellenistic world view would focus on things like, "My kid didn't steal today. He had the opportunity to, but six months ago we were coming home from a store and I realized I had walked out without paying for something. I turned the car around and mentioned that we had to go back because if we kept it without paying for it it would be stealing even though I didn't mean to take it without paying for it. He thought about that and decided not to steal." But how do you assign a numerical grade to "My kid didn't steal."?
This isn't just a philosophical matter. A huge problem with the approach of "reduce it to a series of fact statements and then communicate the fact statements" is that the "education" that results frequently isn't adapted to deal with the practical realities the student will encounter.
Grades are a motivator.
I'd rather have set standards.
Besides, if most students are like (many of) the ones in the college classes I'm in, there's no bleeping way I'd want to rely on them. On the plus side, the instructors are more interested in the deliverables and their requirements.
Or maybe just a lazy teacher
Otherwise give me a good cclor laser printer, a PC with word and a stack of cardstock an I'll fix your graduation rate in about 30 minutes. Of course parents won't have a state provided baby sitter for 6-18 year old their kids.
Waa
To put it simply: NO.
By all means, give a student the information they need, to know what's expected of them and how those expectations will be met, so they have a guideline or standard to correlate their work to, to ensure they're achieving as they choose. Let them know that just 'doing' the assignment isn't enough for an 'A', that an 'A' indicates they've put some concerted effort in the assignment, they've stretched their boundaries and challenged themselves to go beyond a standard measure of capability. Kids want, and need, to know specifically what is expected of them to know how they can achieve. Teachers need to remember, too, that any particular assignment may have a very personal effect on a student, and to broadcast that to their peers, could cause emotional stress.
When I was in school, I would have appreciated better guidelines and expectations stated, and there were certainly assignments I did that I would have been absolutely mortified to have had become part of a class discussion.
I think a good techer can always relate the subject to the real world and show their students how the subject matter can and will affect them in their future.
Grading isn't the problem
Students should be expected to get 100% on everything. That said, everything should be open book (because everyone in the working world has access to information and it would be irresponsible NOT to check your facts) and students should be able to hand in a assignment as many times as they'd like BEFORE a specified deadline. That's how it works in real life: you can ask for help as often as you want, and you can get it done however you want, so long as it's finished by the due date. You say students would take advantage of this by cheating, but what's called cheating in school is called resourcefulness in the workplace. Does the boss care where you got the code from, so long as it works? No. Then why does teacher?
This way, you wouldn't have to assign grades. It would be entirely a pass/fail system and no students should ever fail unless they're not putting enough effort in because every student would be expected to have turned their assignment in before the due date to ensure that it has been completed satisfactorily. If they haven't gotten teacher's approval and they fail? Their fault. Grades would then be a simple issue of averaging out the passed and failed assignments of every student (taking into account the weighting of each assignment).
Not only would this teach the skills that the students are required to learn in school, but it would also teach discipline and work ethic, something that's severely lacking. I would know, as I was one of those students who got As with no effort at all. The work world was quite a shock to me because suddenly I couldn't get away with doing next to nothing anymore. It was a hard learned lesson that cost me a lot more than it ever should have.
hmm
Intellectual property rights. Microsoft getting away with FUD because Linux allegedly contains code written by them which is not GPL. Taking companies to court for using said code. A recent case with a satnav company. So you have to be careful what you recycle.
That's in computing terms as I assume the writer intends. If you're writing an English essay, the submission may simply amount to plagiarism. And copying, as we know means the work goes from source to destination without passing through the brain and without generating any thought.
I suggest that bosses are probably quite interested in people who can analyse problems and come up with solutions.
> those students who got As with no effort at all
Yup, the school system can be entirely unchallenging for intelligent students who are expected to follow a syllabus. And as soon as an educationalist sets a test/exam procedure, especially one leading to a useful certificate, they're making statements between the lines about what's worthwhile and what's not.
Most teachers I know are p*d off because syllabuses and exams are so proscriptive, and so frequent, they are unable to spend enough time working with and extending good students.
Exams are there to ensure *minimum* standards are met. If you're MENSA category, you're not going to fit the system well.
And while we're on it, most courses are pretty much left-brain. They're sequential, top down, etc etc. Little opportunity for lateral thinking. There are plenty people know what the answer is (in Maths say) but it's a bore having to go back and work out how you got there in detail. Or you're so 'intuitive' you can't explain how you got the answer. FAil.
Businesses need both plodders and leapers.
RE: Is crowdsourcing a better choice for grading?
here has been called "peer evaluation" for decades. Most best
practices teachers use it for reinforcement, as a component of
an overall evaluation system. However, there are two distinct
components to "grading" (as noted, passim, in previous posts):
analysis of the quality of the work done and compensation for
the work. Davidson is punting on both counts--especially since
this is writing. First, most students are poor writers with poor
analytic skills when it comes to delineating good writing from
bad. The average high school or college classroom might have,
at most, 25% of its members as solid writers. This is not to say
the majority of students cannot judge the writing: they can
discuss the content and logic of arguments, usually, and the
source material for the exposition. However, in terms of the
mechanics of exposition...not so much, if at all. Second, ask
any student: would you rather your PhD professor tells you
that you write well or your peeps in class? Yes, students grub
for grades. Yes, evaluation should be based on students
meeting 100% of the expectation w/r/t the classroom work.
But, there need to be standards for performance that are clear
to students and move them to higher achievement. Peer
evaluation can help in many areas of the classroom, but it falls
short in others--especially the analysis of writing mechanics,
which is one of the most tedious, time-consuming things that
humanities teachers have to do. I would not want my child in a
college classroom where the teacher says that reading a
student's paper carefully to evaluate it is a waste of her/his
time. She could have achieved her goals by doing the initial
evaluation herself, giving students a guideline for revising and
improving their writing, and then turning to peer evaluation to
see if it was 100% competent work compared to the original
and the guidelines for improvement. Kid gets the "A" if he puts
in the effort, both parts of the evaluation team receive
reinforcement on writing mechanics, and the students are
involved in the evaluation process in a meaningful way. Best
practices like that take huge effort from the instructor, though.
Not likely from someone who has neatly dodged the analysis
and evaluation bullet by automatically claiming that it always
destroys a love of learning.