Is crowdsourcing a better choice for grading?

Is crowdsourcing a better choice for grading?

Summary: According to an article I stumbled across tonight, one Duke University professor thinks so. In a blog post, the professor, Dr.

TOPICS: Browser

According to an article I stumbled across tonight, one Duke University professor thinks so. In a blog post, the professor, Dr. Cathy Davidson, writes,

I loved returning to teaching last year after several years in administration . . . except for the grading. I can't think of a more meaningless, superficial, cynical way to evaluate learning than by assigning a grade. It turns learning (which should be a deep pleasure, setting up for a lifetime of curiosity) into a crass competition: how do I snag the highest grade for the least amount of work? how do I give the prof what she wants so I can get the A that I need for med school? That's the opposite of learning and curiosity, the opposite of everything I believe as a teacher, and is, quite frankly, a waste of my time and the students' time. There has to be a better way . . .

The "better way" involves allowing rotating pairs of students (who also lead classroom seminars) to evaluate whether student writing is acceptable or not. Dr. Davidson takes a very simple approach to the entire grading process:

Do all the work, you get an A. Don't need an A? Don't have time to do all the work? No problem. You can aim for and earn a B. There will be a chart. You do the assignment satisfactorily, you get the points.

The satisfactory piece is judged by those teams of students; it's similarly straight-forward.

Thumbs up, thumbs down. If not, any student who wishes can revise. If you revise, you get the credit. End of story. Or, if you are too busy and want to skip it, no problem. It just means you'll have fewer ticks on the chart and will probably get the lower grade. No whining.

It's an interesting approach, no doubt. Whether it has academic value remains to be seen, but it is abundantly clear that students operate in the real world much differently than they do in a college classroom. Their creative work is immediately evaluated by peers and they receive constant feedback about many aspects of their lives. In the same way, employers want to know if they met deadlines and created deliverables. An employer has little interest in whether the deliverable was "what they wanted to see;" rather, the deliverable needs to meet requirements.

I'm not 100% convinced yet; we'll need to check back on that blog at the end of the semester to see how things went.

Topic: Browser

Christopher Dawson

About Christopher Dawson

Chris Dawson is a freelance writer, consultant, and policy advocate with 20 years of experience in education, technology, and the intersection of the two.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Today, tomorrow and self interest

    With peer reviews, the problem is that the reviewers won't dare to criticize strongly. Or even at all...

    Their self interest is at stake: today they are the reviewers, but tomorrow their work will be reviewed by the same person whose work they are reviewing now.

    In other words: a classical (pun intended) case of conflict of interest.

    That's why an impartial judge is needed: a teacher, whose self interest doesn't play a role and who therefore can be trusted to judge more fairly.

    Perhaps the grade system isn't so bad after all...
  • RE: Is crowdsourcing a better choice for grading?

    Students evaluating students wont work for several reasons: 1) In a copmetitive environment - say Biology - where the profs only give out an A to a certain percentage no matter the performance, students will compete a la Big Brother. 2) The professor designs the class, not the students - who better to judge the outcome. 3) In the work environment, supervisors evaluate subordinates - it involves a strict reporting and responsibility relationship, not a peer thing. 4) Teaching is often not a priority - at most large research institutions teaching takes a back seat to research. At any time profs could give both formative (verbal/written/meetings) and summative feedback (grades) but they CHOOSE mostly to give only grades. Grades are easiest.

    Reality is, grades are not so much for students as for others beyond the student. They are an efficient summary mechanism. Parents want to quickly know if students are succeeding, employers want to know if students have requisite knowledge, grant givers want to rank applications...and so on. These would be awfully hard to do with no summary evaluations or conversly, piles and piles of written formative feedback - ugh - who would review that?
    • Well said.

  • World view

    Chris, this is a bit off topic but if you are going to talk about grading rather than tech (perfectly okay with me, since it does have to do with "education" ...), look into the "Greek, Hellenistic world view".

    Basically, the Western world view is based on world view concepts that arose in Greece several centuries BC. It basically looks at everything as information that can be analyzed, categorized, put into a series of fact statements, and passed on. Other world views, especially oriental, are more "holistic".

    I teach in a church's children's program and I like to use this example to show the contrast:

    Many children's programs have kids memorize the names of books of the Bible in order. That is the Greek, Hellenistic world view. Tommy got them all right but Joey missed two. We can [b][i]grade[/i][/b] that. In an adult scenario, Harry has an MBA from a top university with a 3.8 GPA. Therefore, he knows how to run a business. He got an A- in a Business Ethics course. Therefore he knows all about business ethics and will act ethically.

    On the other hand, the non-Hellenistic world view would focus on things like, "My kid didn't steal today. He had the opportunity to, but six months ago we were coming home from a store and I realized I had walked out without paying for something. I turned the car around and mentioned that we had to go back because if we kept it without paying for it it would be stealing even though I didn't mean to take it without paying for it. He thought about that and decided not to steal." But how do you assign a numerical grade to "My kid didn't steal."?

    This isn't just a philosophical matter. A huge problem with the approach of "reduce it to a series of fact statements and then communicate the fact statements" is that the "education" that results frequently isn't adapted to deal with the practical realities the student will encounter.
  • Grades are a motivator.

    Not everyone has the same passion for learning as this professor. As a matter of fact I would say that few do. And while this professor is not the first educator to dis grades, a proven better solution has not been offered. And in fact, one method may not be enough. There is such a disparity in what is required of a student between k-12 and higher education that different methods may be required. Personally I give the good professor an "F" for what they have proposed.
  • I'd rather have set standards.

    I've never cared for this sort of pied piper chicanery.

    Besides, if most students are like (many of) the ones in the college classes I'm in, there's no bleeping way I'd want to rely on them. On the plus side, the instructors are more interested in the deliverables and their requirements.
  • Or maybe just a lazy teacher

    Done properly a teacher does feedback and makes an effort to address the substandard performance. Often it's address right in the paper. A professional teacher also uses the grading of thier students work as a feedback mechanism for their own skillset. I don't know any teacher that really enjoys grading but most understand the need for it.

    Otherwise give me a good cclor laser printer, a PC with word and a stack of cardstock an I'll fix your graduation rate in about 30 minutes. Of course parents won't have a state provided baby sitter for 6-18 year old their kids.

  • To put it simply: NO.

    Crowdsourcing may be an effective teaching tool, but not a grading tool.
    By all means, give a student the information they need, to know what's expected of them and how those expectations will be met, so they have a guideline or standard to correlate their work to, to ensure they're achieving as they choose. Let them know that just 'doing' the assignment isn't enough for an 'A', that an 'A' indicates they've put some concerted effort in the assignment, they've stretched their boundaries and challenged themselves to go beyond a standard measure of capability. Kids want, and need, to know specifically what is expected of them to know how they can achieve. Teachers need to remember, too, that any particular assignment may have a very personal effect on a student, and to broadcast that to their peers, could cause emotional stress.
    When I was in school, I would have appreciated better guidelines and expectations stated, and there were certainly assignments I did that I would have been absolutely mortified to have had become part of a class discussion.
    I think a good techer can always relate the subject to the real world and show their students how the subject matter can and will affect them in their future.
  • Grading isn't the problem

    The way schools deem students 'worthy' is completely stupid and entirely irrelevant to the working world. A student who gets 90% on a test is considered to have done remarkably well, but what kind of business wants an employee who thinks getting 10% of their work wrong is satisfactory?

    Students should be expected to get 100% on everything. That said, everything should be open book (because everyone in the working world has access to information and it would be irresponsible NOT to check your facts) and students should be able to hand in a assignment as many times as they'd like BEFORE a specified deadline. That's how it works in real life: you can ask for help as often as you want, and you can get it done however you want, so long as it's finished by the due date. You say students would take advantage of this by cheating, but what's called cheating in school is called resourcefulness in the workplace. Does the boss care where you got the code from, so long as it works? No. Then why does teacher?

    This way, you wouldn't have to assign grades. It would be entirely a pass/fail system and no students should ever fail unless they're not putting enough effort in because every student would be expected to have turned their assignment in before the due date to ensure that it has been completed satisfactorily. If they haven't gotten teacher's approval and they fail? Their fault. Grades would then be a simple issue of averaging out the passed and failed assignments of every student (taking into account the weighting of each assignment).

    Not only would this teach the skills that the students are required to learn in school, but it would also teach discipline and work ethic, something that's severely lacking. I would know, as I was one of those students who got As with no effort at all. The work world was quite a shock to me because suddenly I couldn't get away with doing next to nothing anymore. It was a hard learned lesson that cost me a lot more than it ever should have.
    • hmm

      > Does the boss care where you got the code from, so long as it works? No. Then why does teacher?

      Intellectual property rights. Microsoft getting away with FUD because Linux allegedly contains code written by them which is not GPL. Taking companies to court for using said code. A recent case with a satnav company. So you have to be careful what you recycle.

      That's in computing terms as I assume the writer intends. If you're writing an English essay, the submission may simply amount to plagiarism. And copying, as we know means the work goes from source to destination without passing through the brain and without generating any thought.

      I suggest that bosses are probably quite interested in people who can analyse problems and come up with solutions.

      > those students who got As with no effort at all

      Yup, the school system can be entirely unchallenging for intelligent students who are expected to follow a syllabus. And as soon as an educationalist sets a test/exam procedure, especially one leading to a useful certificate, they're making statements between the lines about what's worthwhile and what's not.

      Most teachers I know are p*d off because syllabuses and exams are so proscriptive, and so frequent, they are unable to spend enough time working with and extending good students.

      Exams are there to ensure *minimum* standards are met. If you're MENSA category, you're not going to fit the system well.

      And while we're on it, most courses are pretty much left-brain. They're sequential, top down, etc etc. Little opportunity for lateral thinking. There are plenty people know what the answer is (in Maths say) but it's a bore having to go back and work out how you got there in detail. Or you're so 'intuitive' you can't explain how you got the answer. FAil.

      Businesses need both plodders and leapers.
  • RE: Is crowdsourcing a better choice for grading?

    This is a straw man argument. What is called "crowdsourcing"
    here has been called "peer evaluation" for decades. Most best
    practices teachers use it for reinforcement, as a component of
    an overall evaluation system. However, there are two distinct
    components to "grading" (as noted, passim, in previous posts):
    analysis of the quality of the work done and compensation for
    the work. Davidson is punting on both counts--especially since
    this is writing. First, most students are poor writers with poor
    analytic skills when it comes to delineating good writing from
    bad. The average high school or college classroom might have,
    at most, 25% of its members as solid writers. This is not to say
    the majority of students cannot judge the writing: they can
    discuss the content and logic of arguments, usually, and the
    source material for the exposition. However, in terms of the
    mechanics of exposition...not so much, if at all. Second, ask
    any student: would you rather your PhD professor tells you
    that you write well or your peeps in class? Yes, students grub
    for grades. Yes, evaluation should be based on students
    meeting 100% of the expectation w/r/t the classroom work.
    But, there need to be standards for performance that are clear
    to students and move them to higher achievement. Peer
    evaluation can help in many areas of the classroom, but it falls
    short in others--especially the analysis of writing mechanics,
    which is one of the most tedious, time-consuming things that
    humanities teachers have to do. I would not want my child in a
    college classroom where the teacher says that reading a
    student's paper carefully to evaluate it is a waste of her/his
    time. She could have achieved her goals by doing the initial
    evaluation herself, giving students a guideline for revising and
    improving their writing, and then turning to peer evaluation to
    see if it was 100% competent work compared to the original
    and the guidelines for improvement. Kid gets the "A" if he puts
    in the effort, both parts of the evaluation team receive
    reinforcement on writing mechanics, and the students are
    involved in the evaluation process in a meaningful way. Best
    practices like that take huge effort from the instructor, though.
    Not likely from someone who has neatly dodged the analysis
    and evaluation bullet by automatically claiming that it always
    destroys a love of learning.