Rise of The Robo-Readers
July 13 @ 4pm PST/7pm EST #sschat
co-mods: @scottmpetri & @DavidSalmanson
A primer on auto essay scoring
Q1 What is your definition of AES, robo-reading, or robo-grading? #sschat
Q2 What is greatest hope and/or your worst fear about technology-assisted grading? #sschat
Q3 When is it ok for a computer to assign grades on student work? #sschat
Q4 How can classroom teachers test & evaluate a robograder without disrupting learning? #sschat
Q5 What would parents think if Ts required Ss to use robo-graders before submitting work? #sschat
Q6 What would school admins say if you used a robograder in your classes? #sschat
Q7 How would you use a robograder in your History-Social Science class?
Q8 How could robo-readers help teachers gamify the art and process of writing?
Shameless plug: https://www.canvas.net/browse/ncss/courses/improving-historical-writing has a module on writing feedback & AES. Course is free and open til Sept. 22. #sschat
Teaser Tweets (to promote the chat after Monday – 7/6).
Are robo-graders the future of assessment or worse than useless? http://wp.me/4SfVS #sschat
Robo-readers are called Automated Essay Scorers (AES) in education research. http://wp.me/4SfVS #sschat
In one study, Ss using a robo-reader wrote 3X as many words as Ss not using the RR. http://wp.me/4SfVS #sschat
Robo-readers produce a change in Ss behavior from never revising to 100% revising. http://wp.me/4SfVS #sschat
Criticism from a human instructor has a negative effect on students’ attitudes about revisions. http://wp.me/4SfVS #sschat
Comments from the robo-reader produced overwhelmingly positive feelings for student writers. http://wp.me/4SfVS #sschat
Computer feedback stimulates reflectiveness in students, something instructors don’t always do. http://wp.me/4SfVS #sschat
Robo-graders are able to match human scores simply by over-valuing length compared to human readers. http://wp.me/4SfVS #sschat
None of the major testing companies allow open-ended demonstrations of their robo-graders http://wp.me/4SfVS #sschat
Toasters sold at Walmart have more gov. oversight than robo-readers grading high stakes tests. http://wp.me/4SfVS #sschat
What is the difference between a robo-reader & a robo-grader? http://wp.me/4SfVS #sschat
To join the video-chat follow @ImpHRW, sign into www.Nurph.com. Enter the #ImpHRW channel. Note you will still need to enter #sschat to your tweets.
Promo Video for a forthcoming Turnitin.com product
A longer paper by Shermis & Hamner
Perelman’s full-length critique of Shermis & Hamner
If you are really a hard-core stats & edu-research nerd
National Council of Teachers of English Statement
For Further Research
Williamson, D. M., Xi, X., & Breyer, F. J. (2012). A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31(1), 2-13.