I didn’t really break the entire VLE, but I buggered up an in progress assessment a few weeks ago, after submission of all the student work. This particular assignment, well, every year there’s something that goes a bit wrong, but I exceeded my own expectations this year. IT services are very good at bailing me out.
What is this assignment? Well it’s fairly simple ‘on paper’: students make 5 minute video/screencast presentations on a topic* (this year was ‘chemistry to change the world’), upload them to submit them, then do self- and peer- assessment. I moderate and deal with any technical difficulties. Turns out I create the worst technical difficulties of all.
When I first started this assessment, I put the students into groups by using blogs on the VLE (Blackboard). Then they peer assessed their blog group. I disliked this a bit because there was an air of ‘everyone give really good marks and we all benefit’. There was also the hassle of setting up groups and blogs. Then I explored the VLE a bit and encountered Blackboard’s ‘Self and Peer Assessment Tool‘. I’ll note at this point that the problem I came up against is documented in the guides for the tool but I guess I didn’t quite understand the implications when I read it. I’ve had a fair number of issues using the tool, initially there was a problem with preview users or students being assigned peer-assessment even if they were removed from the module. Felt like a refresh button needed pressed somewhere. Then there was the time that the means of inserting a video seemed to change and I had to provide a fair bit of support to students. This year I honestly thought I’d sorted out these issues or they’d been sorted.
I got the students to share their presentation as a link from a Google Drive folder (we’re on Gmail for campus email), or upload to YouTube or some other appropriate site. That sorted the inserting video problem.
The first self- assessment was a Google Form. And I spent quite a bit of time improving the peer-assessment questions and guidance notes this year to try to address the issue with slightly generous marking. Those were written through the VLE tool.
When it works, all I do to return the marks and feedback to the students is click a button and the peer assessment marks go to Grade Centre. I download the raw marks and calculate the final mark (30% for self-assessment, 70% peer assessment**), moderate where appropriate, impose caps as required and upload the final mark. Easy.
So what went wrong this year? I changed a date in the self- and peer- assessment tool. Apparently when you do this, it breaks it. There are several dates that you must set – the end date for submissions, the start date for peer- assessment and the end date for peer- assessment. My addled brain set the start date for peer- assessment wrong so I brought it forward. Critically I broke it AFTER all the students had submitted their work. This is also known as the worst possible point to bugger it up. If you do this when making the assignment, you just make a new one. If 5 students have submitted, no big deal. But a hundred odd students had submitted.
How to deal with this?
I created a new Google Form with the peer-assessment criteria and scales. I then worked out that all the submitted work could be downloaded as html files by student username (which is the first bit of our email addresses). Of the 100+ submissions, 90% of students had submitted a link for the presentation then plain text copied in for the bibliography. I uploaded the html files to the VLE as ‘add item’, the option that allows easy upload. And thank goodness for the recent update that allows drag and drop uploading. 10% had uploaded videos or attached word documents with the bibliography. These downloaded easily and I popped them in a shared Google Drive folder. I worked out the html files could be edited and in some cases I edited the submitted documents to include the link to the new location for the file. I also warned the students that some presentations were stored there.
The big question was how to allocate the peer-assessment. In the end, I went for a simple option that mostly worked. The submission html files were uploaded in username order so I asked the students to find their own username and assess the two above and the two below them in the list. Those at the start and end of the list should consider it circular and go to the bottom or the top to do their 4. I’d probably do it this way in the future.
From the student point of view, this mostly worked as intended. There were a few email queries but easy to deal with and fewer than previous years. From my point of view, the worst was yet to come. I had 1 google form with around 100 self-assessment marks, and another with around 550 self- and peer-assessment marks** and feedback. With a bit of excel wizardry, I turned the grades into something respectable in about 2 hours. I learned a couple of new tricks doing that so that was OK (compared to the 30 mins it would have taken). But the hour spent copy-pasting feedback into GradeCentre…that was painful.
So ultimately there was a fairly decent way to work around the problems I created with the tool. I need to decide what to use next year for this assignment because I’m not kidding when something new goes wrong every year. Sigh!
* there is limited timetable time for presentations in our 1st year. This allows a presentation opportunity with 1 hour of scheduled class time to introduce the assignment. The recorded presentations also allow a chance to review and reflect on certain aspects of presentations. And the peer- assessment means varied perspectives in feedback and the notion that the audience is a valid judge of presentation. This assignment works particularly well if I spend fewer hours on it than I would watching all the presentations and compiling the grades. This year that would be in the region of 500 minutes to watch all presentations + 180 minutes to compile the grades/feedback. I’d estimate I spent around 8 hours on this assignment but some of that was my own stupid fault fixing what I broke. I invest time in making improvements each year.
** there are two rounds of self-assessment. One on submission, one after carrying out peer-assessment. More on that another time.