Thursday, 31 July 2014

Visualising Programme Assessments: do you know when assessments are due?

From time to time I try to step back from the work I'm doing and think about things from other people's perspectives - through Brookfield's reflective lenses. I find this helps take a holistic view of situations and identify challenges/benefits that you wouldn't normally come across. This is partly how I started thinking about visualising assessment timelines. As an academic, I've tended to only really care about how I'll be assessing within my own modules. What other people do in their modules is down to them.

But is it? Does anybody take a holistic view? How will my assessment requirements impact upon the wider student experience and workload? How will it impact on the admin team? The timetabling team? Marking and Moderators?
I'd love to hear about your views on this in the comments.

I wrote a few months back about visualising assessment timelines, and shifting from having a small number of high stakes assessment activities to more lower stakes assessments. Well in this post, I'm sharing a draft timeline of the entire Medical Curriculum at Liverpool.

The nature of the MBChB curriculum is different - there are no modules/units, so being able to see the bigger picture is increasingly important for the management and admin processes in the school. Furthermore, a simple visual take on assessments would undoubtedly be a useful reference for students.

Over the past couple of months I've met with the Director of Assessment in the School of Medicine to discuss the assessment timings in the new curriculum and have put together a visual overview (see below).
[please remember this is just a draft and not final details - more to be added]

The Year of study goes horizontal. Weeks go vertical. 

Now although some details are missing, you can see the general pattern of the programme from this overview - students are more heavily based at the University in the first couple of years, and then are pretty much never here later on (particularly 5th year). Here the assessments represent something along the lines of a sign off meeting to ensure the student is doing the things they're supposed to be (I'm actually not a specialist on the Medical Curriculum, as you can probably tell).

We can also see that in the back end of semester 2, there are quite a few assessments taking place across a short space of time. Over that 10 week period there is almost at least one assessment per week. The benefit of these visuals is that we can easily see that and plan accordingly. Perhaps we might shift assessments from one week to the next so we can more effectively manage the workload. Are we over/under assessing students across the years? Could we distribute the assessments more effectively?

So the overview can be a pretty useful tool, but a more specific view of (a year/module) can tell us more information. The image below samples Years 1 and 2.
[again, draft]


So this time, we can see the online formative/low stakes assessments (green) relate to the blocks of study (these used to be our PBL modules but have changed slightly in our new curriculum). If we wanted, we could add further details, just as I have done for the higher stakes/summative pieces e.g. 2 x 1.5 hr papers.

The Research & Scholarship refers to one of the themes in the new spiral curriculum. Again we could add more detail here.

What's next?

We could fairly easily make an interactive version of these timelines that enabled students to quickly browse all of their assessment requirements for the whole programme, with the ability to drill down to a specific assessment. We just need the information, and although that can sometimes be a painful process, I can't imagine a situation where any school management and admin teams would not want this overview. In fact, some institutions require assessment type and date, etc, at the beginning of the year - in which case this should prove an easy task.

So why can't/shouldn't we do this for every single programme?

When we do this on a modular programme, we could see how assessments might be clustered at specific points in the calendar. We could also try to emphasise a shift from high stakes to low stakes assessments. This would be less stressful for students if nothing else.

One key factor for consideration is the terminology we use when developing any guidance for students, let alone a visual guide with few explanatory words. For example;
  • 'high stakes' can strike fear into students;
  • 'low stakes' could be taken for 'doesn't matter';
  • 'formative' and 'summative' - well do students even know what they mean?
So as we release these timelines to students, we will have to consider the terminology we use.

I'll be talking about this a bit more at the eAssessment Scotland conference in September. So if you're going, try to forget this post ;-)

Peter
@Reedyreedles

Creative Commons License
The Reed Diaries by Peter Reed is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License

2 comments:

  1. Hi Peter, you might be interested in using the Google tool we developed, based on the escape project, and which has been used worldwide over the last few years: https://sites.google.com/site/mapmyprogramme/home
    Also: http://learningtechnologiesteam.blogspot.co.uk/2012/03/event-report-map-my-programme.html

    and work that Edinburgh have done building upon our work: https://www.wiki.ed.ac.uk/display/casestudies/IS+and+School+of+Education+-+Mapping+of+Assessment+and+Feedback+in+Teaching+Programmes

    ReplyDelete
    Replies
    1. Thanks Simon,
      I'd come across the Escape project and linked to it in a previous post on assessment timelines. Some really good work and love the idea of the Google tools to visualise assessment easily. Will be having a play with that.
      Cheers

      Delete