Sally Jordan 5:00 PM Hello everyone Mark Glynn - DCU 5:01 PM Hi all - Pip Ferguson logging inn'screen Mathew Hillier 5:03 PM Greetings! Note! if you can NOT hear us please use Tools > Audio > Audio setup wizard - AP joined the Main Room. ( 5:04 PM ) - Claire 5:04 PM How can we access they key note presentation? Tim Hunt 5:04 PM Recording of the previous session: http://transformingassessment.com/events_30_june_2016.php (But, don't go and listen now ;-)) 5:04 PM Mathew Hillier 5:05 PM I will be interested to see how the bottom up v top down change can merge somewhere in the middle. Sally Jordan 5:06 PM Yes, that's a good point Mathew Kay Sambell 5:06 PM Indeed! Mathew Hillier 5:07 PM Greetings! Note! if you can NOT hear us please use Tools > Audio > Audio setup wizard Sally Jordan 5:08 PM Get assessment right and other things follow - good point! Mathew Hillier 5:11 PM the last bit is key! (having them use it) 5:11 PM Renee Denham (Griffith University, Australia) 5:11 PM How was workload impacted by increased feedback requirements? Tim Hunt 5:11 PM Wow! *reducing* the time to 3 weeks. From the student point of view, that is still slow. (But even that may be hard( Mark Glynn - DCU 5:11 PM Pip: yes there's evidence that students ignore a lot of even well targeted feedback Kay Sambell 5:12 PM This 'turnaround time' is quite common in Uk now Mark Glynn - DCU 5:12 PM Changing the culture is the hardest bit culture 5:12 PM Tim Hunt 5:12 PM The OU manages 10 working days, but that still feels slow. Renee Denham (Griffith University, Australia) 5:12 PM @Mark, so true. I spend a lot of time marking in the course I teach and giving detailed targeted feedback and so many students don't even read it. (and it means nights of no sleep given the 2 week feedback turn around we do) Maria Indrawan-Santiago 5:12 PM We only have 2 weeks to return assignments Renee Denham (Griffith University, Australia) 5:13 PM (even though their 2nd assignment builds off the first one so the feedback is critical for improving their results) Kay Sambell 5:13 PM Problem that feedback is still commonly (only) seen as tutor commenting on summative assignments? Mathew Hillier 5:13 PM the trick is to think of ways that students must engage with feedback in order to go on to the next bit. Ken Howah 5:13 PM Students often don't engage with feedback because they tend to view an assignment as "over" as soon as they hand it in Dave Morrison 5:13 PM Renee: I'll be discussing your concerns in a few mintues actually Mathew Hillier 5:13 PM e.g they need to write a reflection on the feedback inorder to access the numeric grade ... ;-) Dave Morrison 5:14 PM ...minutes Kay Sambell 5:14 PM FB needs to be built in much more holistically Claire 5:14 PM Mark 2 feedback is the key - Boud and Molloy - great resource Renee Denham (Griffith University, Australia) 5:14 PM *thumbs up* thanks Dave Mathew Hillier 5:14 PM or its part 1 or a multi-part assessment Mark Glynn - DCU 5:14 PM Yes, scaffolded assignments can work - but as Renee indicates, even then they sometimes don't pay attention to it (Pip here) Sally Jordan 5:14 PM @Ken - good point Dave Morrison 5:14 PM ...but have they been given a reason to pay attention? Kay Sambell 5:14 PM ...and don't forget what we called 'informal feedback' (Sambell et al 2013) - Dave Morrison 5:15 PM indeed Ken Howah 5:15 PM We need to connect the feedback to the assessment so that its viewed as one thing in students minds Kay Sambell 5:15 PM :) Renee Denham (Griffith University, Australia) 5:15 PM @matt, that's a good idea. We've put a change in for the final assignment this semester to try a peer assessment element, conduct a critique of another students learning object, they then need to implement changes based on the feedback and justify why they implemented / implemented changes based on some or all of the feedback. Ken Howah 5:15 PM ie, assessment ain't over till the feedbacks acknowledged Sally Jordan 5:15 PM Staff assessment literacy again - in my day job I'm HoD of a physical sciences dept; if we can change practice in my own dept we can change it anywhere! Mark Glynn - DCU 5:16 PM Mark and I are talking about whether grades might be withheld until students show they've engaged with the feedback. What do the rest of you think? (Pip Juliet Williams 5:16 PM Great re. transformative feedback Dave Morrison 5:16 PM Staff literacy came up as the key issue in our work too Mathew Hillier 5:16 PM yes - feedback comes in multiple forms. raising awareness of students that talk, discussions, Q&A etc are also a forms of feedback. Ken Howah 5:16 PM @Mark, could work I guess Mark Glynn - DCU 5:16 PM Good points, Sally! (Pip) Dave Morrison 5:16 PM Mark: that's a solid plan, however the feedback still needs to get to them effectively as well Juliet Williams 5:18 PM yes - evidence based scholarship is so important Sally Jordan 5:18 PM evidence-based scholarship - important! Kathie 5:18 PM Will you point us to the evidence that you used Dave Morrison 5:18 PM absolutely...I've found it is the only thing that gets people listening Claire 5:19 PM there is your answer Matthew - champions from the bottom up to drive change! Kathie 5:21 PM that's the prefect question!! Sally Jordan 5:21 PM change from the bottom up Judy Bloxham 5:21 PM In FE turnaround was 2 weeks and I set myself generally one week to do it. Yes it's hard work but benefits were worth it. I was Union chair, it does help Ken Howah 5:23 PM I find that students are not impressed with 3 weeks or even 2 weeks turnaround. 3 days and they'd be impressed... Judy Bloxham 5:23 PM @ken agree we live in an instant society now Mark Glynn - DCU 5:24 PM That sounds like a very attractive proposition for staff, kDave! (Pip) Sally Jordan 5:24 PM :-) Mathew Hillier 5:25 PM same Mark Glynn - DCU 5:26 PM We encourage tightly-designed online rubrics that provide specific targeted feedback and can be recorded and marked online (Pip) Tim Hunt 5:26 PM "No opportunity to ask..." Staff perceive that (some)students don't read feedback, but that is not all feedback. sorry, all students. 5:26 PM Renee Denham (Griffith University, Australia) 5:26 PM oh what a nightmare, yep! gvm 1 5:27 PM consistent feedbcak format .... what a worry Sally Brown 5:27 PM I am always very impressed by David Boud's work on what he calls 'final language' in feedback, which is destructive because it gives students nowhere to go 'grim' 'dreadful' 'hopeless'. Feedback needs to encourage and inform not destroy Ken Howah 5:27 PM "Useful" is the key word.. Mark Glynn - DCU 5:28 PM Totally agree, Sally. Why would anyone be that destructive? (Pip) Sally Jordan 5:28 PM I have similar evidence of timeliness being the most important thing Kay Sambell 5:28 PM 'Helpful' a good word Renee Denham (Griffith University, Australia) 5:28 PM did they indicate what they considered timely to be? Ken Howah 5:28 PM same thing... Judy Bloxham 5:28 PM Ive had assessment back at MA level complaining about spelling and grammar but only 2 instances in 2000 words high Mark Glynn - DCU 5:29 PM great quote Sally Brown 5:29 PM While 3 day return is ideal, in many contexts it's a matter of balancing the amount and quality of feedback with the timeliness of return. It's a trade off Judy Bloxham 5:29 PM Highlighted as wrong so very incongruous. I think staff search for something to comment about sometimes. A reason to mark you down! Renee Denham (Griffith University, Australia) 5:29 PM cross campus marking becomes a nightmare, you kill yourself getting marking done really quickly, but can't release until other campuses have finished marking so that moderation can occur before release. Juliet Williams 5:30 PM NSS wording is problematic - part of the problem in how we measure 'good' assessment and feedback? Sally Brown 5:30 PM Mark Glynn's work on using technology to support fast and effective work through Moodle is seminal: he has great apps to share Judy Bloxham 5:31 PM Like the interview lift pitch! Mark Glynn - DCU 5:31 PM flattery will get you everywhere Sally :) Renee Denham (Griffith University, Australia) 5:31 PM that's a very interesting point, I'll try to take that on board this semester! Mark Glynn - DCU 5:31 PM Mark's work is great - and I'm not saying that because he's sitting beside me, grin...Pip Mathew Hillier 5:32 PM @Mark - is there a link to find out more ? Mark Glynn - DCU 5:33 PM Several links but more will be posted here shortly https://enhancingteaching.com/ Mathew Hillier 5:33 PM thanks :-) Sally Brown 5:33 PM In relation to Miller's law I like 5 as we have 5 fingers and we can cope with this scale/range Ken Howah 5:34 PM I think it depends on the stakes...a PhD student values detailed feedback, an UG 1st year wants much less Renee Denham (Griffith University, Australia) 5:34 PM http://www.gla.ac.uk/services/learningteaching/aftoolkit/ Mathew Hillier 5:34 PM http://www.gla.ac.uk/services/learningteaching/aftoolkit/ Renee Denham (Griffith University, Australia) 5:34 PM LOL Mathew Hillier 5:34 PM this is a live link lol 5:34 PM Sally Jordan 5:35 PM correct Mark Glynn - DCU 5:35 PM There are ways of providing targeted critical feedback without being offensive (Pip) Mark Carver #2 5:35 PM This looks great, but I don't get a sense of dialogue - have you had to sacrifice this so that the advice is easier for all staff to understand or to cut workload? Judy Bloxham 5:37 PM There is the other side they can't work out quite why you deserve that mark so they pick on something small as the reason for knocking you down. There needs to be a correlation between mark and rationale/feedback Sally Brown 5:37 PM On the matter of tone and 'niceness' it depends on the relationship the assessor has with the student: you have to be more cautious with remote recipients but you can be more robust with people who trust you and you know well Kay Sambell 5:37 PM Dai Hounsell has some good ideas about constructive commenting on the WISE site- I'll try and post the link later when I find it Ken Howah 5:38 PM @sally, totally agree. Feedback can be misunderstood exactly like emails can be gvm 1 5:41 PM i think marking workload can be controlled and yet effective feedback given by giving it in form of questions, short phrases or even single words, e.g. evidence for this? Why? Check this! Dave Morrison 5:42 PM I used similar comments too, works well I think Sally Brown 5:42 PM I concur with Juliet that auditing practice is a really good place to start when aiming to change assessment at an institutional level ARU has done some brilliant work getting course teams systematically to review assessment and feedback practice 5:43 PM Mark Glynn - DCU 5:44 PM I think the data grid is a great stimulus to consider improving assessment (Pip) Kay Sambell 5:44 PM Juliet's point about thinking about FB from the student POV v important Tim Hunt 5:46 PM http://www.testa.ac.uk/index.php/about Mark Glynn - DCU 5:47 PM Juliet, have you found resistance from some who are heavy assessment users, not wanting to move to better but fewer assessments? (Pip) Sally Jordan 5:48 PM opportunities for Qs coming up soon YvonneW 5:48 PM is there a post testa version of the table? Mark Glynn - DCU 5:48 PM Is there a stick or carrot in terms of encouraging participation in the process Kay Sambell 5:48 PM An interesting paper by Harland et al (2015) on what they call an 'assessment arms race' .... Dave Morrison 5:49 PM The stick is usually PSR Sally Jordan 5:49 PM good point Dave Morrison 5:50 PM that said, in our use of TESTA, it was a positive stick. Subjects came wanting to be involved gvm 1 5:50 PM PSR? Dave Morrison 5:50 PM periodic subject review Judy Bloxham 5:51 PM Good example of understanding language PSR Dave Morrison 5:51 PM indeed =) Mathew Hillier 5:51 PM i think this is the paper mentioned just now...An assessment arms race and its fallout: high-stakes grading and the case for slow scholarship. Tony Harland , Angela McLean , Rob Wass , Ellen Miller , Kwong Nui Sim, Assessment & Evaluation in Higher Education Vol. 40, Iss. 4, 2015. http://www.tandfonline.com/doi/abs/10.1080/02602938.2014.931927 Kay Sambell 5:52 PM [Dai Hounsell commenting- http://www.cetl.hku.hk/teaching-learning-cop/high-impact-feedback/] Mathew Hillier 5:52 PM thanks Kay! i will add all these links to the session recording page. 5:53 PM Judy Bloxham 5:53 PM Has anyone tried a two step process, fast for main points then considered/moderated? Mark Glynn - DCU 5:53 PM That would be very helpful Mathew (Pip) Maria Indrawan-Santiago 5:53 PM thanks Matt, that would be great! Mark Glynn - DCU 5:54 PM No Judy; tell us more? (Pip) evelynv 5:54 PM 24 hour feedback means no life for academics!! Renee Denham (Griffith University, Australia) 5:54 PM really more? gvm 1 5:54 PM 24 hr feedback ... with 600 students? Renee Denham (Griffith University, Australia) 5:54 PM What was the average amount of assessment peices per course where they were saying more please? Tim Hunt 5:54 PM evelyn, are you sure? Perhaps no life the day after the assessment deadline, then 3 weeks without marking hanging over your head??? Mathew Hillier 5:55 PM using e-marking tools, e-moderation tools and multiple markers can help process it faster too. Judy Bloxham 5:55 PM @evelynv that's why academics have long holidays. Term time should be committed to tla evelynv 5:55 PM still teaching and more assessment coming in..... Mathew Hillier 5:55 PM not all assessments must be marked by humans too. some self-paced formative online activities will help students judge where they are at. Renee Denham (Griffith University, Australia) 5:56 PM @Judy, we see sessionals (like myself) doing a lot of marking too, and worklife balance is always important. Mathew Hillier 5:56 PM it takes a mix. Dave Morrison 5:56 PM not with 600, not for essays at least. but you can get 24 hour feedback to 600 if you use general feedback to the class, use types of assessment that can be marked fast...it always depends on the assessment itself YvonneW 5:56 PM Why do staff have to be providing the feedback? Isn't there value in developing students skills in critiquing their own work or peers? Sally Brown 5:57 PM 24 hour feedback relies on use of generic comments, statement banks and technology, as well as using different modes including live and oral feedback to groups, all a trade off, and inevitably different for students early on programme needing encouragement and those later needing more detail Judy Bloxham 5:57 PM Have staff at HE level forgotten what teaching actually entails? As someone who has worked in primary school, FE and HE Dave Morrison 5:57 PM In History in our study they mark and return, with verbal and written feedback, essays for upwards of 400 students always within 2 weeks. Mostly this is through well trained tutors Ken Howah 5:58 PM I've heard of some teachers giving an audio file feedback...supposed to be faster Judy Bloxham 5:58 PM @Dave institutional expectations to drive this level of commitment is great Dave Morrison 5:58 PM =) Renee Denham (Griffith University, Australia) 5:58 PM I like rubrics for speed and then dedicate time to a few tailored comments to the student. Kay Sambell 5:58 PM Involving students in evaluative review process is vital Carol 5:58 PM i think just a some people can compose and write feedback and other types of academic writing more quickly than others, I think the same applies to oral feedback. Mark Glynn - DCU 5:59 PM speed is not only benefit of audio. Rhythm and intonation are big advantages of audio feedback Judy Bloxham 5:59 PM Ignore those that give excuse we've always done it this way as an excuse not to change YvonneW 5:59 PM yes - much easier than crafting text Mathew Hillier 5:59 PM alignment ... of strategy, reward/tenure systems, policies, desired course outcomes etc. Dave Morrison 5:59 PM absolutely...tradition is no excuse Mark Glynn - DCU 5:59 PM could not agree more Sally AP 6:00 PM Michael Henderson (Monash) has tried video feedback; see http://ajet.org.au/index.php/AJET/article/view/1878 Mathew Hillier 6:00 PM Michael will be speaking in our next webinar! on this very topic 6:00 PM AP 6:00 PM Great timing! Maria Indrawan-Santiago 6:00 PM (y) Judy Bloxham 6:01 PM @Dave great method of pushing change Mark Glynn - DCU 6:02 PM There were some great assessment ideas presented by students in a 'Student-Led Learning' project here in Ireland last year (Pip) Sally Brown 6:03 PM Key point Juliet: we have to develop the assessment literacy of senior managers Mathew Hillier 6:03 PM Laura P 6:03 PM Thank you very much for a very interesting webinar! Mathew Hillier 6:03 PM you are welcome! Mark Glynn - DCU 6:03 PM Thank you from Pip and I Mathew Hillier 6:03 PM great session :-) Renee Denham (Griffith University, Australia) 6:03 PM yes thanks very much, really interesting! AP 6:03 PM Great session! Look forward to exploring the links. Tim Hunt 6:03 PM I think it has worked really well. Good collaboration. YvonneW 6:03 PM :) Juliet Williams 6:03 PM Thank you! Kay Sambell 6:03 PM Thanks, everyone, for faciltating this Carol 6:03 PM really excellent session even arriving late. bye Judy Bloxham 6:03 PM Worth getting up early to listen in thanks Mathew Hillier 6:03 PM Dave Morrison 6:04 PM Thank you for so many great questions and discussion Sheona Thomson (QUT) 6:04 PM Fantastic session - thank you all. Maria Indrawan-Santiago 6:04 PM Thanks Matt for a great session! Renee Denham (Griffith University, Australia) 6:04 PM thanks all :) looking forward to the next session Sally Jordan 6:04 PM I would encourage everyone to listen to Sue Bloxham's keynote from the Manchester meeting Ken Howah 6:04 PM Thanks all, very though provoking session. Bye all Ellen McIntyre 6:04 PM Thank you. Great information Veronica Bruijns 2 6:05 PM Thanks, interesting Juliet Williams 6:05 PM Thanks for the opportunity - really enjoyed the session Tim Hunt 6:05 PM Do we have up to 3 weeks to complete the feedback form ;-) Sally Jordan 6:05 PM @Time :-) Sally Jordan 6:05 PM I meant @Tim, I've got time on the brain! Sally Jordan 6:06 PM Bye all and thanks again