WEBVTT 00:00:05.000 --> 00:00:05.900 Zurich 00:00:06.000 --> 00:00:06.900 Morning from the not so sunny Yorkshire Dales, UK 00:00:08.000 --> 00:00:08.900 From Muscat, Oman 00:00:11.000 --> 00:00:11.900 Hi Everone, Sipho from Wits, South Africa 00:00:17.000 --> 00:00:17.900 Hi ACU team 🙂 @Leigha @Laurine 00:00:27.000 --> 00:00:27.900 Johannesburg, South Africa 00:01:13.000 --> 00:01:13.900 Rajohane, Matshedisho, South Africa 00:01:50.000 --> 00:01:50.900 Welcome - we have just started - 297 registered for today's session 🙂 00:04:01.000 --> 00:04:01.900 Yes i can hear 🙂 00:04:58.000 --> 00:04:58.900 Please type your questions in the chat 00:07:30.000 --> 00:07:30.900 The relationship between peer FB and academic integrity would be interesting to investigate... 00:09:15.000 --> 00:09:15.900 telling students when they are getting feedback is helpful to i.e sometimes students dont recognise it as feedback. esp in live environments. 00:10:00.000 --> 00:10:00.900 It would, but it also is dependent upon the academic/lecturer ensuring that assessment tasks reduce the possibility of transgressing academic integrity. 00:10:05.000 --> 00:10:05.900 If using something like the workshop to do the peer review, is it better then to NOT have anonymous set? 00:10:08.000 --> 00:10:08.900 The emotional aspects are so important, and something on which teacher display different level of training. 00:11:28.000 --> 00:11:28.900 @Laurine. Good comment. Assessment that is iterative with feedback loops (self, peer and tutor) is proven to enhance academic integrity. See Principle 7 - Academic Integrity Principles at https://teuintegrityproject.wordpress.com/principles/ 00:11:56.000 --> 00:11:56.900 @Fiona, thanks! 00:12:21.000 --> 00:12:21.900 Probably still related to students seeing the value of it, hence not finding an incentive to cheat a system that they recognise as valid and useful? 00:14:08.000 --> 00:14:08.900 Isn't it better to have more than one peer providing feedback on the same work as having only one peer giving feedback gives only one view that may be unhelpful? 00:14:45.000 --> 00:14:45.900 If any of the organisers are able to look at this - a colleague has messaged me to say that there are about 12 attendees stuck in an empty room and can't get into the webinar... Links have taken them somewhere else. 00:15:10.000 --> 00:15:10.900 Sarah - will investigate 00:15:42.000 --> 00:15:42.900 @Mike I would agree, this supports the development of more than the two students 00:16:02.000 --> 00:16:02.900 @mike, @fiona, yes that sounds good 00:17:28.000 --> 00:17:28.900 be interesting to try this in a shorter time frame, we have 8 week delivery with a fair amount of academic writing - and attendant academic integrity issues. Would require quite an intensive approach from both facilitator and students. 00:18:22.000 --> 00:18:22.900 I have found that peer feedback only works if I give marks for it so contributes to module mark 00:18:47.000 --> 00:18:47.900 @mike I was just reflecting on if these students might be more motivated being in an advanced course 00:19:11.000 --> 00:19:11.900 e.g. some students might need additional motivation to participate in peer feedback, but others might be fairly intrinsically motivated already 00:19:20.000 --> 00:19:20.900 and yet allocating marks to peer feedback interferes with the value of peer feedback. Subverts its aims. 00:19:43.000 --> 00:19:43.900 You could provide feedback on their feedback to improve evaluative judgement 00:20:11.000 --> 00:20:11.900 I assess the peer feedback and provide feedback on this with mark 00:20:42.000 --> 00:20:42.900 Totally agree, teacher presence essential 00:20:55.000 --> 00:20:55.900 so far I've given marks to the reviewer but review has no impact on the score for the reviewee 00:20:56.000 --> 00:20:56.900 @Monique, we have a similar situation - effective feedback techniques become time and resource intensive when a course runs over just a few weeks, esp with large classes (>300 students). 00:21:01.000 --> 00:21:01.900 peer feedback gets 10% of total mark and helps support final assessment 00:21:18.000 --> 00:21:18.900 is this session being recorded and can we get copy of the presentations itself? 00:21:22.000 --> 00:21:22.900 I have found many students will only contribute if assessed with makrs 00:21:31.000 --> 00:21:31.900 @mike I am experimenting this for the first time this semester Thank you, very interesting talk 00:21:36.000 --> 00:21:36.900 👏 00:21:42.000 --> 00:21:42.900 @mike that is our experience too 00:21:43.000 --> 00:21:43.900 all good Joanna 00:21:44.000 --> 00:21:44.900 thanks James very interesting 00:21:49.000 --> 00:21:49.900 @Kershee, oh wow and I'm just talking 60 odd! 00:21:58.000 --> 00:21:58.900 How do you think, how the education limitations to the online environment could influence students' competencies in face-to-face interactions? 00:21:59.000 --> 00:21:59.900 Thanks James interesting points to consider 00:22:07.000 --> 00:22:07.900 @michelle ... I give marks based on both quality and quantity of peer feedback 00:22:17.000 --> 00:22:17.900 I ask students to tell me if they agreed or not with FB, and why, and what, if anything they incorporated in he assignment, including from DOING the FB themselves (has some marks in the assessment) 00:22:29.000 --> 00:22:29.900 @mike thank you. 00:22:30.000 --> 00:22:30.900 brings me to a very well written set of criteria and how we work with that 00:22:35.000 --> 00:22:35.900 @Mike @Monique @Joanna it depends on how the purpose of peer review has been set up. Engaging students in practicing how to peer assess and the purpose may encourage participation rather than relying on the 'carrot' of marks. 00:23:43.000 --> 00:23:43.900 Thanks for the link. 00:23:49.000 --> 00:23:49.900 @Laurine I agree that approach is something I have used. 00:24:04.000 --> 00:24:04.900 @ fiona ... I have found very able good students don't need mark carrot but many students only want qualification 00:24:45.000 --> 00:24:45.900 A huge challenge, reflecting on inclusive practice can be transformed for online delivery, especially for campus universities (like mine) 00:24:47.000 --> 00:24:47.900 @mike... yes.... 🙁 00:25:07.000 --> 00:25:07.900 @Laurine excellent point about self FB from students, may help them to focus more rather than maybe disagreeing with the tutor 00:26:18.000 --> 00:26:18.900 asking student to estimate their mark shows whether you need to give them feedback as if they haven't put any effort in and know this you don't need to tell them but if they think they have done well and they haven't then they need the feedback 00:26:21.000 --> 00:26:21.900 @mike - yes I can see what you are saying. Though on occasion good students in the early days does not really know how god they are - reviewing other work through peer assessment can support their developing reality. Peer assessment for the less able student assists them in seeing 'good' work. 00:26:36.000 --> 00:26:36.900 many people (sadly) still think that alternatives are 'dumbing down'..and making it 'too easy' 00:28:18.000 --> 00:28:18.900 @fiona, good point 00:28:41.000 --> 00:28:41.900 @Lauraine, it should be about accommodating a person's needs - time to process, does someone need to assist them at the time, are we able to make an assessment more vision or hearing friendly 00:29:12.000 --> 00:29:12.900 agreed... just hard to convince some...troglodtyes! 00:29:14.000 --> 00:29:14.900 students should not need to 'request' adjustments as then unfair as depends on their confidence ... why can't these adjustments be automatic based on their 'condition ' ? 00:29:29.000 --> 00:29:29.900 True @Monique - I am battling for visual clarity. But training is lagging behind too. 00:29:51.000 --> 00:29:51.900 some students do not wish to request adjustments/additional support. We need to keep such students in mind when designing assessment for inclusion 00:30:01.000 --> 00:30:01.900 @Mike -I agree that's the meaning of 'inclusive' 00:30:07.000 --> 00:30:07.900 @mike agree but this does assume the student is willing to declare the "condition" 00:30:18.000 --> 00:30:18.900 Surely inclusive assessment design would mean that adjustments aren't needed? 00:30:19.000 --> 00:30:19.900 having a range of assessment modalities (UDL style) covers a lot of the problems of individualising 00:30:31.000 --> 00:30:31.900 @maggie, snap! 00:31:17.000 --> 00:31:17.900 giving everyone a choice in assessments and giving lots of time for any assessment for everyone means you don't need to make any adjustments 00:31:29.000 --> 00:31:29.900 spot on! 00:31:44.000 --> 00:31:44.900 @ Michelle, if you do workshops on peer review, its better to spend that time training them to give good peer feedback and practising with exemplars and criteria etc. Then they can give peer feedback online over a period of time. This leads to a better process but you have to help them to be receptive to the feedback practices first. Many students have very negative beliefs about the efficacy of peer feedback. 00:31:47.000 --> 00:31:47.900 does this depend on what you are assessing 00:32:06.000 --> 00:32:06.900 @mike - I agree - thinking about the administrative nightmare .... 00:32:11.000 --> 00:32:11.900 and learning is ... hopefully.. enabled in the absence of anxiety re performance etc 00:32:36.000 --> 00:32:36.900 I think the peer feedback would need to be anonymous in case it's affected by existing relationships between the students 00:32:48.000 --> 00:32:48.900 @Monique the actual time frame of the study was about 8 weeks for one essay with multiple drafts. 00:32:52.000 --> 00:32:52.900 tech could help with that - anonymised commenting functions 00:33:00.000 --> 00:33:00.900 universal doesnt mean everyone's needs are covered to a tee - but certainly more people will be included and encounter less barriers. however one person's need could be the opposite of someone else, thus the 'whole of the needs' can be contradictory. i would say that individual adjustments will always be needed. 00:33:18.000 --> 00:33:18.900 @alice need for teacher to know name of student even if anonymous to other students 00:33:28.000 --> 00:33:28.900 yes agree @mike 00:34:26.000 --> 00:34:26.900 just waiting for the request to 'use a pen' for an e-exam. 00:34:27.000 --> 00:34:27.900 We're trying to move to a system where exams operate in a window rather than at a specific time. So students can choose the timing that suits them. 00:34:28.000 --> 00:34:28.900 @Alice I have quite a lot of data on this, and if the students are trained in peer feedback strategies their relationships actually improve over time. But the teacher has to work to create a receptive environment where everyone understands feedback is about helping each other not pointing out shortcomings. 00:35:10.000 --> 00:35:10.900 I agree @James but I have students who have fallen out on a personal level so it would be inappropriate to have them engage with each others' work! 00:35:19.000 --> 00:35:19.900 @James Wood, thanks for that, it does open up the idea for me. Compressed timeframes have such a huge impact and a sea change would be helpful.I have academic staff more worried about a similarity score so they struggle to get behind the idea of teaching academic structure and expression 00:35:33.000 --> 00:35:33.900 I've used similar methods at King's College London and in South Korea, and there have been no quarrels in the past 5 years. 00:35:45.000 --> 00:35:45.900 inclusive and accessible assessment design needs to be thought of carefully before students actually enrol on course 00:35:58.000 --> 00:35:58.900 @James, yes it's surprising how vicious some can be , and how thoughtful others... 00:36:05.000 --> 00:36:05.900 Do you have thoughts about inclusive assessment for students on clinical placements 00:36:10.000 --> 00:36:10.900 You can use the Workshop Tool in Moodle/Blackboard - it is anonymous - wouldn't that be a way to undertake the peer review? 00:36:49.000 --> 00:36:49.900 What about Peermark in Turnitin (we are limited in Moodle)? 00:36:58.000 --> 00:36:58.900 @Alice, some thought and care needs to be put into who is put in each group. I do offer some student input in this and use different methods. 00:37:14.000 --> 00:37:14.900 that's interesting @James - thanks 00:37:15.000 --> 00:37:15.900 @James I had recently done some research on this. It is very early in the process at the moment. The idea was to achieve what you mentioned about learning from the feedback process itself to improve their own work 00:38:07.000 --> 00:38:07.900 @MIchelle, that sounds like helping them to develop feedback literacy. 00:38:59.000 --> 00:38:59.900 personalised assessment is interesting concept - students should be able to demonstrate that they have met the learning outcomes - Flexibiltiy around how they do thiscan be built in. I fear that QA of this would be a sticking point 00:39:02.000 --> 00:39:02.900 @Mike I fond that putting them in groups of 3 initially is best. They then make their own additional groupings and got into groups of 4 with peers they found particularly helpful. 00:39:45.000 --> 00:39:45.900 @Mike they actually ask who else wants to be in a peer feedback group with them and form new alliances in addition to the original ones. 00:39:52.000 --> 00:39:52.900 @James yes, I think I am trying to achieve a few things. I guess that in a way I am trying to get them to understand the importance of feedback and apply this to all similar elements. I would love to extend this to be a repeated process but I think that time is against me 00:40:33.000 --> 00:40:33.900 @James I have tried both free choice and teacher arranged groups ... advantages and disadvanges in both approaches 00:42:00.000 --> 00:42:00.900 @Mike "Mike I have found that peer feedback only works if I give marks for it so contributes to module mark" - in 5 years of using these methods in my classes I found that convincing 'priming their receptivity to feedback' solves the issue. I tell them about the research that giving feedback leads to better attainment than receiving only. We also look at Zone of proximal development and how dialogic feedback can help to improve it. After that they are willing to try, then when they see results they start to get more into it. 00:42:08.000 --> 00:42:08.900 Thanks all - received your comments from Fabio. As Mathew says, we will still need some adjustments but the increasing number of students + increasing number of acknowledged disability also poses a workload concern (and can make educators' heads' spin). So hopefully inclusive assessment can help with the 'low hanging fruit' (which might actually be quite complex... but we can hope). 00:42:41.000 --> 00:42:41.900 @Joanna, yes! 00:42:48.000 --> 00:42:48.900 @Mike - I ask them out of 5 how much they want to engage with feedback. Then put the 5's together, 4's together, 3's together etc. Even the threes get into it though. 00:43:18.000 --> 00:43:18.900 @james, nice lateral thinking 00:43:49.000 --> 00:43:49.900 @James ... I teach engineering and computer science students many of whom would switch off if I started giving them lots of vygotsky educational theory !! 00:43:54.000 --> 00:43:54.900 @Laurine I've been tinkering with this over many years! a kind of design research method as well as engagement with the literature. 00:44:16.000 --> 00:44:16.900 yep 00:44:17.000 --> 00:44:17.900 And when I suggest flexibiility/choice, it should still be bounded to ensure Learning outcomes can be met 00:44:36.000 --> 00:44:36.900 @james which subjects are your students studying? 00:45:08.000 --> 00:45:08.900 @Mike - I introduce it briefly (as a video), and then ask them to reflect on what it might mean for their learning from feedback. I give some homework marks for that. It then becomes a forum discussion. 00:45:23.000 --> 00:45:23.900 @mike & @james - on the priming to feedback issue, we've found that relating it beyond the unit to their imagined future (and where/who they might receive fb from) also helps with receptivity 00:45:33.000 --> 00:45:33.900 @Mike - advanced academic writing - the research essay students from many fields. Lots of engineers too. 00:45:51.000 --> 00:45:51.900 @Joanna... yes, WIIFM is a powerful driver 00:46:14.000 --> 00:46:14.900 @James @Mike - this is something I currently practice with the year 1 students putting them in groups of 3 that I select purposely not in friendship groups - for a first attempt at peer assessment. Overtime students develop skills in peer feedback and then their own preferred groups for support. 00:46:15.000 --> 00:46:15.900 Joanna - absolutely - I think this is part of feedback literacy and 'understanding the feedback landscape' 00:46:28.000 --> 00:46:28.900 @James there needs coordination across a programme so students don't get same information in every module about peer feedback 00:46:33.000 --> 00:46:33.900 A lot of interesting projects being run in Ireland. The partnership established by institutions and keen dissemination of results are incredibly neat. 00:46:36.000 --> 00:46:36.900 @Fiona, yes, I think that works well. 00:47:39.000 --> 00:47:39.900 Thanks so much @Joanna. this last semester first time ever in my Higher ed teaching experience since '97 not a single extension request or special consideration with 63 students in 2 levels and 3 groups. ( adjustments were for time only) every deadline met online for a Spanish intermediate and advanced. included groups of 3 person online video presentation, peer marking and 3 x generous written peer feedback per presentation, allocated randomly in CANVAS (and online paired orals in Outlook Teams -students received a recording of the performance in chat). 00:47:42.000 --> 00:47:42.900 @mike - yes, Molloy Boud and Henderson (2020) suggest that this should be in the first year. I'd agree, but later reflection tasks to help them to recursively develop feedback literacy is important too. 00:47:58.000 --> 00:47:58.900 @Fiona ... it is great if students can lear/decide to ask peers for feedback without this being required 00:48:10.000 --> 00:48:10.900 @James how many other faculty were involved and do you think faculty need some introduction to their role in the feedback conversation to improve their feedback literacy skills? 00:48:45.000 --> 00:48:45.900 @judy love it feedback literacy skills! ❤ 00:49:11.000 --> 00:49:11.900 @Judy -1. this was insider research. So there is a risk of the novelty effect - but I have had consistent results in two continents and with multiple nationalities. 00:49:45.000 --> 00:49:45.900 @ana maria the format sounds achievable - from further research I'm doing now, students are keen to learn and be challenged in their learning. If the tasks are clear and there is feedback provided they appreciate this 00:50:05.000 --> 00:50:05.900 @Judy 2. Yes, teacher feedback literacy is really really important, actually the model is to contribute to teacher feedback literacy. But it needs to be turned into a training programme and tested in different contexts. 00:50:22.000 --> 00:50:22.900 @james how would you introduce this to faculty? 00:50:44.000 --> 00:50:44.900 Look at Carless and Winstone 2020 on teacher feedback literacy, I see my work as an extension of this for digital settings. 00:51:24.000 --> 00:51:24.900 @Judy good question. First introduce them to the academic evidence for engaging in digital teacher feedback literacy programme. 00:51:27.000 --> 00:51:27.900 @mike - it is possibly due to the initial peer feedback requirement which included the students being asked to reflect on the process not only of providing peer feedback but also on how they felt in receiving peer feedback - a 360 degree review. Some students then begin to see the aedvantages. 00:51:28.000 --> 00:51:28.900 @James great thanks....already an advocate and keen to spread the word! 00:51:43.000 --> 00:51:43.900 @Judy then examples of students engaging with feedback through the methods 00:52:22.000 --> 00:52:22.900 @James wood in my area we call it Language Assessment Literacy globally and it comprises the feedback literacy for teachers and learners 00:52:31.000 --> 00:52:31.900 @Judy, finally offering workshops and one to one help on using the technology (its very easy) in the right ways. See 'workplace learning' Eraut (2009). 00:53:20.000 --> 00:53:20.900 @Ana yes, teacher feedback literacy interacts with student feedback literacy. This is one one of my slides (Carless and Winstone 2020). 00:53:39.000 --> 00:53:39.900 https://instructionalmoves.gse.harvard.edu/using-peer-instruction-improve-student-learning 00:53:57.000 --> 00:53:57.900 @james also working in online and we use a coversheet for student self-assessment to guide the marker to areas they want feedback.....thanks for all the advice and links 00:54:02.000 --> 00:54:02.900 thanks, mike 00:54:22.000 --> 00:54:22.900 great thanks James 00:54:31.000 --> 00:54:31.900 There is lots of evidence that peer feedback improves student learning. But also that students don't engage well for the reasons I talked about. 00:54:46.000 --> 00:54:46.900 I hope the chat is saved for distribution? 00:55:01.000 --> 00:55:01.900 yes - will be published on the TA site 00:55:06.000 --> 00:55:06.900 @judy#2 we do that in the spanish class also. feedback loop or feedback dialogue 00:55:15.000 --> 00:55:15.900 I think crisis=opportunity is the lasting meme for 2020 00:55:25.000 --> 00:55:25.900 Love it Laurine 00:55:33.000 --> 00:55:33.900 @ana maria like Language Assessment Literacy too. 00:55:36.000 --> 00:55:36.900 @Judy - the coversheet is great for peer and teacher feedback. See Carless and Winstone 2019 for more about 'feedback requests'. This is thought to help students to see feedback as something they need to construct meaning from through social processes. 00:55:49.000 --> 00:55:49.900 https://edtl.blog/ 00:55:53.000 --> 00:55:53.900 live link 00:56:22.000 --> 00:56:22.900 @Ana - do you have any sources for language assessment literacy? Smith et al - introduced assessment literacy in 2013 - is it derived from this? 00:56:26.000 --> 00:56:26.900 thanks! 00:56:39.000 --> 00:56:39.900 we use feedback requests cover sheet as well - really successful and good feedback from students 00:56:44.000 --> 00:56:44.900 thank you! Thanks Suzanne and Rob 00:57:00.000 --> 00:57:00.900 Thanks very nice presentations 00:57:05.000 --> 00:57:05.900 @Janet 👍 00:57:33.000 --> 00:57:33.900 slides, recording and chat will be shared via the transforming assessment site. 00:57:48.000 --> 00:57:48.900 thanks, mathew 00:57:55.000 --> 00:57:55.900 @james Inbar Lourie or Tsagari or Poehne 00:58:03.000 --> 00:58:03.900 @james and others.....I work with CME in Dundee who were involved in this study...Barton, K. L., Schofield, S. J., McAleer, S., & Ajjawi, R. (2016). Translating evidence-based guidelines to improve feedback practices: the interACT case study. BMC medical education, 16(1), 53. 00:58:04.000 --> 00:58:04.900 Thanks to all presenters. I am looking forward to the full conference in 2021. 00:58:05.000 --> 00:58:05.900 thank you for the presentations - very timely and relevant. 00:58:14.000 --> 00:58:14.900 To all presenters and chair: Thank you 00:58:17.000 --> 00:58:17.900 How too encourage + engage academics reluctant to engage or frightened by digital ed? 00:58:19.000 --> 00:58:19.900 please fill our session feedback survey before you go! http://taw.fi/feedback 00:58:28.000 --> 00:58:28.900 yes, very stimulating! 00:58:30.000 --> 00:58:30.900 Thank you presenters 00:58:42.000 --> 00:58:42.900 @julian, yes indeed 00:58:45.000 --> 00:58:45.900 Thanks for the presentations! Very informative! 00:58:50.000 --> 00:58:50.900 please fill our session feedback survey before you go! taw.fi/feedback 00:58:59.000 --> 00:58:59.900 The community page of this website is not working - how do we join? https://edtl.blog/community/ 00:59:00.000 --> 00:59:00.900 Thanks so much goodnight from Melbourne 00:59:09.000 --> 00:59:09.900 @Ana thanks 00:59:17.000 --> 00:59:17.900 Many thanks to the presenters and also the informative comments in the chat 00:59:18.000 --> 00:59:18.900 Thanks all 00:59:19.000 --> 00:59:19.900 Thank you for your interesting presentations 00:59:25.000 --> 00:59:25.900 please fill our session feedback survey before you go! taw.fi/feedback 00:59:28.000 --> 00:59:28.900 Thank you and goodnight from University of Sydney @Judy, I haven't read that but I will take a look thankyou! 00:59:29.000 --> 00:59:29.900 Thank you all for such intensive and good sharing! and the great chat engagement 00:59:38.000 --> 00:59:38.900 Thank you very much! 00:59:54.000 --> 00:59:54.900 please fill our session feedback survey before you go! taw.fi/feedback 00:59:55.000 --> 00:59:55.900 If anyone has further questions feel free to get in touch jameswoodsnu@gmail.com 00:59:56.000 --> 00:59:56.900 Thanks 01:00:01.000 --> 01:00:01.900 Thanks all This was fantastic. Thank you all! Thank you.