WEBVTT

00:00:38.000 --> 00:00:38.900
<v Shaobo Huang>
Hello, everyone

00:02:06.000 --> 00:02:06.900
<v Laurine Hurley>
how good would that be?

00:02:14.000 --> 00:02:14.900
<v Dr Mojtaba Ammari>
Hi Shaobo 

00:02:24.000 --> 00:02:24.900
<v Shaobo Huang>
exactly

00:02:42.000 --> 00:02:42.900
<v Shaobo Huang>
Hi Mojtaba

00:03:32.000 --> 00:03:32.900
<v Blazenka Divjak>
strong vision

00:04:53.000 --> 00:04:53.900
<v Dr Mojtaba Ammari>
Categorizing the areas of improvements into ten themes is a very creative way to start with

00:05:51.000 --> 00:05:51.900
<v Laurine Hurley>
a variant on VU (Melbourne) block model.. makes sense that not all modules need the same time

00:06:21.000 --> 00:06:21.900
<v Blazenka Divjak>
It is like program generic learning outcomes?

00:06:40.000 --> 00:06:40.900
<v Shaobo Huang>
Agreed and they align well with the assessment system Sean will present today

00:07:34.000 --> 00:07:34.900
<v Shaobo Huang>
Hi Blazenka, we do have program learning outcomes generated from the themes. 

00:09:11.000 --> 00:09:11.900
<v Blazenka Divjak>
Yes, profession-specific as far as I understand. But there are also generic LOs and they look like your 10 categories. Sometimes it is just terminology...

00:09:50.000 --> 00:09:50.900
<v Blazenka Divjak>
I like you approach and just want to relate with my reference frame

00:11:30.000 --> 00:11:30.900
<v Dr Mojtaba Ammari>
Looks much better and more opportunity for some low or medium stakes if I understood the diagram

00:12:29.000 --> 00:12:29.900
<v Shaobo Huang>
That is true

00:12:42.000 --> 00:12:42.900
<v Dr Mojtaba Ammari>
👍 

00:14:34.000 --> 00:14:34.900
<v Dr Mojtaba Ammari>
replacing final exams with those low an medium stakes is excellent

00:14:44.000 --> 00:14:44.900
<v Mathew Hillier>
more on the UoS website https://engineering.usask.ca/programs/Academic/re-engineered.php

00:15:29.000 --> 00:15:29.900
<v Shaobo Huang>
Thank you, Mathew

00:17:02.000 --> 00:17:02.900
<v Laurine Hurley>
this would drive the writing of better LOs

00:17:17.000 --> 00:17:17.900
<v Laurine Hurley>
pillar #1

00:17:51.000 --> 00:17:51.900
<v Dr Mojtaba Ammari>
Absolutely  - and match them well with appropriate assessment tasks

00:18:26.000 --> 00:18:26.900
<v Laurine Hurley>
win-win!

00:18:28.000 --> 00:18:28.900
<v Shaobo Huang>
Yes, we designed assessment based on the LOs

00:18:39.000 --> 00:18:39.900
<v Dr Mojtaba Ammari>
👍 

00:19:12.000 --> 00:19:12.900
<v Dr Mojtaba Ammari>
This in turn, promote engagement and also progression rate from Y1 to Y2

00:20:02.000 --> 00:20:02.900
<v Blazenka Divjak>
Do you attach weights to each LOs based on some methods? Like MCDM?

00:20:14.000 --> 00:20:14.900
<v Blazenka Divjak>
That's how we do it.

00:20:50.000 --> 00:20:50.900
<v Dr Mojtaba Ammari>
looks like you have separated core and applied part for each one the LOs

00:21:41.000 --> 00:21:41.900
<v Shaobo Huang>
each LO has a weight and we defined the weight based on the importance of the LO 

00:21:49.000 --> 00:21:49.900
<v Blazenka Divjak>
How?

00:22:32.000 --> 00:22:32.900
<v Blazenka Divjak>
Based on which criteria? Method?

00:22:46.000 --> 00:22:46.900
<v Cesar O-S (Curtin)>
What happens if a student fails the pass/fail components?

00:23:01.000 --> 00:23:01.900
<v Dr Mojtaba Ammari>
CL1 (n1%), CL2 (n2%) , CL3(n3%) and the criteria comes from a rubric perhaps

00:23:11.000 --> 00:23:11.900
<v Dr Mojtaba Ammari>
rubric*

00:24:06.000 --> 00:24:06.900
<v Laurine Hurley>
the multiple chances is key... it reflects what learning actually IS

00:25:40.000 --> 00:25:40.900
<v Shaobo Huang>
Hi Blazenka, you made a good point. 

00:26:38.000 --> 00:26:38.900
<v Shaobo Huang>
The student won't fail a LO unless by the end of the module, s/he doesn't pass the threshold after multiple chances of assessment

00:27:25.000 --> 00:27:25.900
<v Blazenka Divjak>
We use group decision making based on multi criteria to estimate weights of LOs. 

00:27:47.000 --> 00:27:47.900
<v Shaobo Huang>
Laurine, I agree with you. CBA emphasises the learning progress

00:27:52.000 --> 00:27:52.900
<v Laurine Hurley>
again reflecting real life.. think of athletes qualifying for Olympics.  

00:28:50.000 --> 00:28:50.900
<v Blazenka Divjak>
I agree

00:29:28.000 --> 00:29:28.900
<v Shaobo Huang>
Blazenka, setting criteria for weighting LOs is a great method

00:29:33.000 --> 00:29:33.900
<v Dr Mojtaba Ammari>
CBA combines assessment with students learning journey- they learn by engaging to the assessment tasks

00:29:50.000 --> 00:29:50.900
<v Blazenka Divjak>
climbing ladders

00:29:57.000 --> 00:29:57.900
<v Dr Mojtaba Ammari>
yes

00:30:26.000 --> 00:30:26.900
<v Laurine Hurley>
yes!  

00:30:27.000 --> 00:30:27.900
<v Cesar O-S (Curtin)>
Do you have any pressure for students getting certain marks? What happens if all students are 50%'s?

00:30:50.000 --> 00:30:50.900
<v Shaobo Huang>
I will be back in a few

00:35:54.000 --> 00:35:54.900
<v Karen Vernon-Parry>
Do you issue a "map" of where/when each LO will be assessed?

00:36:20.000 --> 00:36:20.900
<v Dr Mojtaba Ammari>
exactly and do you have any plan to reduce the load on lower-level students?

00:36:30.000 --> 00:36:30.900
<v Shaobo Huang>
Sorry... baby cried

00:38:05.000 --> 00:38:05.900
<v Shaobo Huang>
back to Cesar's question. If all students are 50%s, it would indicate something went wrong. certain adjustments should be arranged before the next assessment of the LO

00:38:39.000 --> 00:38:39.900
<v Cesar O-S (Curtin)>
Thanks

00:38:54.000 --> 00:38:54.900
<v Shaobo Huang>
Karen, yes, the assessments were transparent to students. They knew which LO will be assessed 

00:39:19.000 --> 00:39:19.900
<v Shaobo Huang>
in which assignments and/or module tests

00:39:26.000 --> 00:39:26.900
<v Dr Mojtaba Ammari>
do they know when the LO is going to assess?

00:39:31.000 --> 00:39:31.900
<v Cesar O-S (Curtin)>
How many "high-level" coordinators are in the program? Somebody has to have all this in their heads!

00:40:13.000 --> 00:40:13.900
<v Shaobo Huang>
Mojtaba, for lower-level students, I recommended them focus on Type B questions 

00:40:36.000 --> 00:40:36.900
<v Amer (Macquarie Uni Sydney)>
Do they get grades or a threshold based mark ?

00:40:54.000 --> 00:40:54.900
<v Laurine Hurley>
it would be good to poll them again later in the program to see  - with hindsight - which assessment mode better prepared them for learning and working

00:41:14.000 --> 00:41:14.900
<v Amer (Macquarie Uni Sydney)>
Say, if an LO has 50% weight, how are the  students marked? 

00:41:35.000 --> 00:41:35.900
<v Laurine Hurley>
working, as in practical/placement

00:41:38.000 --> 00:41:38.900
<v mike>
Question: Will students find the change to a different approach in their yrs 2,3,4  difficult?

00:41:38.000 --> 00:41:38.900
<v Shaobo Huang>
do they know when the LO is going to assess?- Kind of. When the students receive an assignment, they know which LOs would be assessed. They were informed that each LO would be assessed 3+ times. 

00:41:55.000 --> 00:41:55.900
<v Amer (Macquarie Uni Sydney)>
Please explain the grading.

00:42:07.000 --> 00:42:07.900
<v Sarah Baillie>
Excellent, thank you. Is the curriculum also flipped? Apologies if I missed that?

00:43:40.000 --> 00:43:40.900
<v Cesar O-S (Curtin)>
What happens if a student gets full marks in the first module assessment. Can he not do the remaining 2?

00:43:41.000 --> 00:43:41.900
<v Shaobo Huang>
How many "high-level" coordinators are in the program? Somebody has to have all this in their heads!- There was one course-level coordinator and the coordinator and the instructors met before the semester started to set up the "plan"

00:44:06.000 --> 00:44:06.900
<v Cesar O-S (Curtin)>
Thank you Shaobo

00:44:20.000 --> 00:44:20.900
<v Shaobo Huang>
During the semester, the instructors would follow the plan in designing the assessments

00:45:07.000 --> 00:45:07.900
<v Shaobo Huang>
Do they get grades or a threshold based mark ?- They receive percentage grades. 

00:45:42.000 --> 00:45:42.900
<v Shaobo Huang>
And we need to use percentage grades to "average" the grades for a LO if the later performance was worse than earlier one

00:45:47.000 --> 00:45:47.900
<v Mathew Hillier>
As a comments: The TAFE (vet sector) system in Australia runs on CBA. Also - Moodle LMS have features to help with CBA where each activity can be tagged against the 'competency'.

00:46:15.000 --> 00:46:15.900
<v Laurine Hurley>
Mathew, how hard is this to do in Moodle?

00:46:17.000 --> 00:46:17.900
<v Amer (Macquarie Uni Sydney)>
So, they are marked out of 100?

00:46:39.000 --> 00:46:39.900
<v Amer (Macquarie Uni Sydney)>
and then re-weighted out of 50% if the LO weight is 50%?

00:47:20.000 --> 00:47:20.900
<v Mathew Hillier>
@ Laurine - set up needed. ie. define all the competencies in advance - just as sean explained. more up front work needed.

00:47:21.000 --> 00:47:21.900
<v Shaobo Huang>
it would be good to poll them again later in the program to see  - with hindsight - which assessment mode better prepared them for learning and working- Thank you for the suggestion, Laurine. We  did cohort assessment last summer to collect baseline data. We plan to do that again to see if this program help students gain long term learning

00:47:47.000 --> 00:47:47.900
<v Laurine Hurley>
I'd love to know the outcome, Shaobo

00:48:08.000 --> 00:48:08.900
<v Laurine Hurley>
@Mathew, thanks

00:48:23.000 --> 00:48:23.900
<v Mathew Hillier>
re Moodle and CBA (called CBE). https://docs.moodle.org/310/en/Competencies

00:48:28.000 --> 00:48:28.900
<v Laurine Hurley>
thanks

00:48:49.000 --> 00:48:49.900
<v Shaobo Huang>
Say, if an LO has 50% weight, how are the  students marked? - In one assessment, each LO was marked as usual and we track the percentage grades. The grades are weighted in the final grade calculation at the end of a module. 

00:50:24.000 --> 00:50:24.900
<v Mathew Hillier>
you probably need to start with the end game and then work backwards.

00:50:33.000 --> 00:50:33.900
<v Shaobo Huang>
Question: Will students find the change to a different approach in their yrs 2,3,4  difficult?- Sean is answering this question. 

00:50:55.000 --> 00:50:55.900
<v Shaobo Huang>
Please explain the grading.- Hi Amer, I am behind the questions as I couldn't type fast enough

00:51:13.000 --> 00:51:13.900
<v Laurine Hurley>
😩 

00:52:10.000 --> 00:52:10.900
<v Shaobo Huang>
Is the curriculum also flipped?- The instructor can implement flipped classroom with CBA. But this course was not taught as a flipped class. 

00:52:42.000 --> 00:52:42.900
<v Sarah Baillie>
👍 

00:52:43.000 --> 00:52:43.900
<v mike>
Question: Is it possible to automate your Type B 70% threshold assessments?

00:53:22.000 --> 00:53:22.900
<v Shaobo Huang>
What happens if a student gets full marks in the first module assessment. Can he not do the remaining 2?- No, the modules are assessed independently. but higher grades in module 1 can help raise the final grade from the course for a student

00:55:59.000 --> 00:55:59.900
<v Shaobo Huang>
So, they are marked out of 100? and then reweighted out of 50% if the LO weight is 50%? - Every time an LO may be marked out of a different max point, depending on the marking rubric the instructor designed. Then the grade for an LO was turned into a percentage number 

00:56:18.000 --> 00:56:18.900
<v Amer (Macquarie Uni Sydney)>
Thanks @Shaobo Huang 

00:58:22.000 --> 00:58:22.900
<v Shaobo Huang>
Is it possible to automate your Type B 70% threshold assessments?- depends on the nature of the course I think. For this course, we would like to and needed to assess the methods students used to solve a problem. So the type B questions in this pilot course were not automized. 

00:58:36.000 --> 00:58:36.900
<v Laurine Hurley>
thank you both!

00:59:34.000 --> 00:59:34.900
<v mike>
Better to turn off chat when you want people to concentrate and then turn it on specifically for interactions.

00:59:43.000 --> 00:59:43.900
<v Shaobo Huang>
But one example of Type B questions that can be automized can be concept inventories. However, the current concept inventory (multiple-choice) questions couldn't help us assess all of the LOs defined in this course. 

01:01:05.000 --> 01:01:05.900
<v Karen Vernon-Parry>
thanks - really interesting

01:01:14.000 --> 01:01:14.900
<v Shaobo Huang>
Better to turn off chat when you want people to concentrate and then turn it on specifically for interactions.- you are very correct. some students found the chat could be distracting. but there were so many aha moments that happened in the chat as well. 

01:01:30.000 --> 01:01:30.900
<v Amer (Macquarie Uni Sydney)>
Thanks both - v interesting and a game changer in higher ed in many ways

01:02:16.000 --> 01:02:16.900
<v Cesar O-S (Curtin)>
Thank you all. See you next time.

01:02:17.000 --> 01:02:17.900
<v Harvey Woolf>
Thank you.  

01:02:24.000 --> 01:02:24.900
<v Dr Mojtaba Ammari>
Excellent approach and presentation many thanks for sharing it with us. Thanks Sean, Shaobo and Matthew

01:02:27.000 --> 01:02:27.900
<v Fiona #2>
Thank you so much.  Take care out there (from NZ).

01:03:54.000 --> 01:03:54.900
<v Shaobo Huang>
I'd love to know the outcome- hi Laurine, the average grade of this course was 79 out of 100 (380 students) . the course that were taught in traditional way in 2019 was 65 out of 100 with 377 students. 

01:04:11.000 --> 01:04:11.900
<v Blazenka Divjak>
Thank you

01:04:17.000 --> 01:04:17.900
<v Sidney Martin>
great session!

01:04:29.000 --> 01:04:29.900
<v Dr Mojtaba Ammari>
Thanks