Running Online Exams

We are now doing 100% online teaching, which means the exam is also done online. Here are some things that I have done in setting/administering the exams (hopefully this will be useful to some of you). My objective is for all students to have a qualitatively similar individual exam with minimal incentives to help one another.

  1. Question Randomizing. I use Qualtrics to set the questions. My main objective is to test all students on the same set of N concepts. For each concept, I will have M different but interchangable questions. My assumption is that any students who know a concept will be able to answer any of the M questions. The exam is randomized such that (i) the question sequence varies across students (i.e., students are tested on different concepts at different point during the exam) and (ii) the question for each concept varies across students (i.e., students are tested by different questions for each concept). If N and M are large (and the exam duration is relatively short), then the challenge for students to help one another during the exam is going to be greater.
  2. Forward Move Only. I set the questions just that students answer one question at a time, and they have to answer a question before moving on to the next. They cannot go back to answer a previous question.
  3. No Incentives To Be Helpful. Because of the question randomizing, I told students that they should turn of their mobile phone and log off from all social media and commuincation accounts (whatsapp/FB/IG/email/etc.). This is because the short exam duration (30 minutes) and format (randomized questions + move move only) mean they have little time and incentives to give help to others; so getting off their phones and accounts take away any pressure for them to give help. And because other students may not be able to help them due to the exam format, they should not waste time asking or waiting for help.
  4. Open-Everything. The exam is open book and students can have access to any course materials, online resources, etc. The only thing they cannot do is to have live support from their friends, parents, spouses, children, etc.
  5. Clear EoE. If your exam questions are not numbered in the system (like in my case, due to randomization), make sure you have a very clear “end of exam” page and communicate this to the students before the exam begins so that they know what to expect. I tell students that they basically have to keep answering any questions that they see until there is no more question to answer. My “end of exam” page shows

You are done with the exam, but…

– Do not turn off the webcam or log off until the instructor says you can.
– Remain seated in front of your webcam, fold your arms, keep your hands away from any electronic devices (phones, computer, etc.)
– Do not communicate any aspects of this exam with other students until the instructor sends out an “all-clear” annoucement.

If you have any good tips about running online exams, do share.

Shocking! What I teach actually works!

Received a feedback from a student who took my Introduce to IS course in Spring 2018:

Given that this course is offered to pre-major freshmen/sophomores, I seldom get to hear from them after the course. But it always feels great when students come back to tell you that they could apply what they learnt in their work subsequently. For me, this course outcome is more meaningful than students’ grades or satisfaction. Yet, as I told the student above, it is more than just my teaching or the course content. You need students with the right attitude and who want to be inspired. We don’t often get such students, but when we have them in our classroom, it makes a world of difference to our teaching and the learning environment.

Nonetheless, I still take whatever credit that is given to me for the student’s outstanding performance at the bank 🙂

Lovely start to 2018

Received a new year greeting from a former student today:

Happy new year to you! Long time have not contacted you but I really want to thank you again at this time for the IS intro courses you brought to me! I am now doing a tech-startup intern which is platform-oriented, and I found the concept I learnt from the course really helps me understand the work and think about the business logically. I am now doing the major of dual degree in CS and business and want to explore the tech industry more. Hope I have the chance in the future to hear great insights from you like I did in the 2010 courses! Thank you so much! Wish you a happy and enriching 2018!

It is always nice to hear from students that what they learnt in my courses actually help them in their work and cause them to explore more. Coincidentally, I was in the middle of preparing and revising the syllabus for the said Intro. to IS course for the coming semester when I received the email. For the last few years, my colleagues and I have been teaching this course in a “reformed” way: instead of simply teaching off an introductory textbook, we try to make the course more relevant and applicable by covering certain key IS topics and ideas in today’s economy (e.g., platform economics, social media strategy, big data analytics, etc.) as well as traditional core IS concepts. However, some students feel that the way  this course has been taught does not help them in the other IS courses that they take subsequently. A senior colleague heard such feedback and suggested that we should perhaps teach the Intro. to IS course in a more “traditional” manner — just like how introductory courses for marketing and accounting are being structured, where the focus is on the fundamentals, basic principles, etc. in the respective disciplines. Over the past few weeks, I have been trying to decide whether/how to revert the course to the way it was taught in the old days and which topics to drop. Well, the student’s greeting just made the decisions clearer.

 

 

It is not all rosy

A few weeks ago, I noticed that the feedback that I shared on this website about my undergraduate course Introduction to Information Systems (ISOM2010) is mainly positive. I feel I should also show other types of feedback that I have received so as to give a more accurate picture. However, students only email or tell me about their course experience when they have nice things to say (understandably so). Although students had given some not-too-positive feedback in the course evaluation previously, those comments were usually not very “juicy” in the sense they were mainly about the heavy workload, etc., which I actually show to current students in class so as to help them manage their expectation. So I thought I should wait for the latest evaluations to come in and see if I can put together more “dark side” of my teaching before making a post. Right from the first lecture in this semester, I could sense that the vibe among certain students in one of the sections (L2) wasn’t too positive.

Well, I received the evaluation reports this morning and I must say that this year’s students, especially those from L2, did not disappoint. A few students wrote passionately about their horrible experience with my teaching. In fact, I was so bad that I achieved a personal career low instructor rating. A pretty humbling experience, I must say. Below, I provide this semester’s evaluation reports in their entirety. This information will help future students who are enrolled in my ISOM2010 course (or thinking to do so) to know what to expect and perhaps run away (i.e., ditch the course) while they can. For good measure, I’m including the evaluation report for the MBA course on digital marketing (ISOM5390) that I taught in this semester too.

[Quick tip for navigating the reports: Q11 is the instructor rating score; Q14 is about the weak points of the course/instructor.]

ISOM2010 L1:

Download (PDF, 130KB)

ISOM2010 L2:

Download (PDF, 134KB)

ISOM2010 L3:

Download (PDF, 129KB)

ISOM5390:

Download (PDF, 150KB)

Early feedback for ISOM2010

Although there are two more lectures to go for the undergraduate course Introduction to Information Systems  (ISOM2010), I have received a student’s feedback about the course.

It is my pleasure to be in isom 2010 course this semester which refresh my mind a lot about the newest trend in IS as well as the details and situations behind them. Before taking this course, I know little about the topics like big data and analytics, however, this course gives me more insights and understanding about them and motivates me to self-learn and read more readings related to machine learning and related stuff. The presentation has also reinforced my understanding about IS concepts as well. The whole course is well-designed, I really love this course a lot and it also triggers me to take ISOM as my major in the future!

The pleasure is all mine. It is a bit early to receive course feedback, but I’m not complaining, especially when the feedback is positive 🙂 It’s always nice to know the planned course activities/readings/topics have some intended effects on some students.

Digital and Social Media Strategy for MBAs – 2nd run

Just wrapped up the MBA course on digital and social media strategy (ISOM5390). Once again, I’m fortunate to have a great group of eager and motivated students to keep me on my toes.

Happy ending with the 2nd batch!

After the last class, I received feedback from some of the students about the course:

Since this class material was new to me, this was not easy task for me. However, I will be involved in the marketing project after  MBA, so this class was practical and helpful for me to understand how to think about marketing. Thank you for your class. If I can, I want to take your class again.

I want to thank you again for your class, it is beyond my expectation. As I mentioned in last class, this course brings fresh knowledge about social media and those guest speakers’ insightful sharing did put us in a real business world, so it was a very pleasant and achieving learning process. Hope more of other future MBA students can benefit from your course!

It is one of the most practical class I had so far. Will definitely recommend to other MBAP (especially marketers). THANK YOU again.

Glad to know that some students found the course valuable.

Webinar — been there, done that….

Offered a webinar on “Platform Disruptions” for the HKUST Business School’s Executive Education Office. It was a pretty fun and eye-opening experience for me. The technical setup was pretty simple; in fact, the technology for conducting such remote teaching seems pretty matured.

Behind the scene -- all the equipment to produce the webinar...

Behind the scene — all the equipment to produce the webinar…

There were about 20+ participants who logged in during their lunch break for 30-min talk. The main challenge I found is the lack of direct interaction between the participants and I. Although you can run simple polls or MCQs during the webinar to stimulate participation and engagement, these just aren’t the same as the interactions in a traditional classroom setting. Would love to challenge myself to give another webinar more creatively.

Feedback on new MBA course

I developed and delivered a new course on Digital and Social Media Marketing for the MBAs in the last semester. And I was fortunate to have a fantastic group of students for the first run of the course.

The inaugural MBA class on Digital and Social Media Marketing

I’m quite happy with how it turned out and, based on the the course evaluations, I think the students had a positive experience too. Some of the students’ feedback:

A lot of very insightful digital marketing strategies were shared and discussed in the lecture.

It taught me a new thing because I don’t know anything about digital marketing at the first time. On top of that, the class challenges your logic as the prof keep answering us with new questions that enrich the class discussion.

Current day digital marketing trends shown in the class. Great teaching skills by professor

Looking forward to the next run of the course!

Executive Ed @ CITIC Pacific

I had the privilege to conduct a workshop on “Platform Strategy” for senior executive from CITIC Pacific. It is always exciting to share insights into my favourite topic with others. A fun exercise during the workshop was to re-model one of CITIC’s existing businesses into a multi-sided platform.

One thing that I enjoy about Executive Education is the challenge to apply concepts into specific contexts that matter to the executives. This really pushes me to think very carefully in planning and delivering the course.