Examview Pro, Desire2Learn and Online Evaluation

Before I even begin to write this blog post I have to give a big thank you to a colleague of mine. This year I have had the pleasure of working in the same office area as Dean Hodder. When one innovative person has another innovative person next to them, what can be imagined and implemented can sometimes be just amazing. Dean has been my partner in crime in what I am about to write. Thank you Dean... and although we will be in constant contact virtually, I will miss our face to face collaboration. And on that note... Here goes

In our online teaching we have been facing two major issues... Validity and reliability of assessment items.

Far too often I am correcting work that seems to be completed by one student and replicated by others. Collaboration I encourage, copying is what I suspect is really happening.

More seriously and seen less frequently are test workings that also appear to be copied. With mobile technology and in the absence of a sharp eye, the sharing of answers in this day and age is quite simple.

The question for me was how to create assessment items that avoid these two scenarios so that validity and reliability of those items is secure. Here's the solution as I plan to implement it in the upcoming school year.

Publishers have created Examview Pro data banks that contain many different questions. The questions all contain a rationale... essentially a detailed solution. The really smart publishers have created these questions to be bimodal and dynamic. That way the questions can change with the click of a button and be either a selected response or a constructive response question. Our plan next year involves both using and creating these dynamic questions (with solutions) so that we can create a huge repository of question items that we can draw from for work samples and tests. It takes more time to create one dynamic question but a lot less time to create 200 items that assess the same curriculum outcome. Whether they are selected or constructive response items is irrelevant because the question type is bimodal.

So how does this fit into online learning? Desire2Learn? At first glance Examview Pro and Desire2Learn do not talk to one another. You can export as a zip file and upload to many different LMS providers... namely Blackboard, Angel and my favorite... WebCt. It would seem that if a file inside Examview Pro is exported as a file compatible with WebCt and uploaded to Desire2Learn, Desire2Learn will recognize the questions and display them properly. So... the questions will display inside Desire2Learn. However, to ensure validity and reliability I would like to create an online worksample and test where students receive unique questions... random questions. That functionality exists in Examview Pro but because it and Desire2Learn don't really talk to one another, creating random question sets to achieve validity and reliability appeared to be problematic. Here's our solution.

When the Examview Pro zip file that contains all the questions is uploaded, it can be uploaded as a question bank or a quiz within Desire2Learn. If the questions are uploaded to the question bank, a new quiz can then be created within Desire2Learn. Question sets for that quiz can be created where questions can be chosen randomly from the questions that you assign it from the question bank. Desire2Learn will then randomize the questions that students will see. If you can easily create numerous questions (with dynamic Examview Pro questions), the probability of the student seeing the same question is remote.

So what does this really mean for me, as an online teacher.

1) Students will receive unique tests and worksamples.

2) Selective response items on the test and worksample will be automatically graded. The item analysis that teachers often like to see on such items is automatically done inside Desire2Learn.

3) Because the rationale/solution is built for each question within Examview Pro, the answer key that students would want to see after the test is completed is automatically uploaded to Desire2Learn. It would be accessible as soon as students submit the test... their unique solution key. If a constructive response question is given, the teacher can go inside the students test and see the solution key for that test and grade the workings to the selective response questions that students scan and send.

Now for the fun part... Creating dynamic assessment items. I'll keep you updated.