Proctoring Solutions

What is it?

Proctoring in the classroom simply means observing students who are taking a test. But the University of Miami has online degree programs where students may never come to campus. So this presents us with a problem. The university needs a solution for it’s online courses and degree programs, but that solution must be useful for campus-based classes. This short report documents our ongoing efforts to evaluate several proctoring tools and to describe potential solutions, either human proctored, automated or some combination. 

As an example of proctoring, consider a class that has a mid-term exam. This situation means enforcing time limits, and the use of external resources other than the exam question form. Online assessment poses a particular challenge for proctoring as students are in different places, and may, or may not be taking the exam at the same time. Monitoring the students and their surrounding during a test requires a technical solution. In order to deter, prevent, and stop cheating from a distance, several approaches can be used: a human proctoring service approach, an automated limitation approach, and an automated surveillance approach.

The first approach, A human proctoring service approach, connects a human proctor to the student’s computer, webcam, speakers and microphone. The proctor communicates with the students and watches the student as he or she are taking the exam. The advantage of this human proctoring is that the student and the proctor can communicate and clarify any misunderstandings, and that a proctor may make judgement calls as they are monitoring the student. However, this may be a disadvantage as proctors may be reluctant to report a student.

The second automated approach, involves installing a software program on the student’s computer. The program works to limit the student use of the computer to only those functions allowed and need for taking the exam. The advantages of this approach is that it does not cost much and prevents the use of search engines, VOIP, IM, email and other commonly used communication and information tools. The disadvantages of this approach is that it may not limit direct human interaction and that the students may face technical problems when installing the proctoring program on their device.

The third approach, automated surveillance, uses web-based tools to automatically record video and audio from the student’s computer while they take an exam. These recordings are made available to the instructor for later review. The advantages of this approach is that it does not cost much and provides concrete records and proof of conduct (and misconduct). The disadvantages of this approach are that the instructor cannot intervene to stop cheating as it happens. This approach works to deter.

Who’s using it?

Higher Education  Faculty and students in higher education use online proctoring services as a “solution for ensuring the academic integrity of online exams,” or to give flexibility for the date, time, and place of exams. Academic Integrity is considered a core value to many online and face-to-face educational institutions, therefore many institutions have strict policies and procedures that reflect how their organization maintains the integrity of their exams. 

Similarly, K-12 Institutions use proctoring services for students in a variety of educational settings (public, private, magnet, homeschool, virtual school). 

Standardized exams/ Professional Certificates/ State Licenses  Professional organizations and training industries have also partnered with online proctoring services in order to bring testing for state-required licenses or expanding employee knowledge to the online setting. This is useful for employees who work from home or alternate sites.

Usage scenarios

One existing solution at the University of Miami  is the use of Respondus LockDown and Respondus Monitor which offers 200 seats per year at the request from faculty.  While this may seem like an adequate solution it require a faculty person to review video of each student as they are taking the exam. This could amount to hundreds of hours of video per class. At the time of this paper, there are no instances of faculty at the University of Miami applying an online proctoring solution for their courses. However, we have many online degree programs that require online exams. As the number of online tests is increasing, the university should consider the Academic Integrity of its online programs, courses and exams to establish policies for the online cheating, exams and identity verification.

Most institutions, especially those with online degree programs, provide a comprehensive proctored exam policy. For example, the Proctored Exam Guidelines by Troy Online | Alabama’s International University, which includes guidelines to assist faculty for when they should create online proctored exams and how to create their online exams. This approach assists faculty members in understanding when a proctored exam would be appropriate for their course, and also guides them through the step by step process on administrative items with their program, the exam settings within the Learning Management System (Blackboard), and how to successfully interact with students and proctoring services (ex: ProctorU).

Some institutions have less rigid policies, where the Academic Integrity policies are used to discourage cheating and for auditing purposes, rather than as a requirement for all students. For example, Florida Virtual School is an accredited online school where K-12 students can access fully online courses for credit toward graduation. As part of their aim to ensure the Academic Integrity of the courses, students can be chosen at random from each course and are required to take a fully online exam with an approved proctor. In addition, students that are chosen must successfully pass their proctored exam with a grade of 59.9% or higher in order to be eligible for course credit.

Faculty Scenario

This is a hypothetical scenario that one of our Instructional Designers experienced at another Florida state university. For the purposes of this example we will describe the the Instructor as (Dr. Smith). This could take place in an online or campus-based course. First Dr Smith posts some information in his syllabus about an online quiz. He described the scenario in class and tells his students that there will be an online proctored exam, and that they must have a webcam. He tells the students they must go to the ProctorU website and create an account, and install the necessary software. Next Dr. Smith calls ProctorU to inform them that there will be an exam, and he must tell them the time, date, course number, and section. The instructor then tells the students a phone number that the students can call. ProctorU personnel will then talk to the student to guide them through the process. There is a software installation which permits the student to talk to the proctor. They may also call a student support helpline. Once installed students must have a student ID available at the time of the exam and show that ID to the proctor (via their webcam). Once the ProctorU proctor verifies the student’s identity and that they are indeed the student on the ID. The proctor will ask the student to answer questions on screen that will also verify their identity. Then they must pan the camera around the environment to ensure their is no one else present in the room, and no phones, tablets or additional computers are present. Then the student will take the exam. The ProctorU proctor watches the student and they can also see the students screen during the exam. Recordings are made and stored. Dr Smith then will be contacted if a student has been observed cheating.

Why is it significant?

Academic Integrity software is a tool, that educators can use to enforce an Honor Code that all UM students must follow. Its use detects students who might be trying to cheat or plagiarize and deters other students from trying the same. It demonstrates that the University is intent on maintaining an environment of ethical behavior. This in turn helps the student, because students who cheat will miss out on the opportunity to gain knowledge needed to succeed in their careers. In this digital age where students find new ways to falsify their knowledge on graded work, these new program help the University maintain its academic integrity.

What are the downsides?

While it may be argued that proctoring solutions support student learning by deterring cheating, or even promote studying, every solution has its advantages and disadvantages. As an example ProctorU, probably the number one proctoring solution, requires a student to pay a proctoring fee per test. While universities may find this a useful solution, because there is no cost to the university students may feel this is an imposition. They could have many tests in a semester at $10-$12 per test, especially if degree were online. Other proctoring solutions also have their drawbacks.

Other solutions require a faculty person to review hours and hours of student video to determining if that student is cheating. While there are automated solutions, these aren’t perfect and require the software provider to monitor the internet and detect cheaters behavior on mobile devices like personal phones and tablets. There is no perfect solution and our faculty are generally unaware of all the types of cheating that can occur even in graduate level courses. If we are to maintain some level of integrity we should explore options and guide faculty.

Where is it going?

Online proctoring will continue to evolve as learning assessment strategies and technology evolves. Unfortunately, so will so will academic misconduct. However, the adoption of some learning assessment strategies might deter cheating

The development and application of emerging technologies. This, however, will not change the basic approaches to online proctoring and we could expect that proctoring  will be based on at least one of the three approaches mentioned in this document. That is, human proctoring may still be in use, although new more effective forms of automated and surveillance will also offered. Staying aware and implementing these technologies is not only the job of the faculty but also those supporting faculty.

What are the implications for teaching and learning?

Multiple choice tests offer a convenient means of testing student learning. Multiple choice exams have been around for a century and are probably the most often used form of testing in undergraduate learning environments. This is because of the need for undergraduates to learn the terminology of the field of study. This form of examination is probably not suggested at the graduate level although some faculty who teach undergraduate level courses are resistant to this idea.
Although Multiple choice exams are common, faculty need to be aware of the limitations of online quizzing. Multiple choice quiz questions are a way to assess learning, but quiz questions must be written in such a way that they promote higher levels of learning. Question should be written to have a learner synthesize or evaluate some data or situation to make an interpretation. However a faculty person considers a question with a right answer and then provides some distractor questions. Learners may not know the answer, but though the process of elimination to guess the right answer. Or worse yet simply guess correctly. This does not encourage subject matter learning and only encourages those who are good test takers.

Discussion and conclusions

Our current solution Respondus Monitor records webcam video of each students, but for a student to be caught cheating the faculty person must review hours and hours of video. This is especially problematic if there are hundreds of students in a course, which could amount to hundreds of hours of video per test. In addition to faculty and their issues with the current solution, institutions also have issues. If we were to adopt ProctorU for instance who would manage the helpdesk, or interact with ProctorU when communications issues arose? Typically this falls upon a team like our learning platforms team, but they already have plenty of support issues to contend with, and may not have the necessary bandwidth to support this solution. So should it be a student helpdesk duty? All of these question must be considered.

In addition to logistical issues, legal and ethical issues must also be handled. If for instance a student is caught cheating online, we will need to extend our existing student rights and responsibilities policies to handle online cheating. Most state universities have had these problems solved for years, perhaps even a decade or more. Does that mean a state university has a higher level academic integrity? These are all concerns that must be addressed at the institution level. This does not even begin to consider the funding issues, or licensing problems that may arise. However it is very important that continue to we deter cheating. We can not say we do so in a meaningful way during an online exam, and that is a given even though we have multiple online degree programs.

Finally if a traditional test is used to test learning it may be defeated by cheating. Given an online testing environment or even fully-online degree program, faculty need to be assured that students do not have an unfair advantage over their peers. Historically faculty have been guided to place a low percentage of the final grade on multiple choice exams. Partially because they could be defeated by cheating but also because they do not typically test higher levels of learning.

However if there are tools and techniques to limit or deter cheating, shouldn’t they should be used?