Where We’ve Been, Where We Are, and Where We’re Going with Artificial Intelligence in Online Proctoring

artificial intelligence, brain, think-3382507.jpg

This story was first published in EDUCAUSE’s Industry Insights column. Artificial intelligence (AI) is a buzzword these days in almost every business. Though it dates back to the 1950s, its application has reached a tipping point in the last decade and is fast progressing. AI has begun to pervade several aspects of higher education, although it has been led by a few early adopter faculty members, departments, and particular universities rather than by the entire institution. Here are a few examples:

  • To aid in the applicant evaluation process, the University of Arizona admissions office is deploying AI-driven keyword analysis tools.
  • Georgia State University used an AdmitHub chatbot to help students with typical financial aid and enrollment questions.
  • A Georgia Tech professor created his own virtual teaching assistant using IBM Watson technology.
  • A Texas A&M professor is using an AI-powered peer review tool to help cultivate learning engagement through writing and debate, according to a recent piece in the Chronicle of Higher Education.
  • In the same Chronicle article, the importance of AI in adaptive courseware, which is being deployed and evaluated by several colleges in order to achieve a more personalized, scalable learning method, is discussed.

Each of these examples of how AI has helped or enhanced a faculty member’s, department’s, or university’s systems and processes gives peers more confidence to consider how AI can help or enhance their own systems and processes. Each of these early adopters wanted to improve student results rather than replace the human element of teaching and learning. So, what exactly does artificial intelligence entail? AI is a computerized system or machine that is designed to mimic human intellect. One of the fundamentals that characterizes the discipline of machine learning AI is that it incorporates an element of “learning”—either “supervised,” where the training data contains a specified desired outcome, or “unsupervised,” where the training data does not contain a desired outcome. In late 2013, TESO began investigating if artificial intelligence could improve our online proctoring offerings. Almost all instances of cheating had a pattern of connected and recurrent activities, which we discovered. We determined that we could educate a machine to recognize these actions because we were already educating our live proctors to notice them. Pattern recognition is, after all, one of the most popular applications of artificial intelligence. We found three areas where artificial intelligence can assist us in detecting and preventing academic integrity violations:

  • Identity theft is a serious problem.
  • Behaviours of cheating
  • Theft of content
industry, web, network-6564530.jpg

We decided that spending time, money, and effort on an AI system that focuses on these areas would improve not only our clients’ outcomes but also our business operations. The purpose of incorporating AI into our proctoring platform was not to replace humans, but to improve the accuracy of both live and automated proctoring. Furthermore, AI can aid in the reduction of human errors, the detection of things that humans will inevitably miss, and the scalability of services. We applied the first of our AI events in the form of facial recognition and basic thresholds for audio and visual cues while designing an automated proctoring solution. A typical automated proctoring system is not new in terms of technology. More than four years have passed since the algorithms that power these systems were created. However, unless a developer makes a system-wide update, such algorithms remain static. The primary distinction between the old and new automated systems is that AI technology will continue to learn, adapt, and improve with each exam proctored. And AI isn’t simply used in our automated service; it’s also used in our live proctoring. The AI is being taught utilizing an aided learning methodology across both service tiers, with our own human proctors acting as the system’s teachers. The four steps of our supervised machine learning process are as follows:

  • Data is segmented and labeled by humans.
  • We produce a “event” in the algorithm once enough data has been segregated around one label.
  • To trigger the newly constructed event, we pass all current data through the algorithm.
  • Humans review the data to validate whether or not the event occurred, improving the system’s accuracy in detecting that specific event.
web, network, information technology-4861609.jpg

This method is straightforward at first glance, but it can rapidly become convoluted. Each action or pattern in this supervised learning model of machine learning requires a minimum of 20,000 data points to become a “event” in the system. Once the model has been trained on that event, it must be fed more and more “training data” in order to improve its accuracy around that one event. Cheating can manifest itself in a variety of ways, but let’s focus on one. Consider a test-eyes taker’s and head moving fast to the right, away from the screen. To train the system to recognize that one rapid move as an event, it would require 20,000 repetitions. To improve the accuracy of the system recognizing and flagging the behavior, an exponential number of further instances must occur. Multiply that number by the dozens of cheating actions we could educate the algorithm to recognize as “events.” As you can see, this procedure necessitates a significant amount of data. Where did all of that information originate from? We proctor over a million tests per year as the industry’s leading online proctoring firm. All of the exam data can be utilized to train the AI model once it has been anonymised for obvious privacy concerns. Only one cheating behavior is described in the example above. So, what about content theft and identity fraud? The training procedure in these domains is comparable, although the techniques are slightly different. Advanced facial recognition, object recognition, plane detection, speech-to-text, eye movement detection, and voice detection are just a few of the machine learning technologies that TESO is striving to integrate. Our AI model is still being trained, but as it gets smarter, it will be able to distinguish between an adult speaking, a child speaking, a baby crying, and a dog barking.

These are all things that people can readily perform, but the system is being taught which of these represent a threat to academic integrity and which may be categorized as innocuous. As new technology evolves and becomes available to the test-taker community, the route to developing a completely accurate AI model for online proctoring will continue to evolve. The rate of technical innovation has been increasing at an increasing rate, as reported in another recent article. We will be able to incorporate more technology into our own solutions and add it to our AI model when more technology is adopted in PCs, mobile devices, and wearables.

When we consider the future of higher education, we imagine a world where technology will surpass our physical capabilities. We imagine a day in which adaptive courseware and adaptive testing are so integrated with online proctoring that cheating will require severe means. Read our viewpoint on AI or contact us if you’d like to continue the conversation or learn more about how we’re using technology to improve our online proctoring solutions.

Leave a Comment

Your email address will not be published.