Research Help: Generative AI & Its Use in Universities: AI Detection and Evaluation Tools

Evaluating Content Produced by generative AI tools

Just like any other information source, content generated from AI systems need to be evaluated for accuracy, bias, and validity.

The ROBOT Test (McGill University)
The ROBOT test suggests five criteria that can be used to evaluate content generated by AI. They are Reliability, Objective, Bias, Ownership, and Type.
VALID-AI (University of Toronto)
VALID-AI evaluates AI generated in the following topic areas: V -validate data, A - analyze algorithms, L - legal and ethical considerations, I - interpret how it works, D - diversity and bias, A - accuracy check, and I - are you sing AI tools in an ethical way.

AI Detection Tools

AI detector tools have been created to try to determine whether a work was created by AI. They review text to identify patterns and attributes generally associated with AI generated content. Factors reviewed include diction and low variance in sentence length. The validity and reliability of these tools are questionable and should be used with caution. They can often generate false positives and negatives. Below are some articles and reviews that address some of the concerns with AI detection tools

Coffey, Laura. "Professors Cautious of Tools to Detect AI-Generated Writing." Inside HigherEd, February 9, 2024. Link to article.
Clark, Elijah. "The 10 Greatest AI Content Detector Tools." Forbes ,December 14, 2023. Link to article.
Chaka, Chaka. "Detecting AI Content in Responses Generated by ChatGPT, YouChat, and Chatsonic: The Case of Five AI Content Detection Tools." Journal of Applied Learning & Teaching vol. 6 no. 2 (2023). Link to article

Concerns about AI Detectors

Source: Clark, H. (2023, Sept. 22). Why AI detectors are problematic and what to do instead. InfusedClassroom. https://www.hollyclark.org/2023/09/22/why-ai-detectors-are-problematic-and-what-to-do-instead/