Critical Evaluation of AI Generated Results: The FLUF Test for Information Literacy

Outline

Session Outline: Implementing AI in Education with the FLUF Test

I. Introduction to AI in Education
*Overview of AI in Professional Practice
*Growing role of AI in teaching and learning
*Common concerns: Ethics and implementation
*The Shift in Information Literacy
*Traditional frameworks: CRAAP Test, SIFT Method, CARRDSS system
*Need for updated frameworks in the AI era

II. The FLUF Test Framework
*Introduction to the FLUF Test
*Purpose: Improve AI prompt creation and critique AI-generated results
*Applicability: Language models, images, video, music, and more
*Key Components of the FLUF Rubric
*Format: Structure and presentation of AI-generated content
*Language: Clarity, tone, and appropriateness of language used
*Usability: Relevance and usefulness of the generated content
*Fanfare: Avoiding unnecessary embellishment or over-complication and considering the audience

III. Application of the FLUF Test
*Identifying “FLUF” in AI Results
*Recognizing areas where AI content falls short in format, language, usability, or fanfare
*Improving Results with Reprompting
*Process of reprompting, regenerating, and refining AI output
*Case examples: Before-and-after comparisons of AI content

IV. Ethics and Critical Evaluation of AI Content
*Digital Citizenship and Ethical AI Use
*Addressing biases, assumptions, and perspectives in AI-generated content
*Ensuring inclusivity and minimizing bias
*Aligning with Information Literacy Standards
*Incorporating critical evaluation skills from CRAAP, SIFT, and CARRDSS
*Fostering digital literacy through the FLUF framework

V. Practical Demonstration
*Creating Effective AI Prompts
*Crafting prompts that minimize FLUF in various types of AI-generated content (LLMs, images, etc.)
*Using the FLUF Rubric to Evaluate AI Results
*Hands-on activity: Participants evaluate AI-generated outputs using the FLUF Test

VI. Reflection and Best Practices
*Adapting AI in Classroom Settings
*Strategies for teaching students ethical AI use and digital literacy
*Reprompt, Regenerate, Repeat
*Embracing an iterative process in AI prompt refinement

VII. Conclusion
*Takeaways for Educators
*The FLUF Test as a tool for prompt creation and AI critique
*Ethical implementation of AI to support student success and equity

VIII. Q&A and Next Steps
*Discussion on integrating AI frameworks in educational practice
*Resources for ongoing AI literacy development

Supporting research

Critical Evaluation of AI: FLUF Test (Parker, 2023)
The FLUF test is presented to encourage users to expand their media and information literacy, and extend their use of critical evaluation of all information - no matter how it is obtained.

Here are some of the foundational pieces for this work:

The 80/20 Rule (Pareto Principle, 1896)
*The balance between human insight and technological capability
*80% research and regeneration of online sources
*20% human critique, creativity, and culmination to create a final output

Information Literacy: Frameworks for Critical Evaluation of Online Resources
*CRAAP (Blakeslee, 2004) – currency, relevance, authority, accuracy, purpose
*CARRDSS (Valenza, 2004) – credibility, accuracy, reliability, relevance, date, sources, scope
*SIFT (Caulfield, 2019) – stop, investigate, find, trace
*5 Key Questions (Thorman & Jolls, 2003) – creator, techniques, perceptions, bias, purpose

ISTE Standards for Educators (2017): Citizen Standard

When considering AI policies, mentoring students in ethical and appropriate use of digital resources falls under the guidelines of digital citizenship. According to the ISTE Educator Standard for “Citizen”, “Educators inspire students to positively contribute to and responsibly participate in the digital world.” (International Society for Technology in Education [ISTE], 2017).

2.2a. Create experiences for learners to make positive, socially responsible contributions and exhibit empathetic behavior online that build relationships and community.

Indicator 2.2a provides guidance on how students should be interacting and socially responsible in their use of AI

2.3b. Establish a learning culture that promotes curiosity and critical examination of online resources, and fosters digital literacy and media fluency.

In 2.3b, we see the essence of critical evaluation. Traditionally, information literacy skills included critically evaluating online sources using protocols like SIFT (Caulfield, 2019); CARRDSS (Valenza, 2004); CRAAP (Blakeslee, 2004); and 5 Key Questions (Thoman & Jolls, 2003). We extend this to include FLUF (Parker, 2023) for critical evaluation of AI.

2.3c. Mentor students in safe, legal and ethical practices with digital tools and the protection of intellectual rights and property.

Indicator 2.3c directs educators on mentoring of students on ethical practices while using digital tools like AI, and reminds us to cite our use of AI as well as adhere to copyright, intellectual property, and fair use guidelines.

2.3d. Model and promote management of personal data and digital identity, and protect student data privacy.

Finally, in 2.3d the emphasis is on data privacy, digital identity, and personal data. This indicator causes users to pause when uploading research, sensitive information, or personal data into an AI generative too
​

​FLUF Test: A Framework for Critical Evaluation of Content Generated with Artificial Intelligence © 2023 by Dr. Jennifer L. Parker is licensed under CC BY-NC-SA 4.0