A Quantitative Analysis of Information Verification Habits and Cross-Checking Behaviors Toward AI Hallucinations Among College Students
Article Sidebar
Main Article Content
This study investigated the information verification habits and cross-checking behaviors of Computer Studies students when encountering Generative Artificial Intelligence (AI) hallucinations. The primary objective was to determine how students' demographic and AI usage profiles influence their active verification methods, error detection success rates, and perceived reliability of AI tools. A quantitative descriptive-correlational design was adopted, utilizing a validated, scenario-based digital survey deployed via Google Forms to 200 Bachelor of Science in Information Technology (BSIT) and Bachelor of Science in Computer Science (BSCS) students at Quezon City University. The findings revealed that 84.5% of the respondents were highly active daily or weekly users of generative AI, primarily utilizing general web chatbots (58.5%) and dedicated Integrated Development Environment (IDE) assistants (41.5%) for debugging (45.5%) and code generation (33.5%). Students demonstrated a structured baseline of verification rigor (Composite Mean = 3.13, Agree), heavily prioritizing external validation such as manually confirming the existence of new libraries (Weighted Mean = 3.23) and cross-referencing official documentation (Weighted Mean = 3.20) rather than trusting the AI to self-verify its own outputs (Weighted Mean = 2.94). However, a significant cognitive vulnerability to automation bias was established, as students explicitly agreed that convenience and speed outweigh the risks of incorrect syntax (Weighted Mean = 3.11) and falsely perceived models as possessing deep contextual understanding (Weighted Mean = 3.20). Furthermore, a statistically significant negative correlation was found between the primary tool utilized and verification habits (r = -0.177, p = 0.012), proving that seamless inline code generation within dedicated IDE assistants suppresses manual auditing. Conversely, verification rigor remained entirely uniform across all academic year levels (F(3, 196) = 1.35, p = 0.261), overall usage frequencies (r = 0.053, p = 0.453), and task complexities (r = -0.097, p = 0.170). Ultimately, the study concludes that cognitive over-reliance is driven primarily by interface delivery friction rather than student seniority, designating an urgent institutional need to transition away from restrictive policies and proactively embed formal "AI Auditing" instruction, mandatory external citation protocols, and foundational mechanics of Large Language Models directly into the core programming curriculum.
Downloads
References
An exploration of online learning habits and academic productivity among BSIT students. (2025). International Journal of Research and Scientific Innovation (IJRSI), 12(10), 3530–3544. https://rsisinternational.org/journals/ijrsi/uploads/vol-iss-pg3530-3544-202510_pdf.pdf
Babbie, E. R. (2020). The practice of social research (15th ed.). Cengage Learning.
Bluman, A. G. (2018). Elementary statistics: A step-by-step approach (10th ed.). McGraw-Hill Education.
Calderon, J. F., & Gonzales, E. C. (1993). Methods of research and thesis writing. National Book Store.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.
Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed.). SAGE Publications.
Flick, U. (2018). An introduction to qualitative research (6th ed.). SAGE Publications.
Fowler, F. J. (2013). Survey research methods (5th ed.). SAGE Publications.
Göksel, N., & Akgül, M. (2021). Social media trust and verification habits of undergraduate students. ERIC Document Reproduction Service. https://files.eric.ed.gov/fulltext/EJ1395193.pdf
Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the behavioral sciences (10th ed.). Cengage Learning.
Horowitz, M. C., & Kahn, L. (2024). Bending the automation bias curve: A study of human and AI-based decision making in national security contexts. International Studies Quarterly, 68(2), Article sqae020. https://doi.org/10.1093/isq/sqae020
Information literacy: How students search for & verify health information. (2025). Honors in Communication and Language Studies, Bryant University. https://digitalcommons.bryant.edu/cgi/viewcontent.cgi?article=1001&context=honors_communicationlanguagestudies
Information verification among undergraduates. (2025). PhilArchive. https://philarchive.org/archive/DAGPSH
Ji, Z., et al. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38. https://doi.org/10.1145/3571730
Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396–403. https://doi.org/10.9734/BJAST/2015/14975
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 1–55.
Mehra, V., et al. (2025). Quantifying shortfalls in students’ AI-supported programming practices. International Information Systems Conference (IIS), 458–471. https://iacis.org/iis/2025/1_iis_2025_458-471.pdf
Navigating the infodemic: A qualitative study of university students’ misinformation vetting. (2024). BMC Public Health, 24(1), 1–12. https://pmc.ncbi.nlm.nih.gov/articles/PMC10829486/
Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.
Rowley, J. (2014). Designing and using research questionnaires. Management Research Review, 37(3), 308–330. https://doi.org/10.1108/MRR-02-2013-0027
Shoufan, A., & Esmaeil, A. (2026). AI hallucination from students' perspective: A thematic analysis. arXiv. https://doi.org/10.48550/arXiv.2602.17671
Soares, F. A., Franco, M. F., Scheid, E. J., & Granville, L. Z. (2025). Trust, but verify: An empirical evaluation of AI-generated code for SDN controllers. arXiv. https://doi.org/10.48550/arXiv.2510.20703
Taherdoost, H. (2016). Validity and reliability of the research instrument: How to test the validation of a questionnaire/survey in a research. International Journal of Academic Research in Management, 5(3), 28–36.
The hallucination effect: Correlating generative AI usage frequency with source verification habits among STEM students. (2024). IMCC Journal. https://myjournal.imcc.edu.ph/publication/the-hallucination-effect-correlating-generative-ai-usage-frequency-with-source-verific
UiTM Generation Z study. (2025). Universiti Teknologi MARA.
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in our journal are licensed under CC-BY 4.0, which permits authors to retain copyright of their work. This license allows for unrestricted use, sharing, and reproduction of the articles, provided that proper credit is given to the original authors and the source.