AI for Evaluators: Opportunities and Risks
DOI:
https://doi.org/10.56645/jmde.v19i45.907Keywords:
evaluation, technology, artificial intelligence, evaluation practiceAbstract
We are all familiar with how computers and smartphones have transformed work and productivity. We now stand at a new threshold with another technology that is predicted to further transform the way we work: Artificial Intelligence (AI).
In this article, we hope to take a sober and practical look at AI from the perspective of professional evaluators. We examine how we might use it, how it might transform the nature of our tasks, what threats it might pose to our field, and what evaluators might do to protect themselves and our society from potential adverse effects of this emerging technology.
Downloads
References
Brants, T., Popat, A. C., Xu, P., Och, F. J., & Dean, J. (2007). Large language models in machine translation. Google, Inc. http://research.google/pubs/pub33278.pdf
Bruce, K. (2023, March 5). ChatGPT and evaluation: 3 key takeaways [Post]. LinkedIn. https://www.linkedin.com/pulse/chatgpt-evaluation-3-key-takeaways-kerry-bruce/
Cantor, M. (2023, April 7). This economist won every bet he made on the future. Then he tested ChatGPT. The Guardian. https://www.theguardian.com/technology/2023/apr/06/chatgpt-ai-bryan-caplan-interview
Castelvecchi, D. (2016). Can we open the black box of AI? Nature News, 538(7623), 20–23. DOI: https://doi.org/10.1038/538020a
Durán, J. M., & Jongsma, K. R. (2021). Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. Journal of Medical Ethics, 47(5), 329–335. DOI: https://doi.org/10.1136/medethics-2020-106820
Gabriel, I. (2020) Artificial intelligence, values, and alignment. Minds and Machines, 30, 411–437. https://doi.org/10.1007/s11023-020-09539-2 DOI: https://doi.org/10.1007/s11023-020-09539-2
High-Level Expert Group on Artificial Intelligence. (2019). A definition of AI: Main capabilities and disciplines. European Commission. https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=60651
Kan, M. (2023, March 2). ChatGPT is coming to an app near you: OpenAI launches API for its chatbot. PC Magazine. https://www.pcmag.com/news/chatgpt-is-coming-to-an-app-near-you-openai-launches-api-for-its-chatbot
Marcus, G. (2022, December 1). How come GPT can seem so brilliant one moment and so breathtakingly dumb the next? The Road to AI We Can Trust. https://garymarcus.substack.com/p/how-come-gpt-can-seem-so-brilliant?
Microsoft. (2023, March 16). Introducing Microsoft 365 CopilotA whole new way to work. Microsoft 365 Blog. https://www.microsoft.com/en-us/microsoft-365/blog/2023/03/16/introducing-microsoft-365-copilot-a-whole-new-way-to-work/
OpenAI. (2022a, September 28). DALL·E now available without waitlist. OpenAI Blog. https://openai.com/blog/dall-e-now-available-without-waitlist/
OpenAI. (2022b, November 30). Introducing ChatGPT. OpenAI Blog. https://openai.com/blog/chatgpt/
Patton, M. Q. (2011). Essentials of utilization-focused evaluation. SAGE Publications.
Pethokoukis, J. (2016, June 6). What the story of ATMs and bank tellers reveals about the ‘rise of the robots’ and jobs. AEIdeas. https://www.aei.org/economics/what-atms-bank-tellers-rise-robots-and-jobs/
Scriven, M. (2013). The foundation and future of evaluation. In S. I. Donaldson (Ed.), The future of evaluation in society: A tribute to Michael Scriven (pp. 11–44). Information Age Publishing.
Simon, N. (2023, April 15). ChatGPT in evaluationAn opportunity for greater creativity? University World News. https://www.universityworldnews.com/post.php?story=20230412111133714
Walker, L. (2023, March 28). Belgian man dies of suicide following exchanges with chatbot. The Brussels Times. https://www.brusselstimes.com/430098/belgian-man-commits-suicide-following-exchanges-with-chatgpt/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Aaron Wilson Kates, Kurt Wilson
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
