With the ascendant popularity of OpenAI's ChatGPT, Microsoft’s Copilot, Google's Gemini, and others, enthusiasm for the possibility of "Artificial Intelligence" has been riding higher than any time since the advent of ELIZA in the 1960s. Although these tools are reaching a state where they can be useful for a variety of tasks and may appear intelligent, they don’t have an understanding of the questions being asked or responses they provide. Applications such as ChatGPT, Copilot, and Gemini are a category of tools known as "LLMs" (short for "Large Language Models") or "generative AI", which we will refer to in this article. Although this article is written with the current generation of "artificial intelligence" in mind, the advice holds true for nearly any other tool or technology we're likely to encounter and so throughout we will refer to these and other related technologies generally as "tools".
Tools on the market today that are commonly referred to as “AI,” “Generative AI,” or “LLMs” (primarily transformer-based deep neural networks) lack a capacity for reasoning, critical reflection, or self-reflection in the ways we commonly associate with human intelligence. These tools receive an input or a request and produce a statistically likely output or response based on the data with which they have been trained. This in no way diminishes the utility of these tools, but does help us reframe them from humanlike intelligences to software tools operated on a commercial basis by a wide variety of organizations with their own distinct agendas. Like with any other emerging and exciting tool, it's important to consider a few key questions before using them:
- What will happen to any information I enter into this tool, and is it appropriate to share this information with this tool and its operators?
- In what ways might the material produced by this tool embed systemic biases, and how might the uncritical application of the results from the tool potentially reinforce personal and systemic biases?
- How will I critically assess the accuracy of the tool’s output, and does material produced by the tool require special attribution?
What happens to the information I enter into tools like ChatGPT, Copilot, or Gemini?
Unless specifically indicated otherwise, most of these tools take the information shared with them along with feedback from the users they interact with and integrate that material to further refine or train the LLM, which could appear in output presented to any future user of the tool. That means that in many cases it wouldn't be right to consider an interaction with an LLM to be a private interaction.
When you're not sure how an LLM might integrate the data provided to it, consider your interaction with it to be a public interaction: don't share any information you wouldn't put on a poster in a public building.
For a brief overview of what PSU considers public, confidential, or restricted data please consult PSU's Information Security Policy.
Is it appropriate to share information with LLMs or AI tools?
When working with private data about people, institutions, or which is otherwise confidential, it's very important to consider the rights of the affected parties and your obligations to them. Unless an agent of the university (such as an employee or contracted professional) is completely in control of the tool (whether it's an LLM or other software tool), it is likely that it is not appropriate to share restricted or confidential (non-public) data. For example, a personal ChatGPT, Copilot, or Gemini account would not be an appropriate place to ask the tool questions related to confidential or restricted information. When in doubt about a tool's suitability for use, consult with your TAG or the OIT Helpdesk.
For new technologies, PSU's Information Security Team is happy to provide advice to you on contract terms, data classification, and technical properties of any tool of interest – simply submit a request for IT Security Consultation.
In what ways might this tool embed systemic biases?
LLMs are trained on broad swaths of media produced by humans, including the best and worst output of humanity. Just like institutions can passively embed bias in their structures, so too can LLMs inadvertently embed and repeat biased views such as making unfair assumptions about the gender of a person based on their job role (along with a litany of other regressive views).
When the output of an LLM is used uncritically, you may pass these embedded biases along unintentionally and cause harms which you did not intend or which are inappropriate for the intended purpose.
How will I critically assess the output of this tool?
One of the most amazing abilities of LLMs is to recite facts and provide explanations in a tone which belies a high degree of expertise. While their accuracy can be stunningly impressive in some cases, their output is not always consistently reliable – just like a forum post you might find via a cursory Google search may not be reliable despite sounding authoritative. LLMs are often trained off these posts just as well as actual expert content without the wherewithal to distinguish the underlying facts.
When you enlist a tool on a task which could have profound implications such as generating boilerplate code, producing legal citations, or reviewing existing literature on a topic, consider carefully if you are sufficiently expert in the area to identify and correct any errors. If you are not, consider the tactics and other sources you might use to reduce the risk of using incorrect or misleading output.
When using any new tool, whether it’s branded as “AI” or not, it's important to consider the strengths, weaknesses, unique opportunities, and limitations of that tool. In this blog post we've examined a few of those limitations, particularly as they relate to our obligations to other humans. While this blog is not fit for a detailed examination of the emerging and complex field of ethics as they pertain to these tools, hopefully this brief overview provides some help in evoking some of the key issues which we hope our community will consider critically in their explorations of this exciting new technology.