Caveats
Inform users about shortcomings or risk in the model or the technology overall
Overview:
How do we encourage responsible use of an emerging technology?
Users span technical backgrounds. Familiarity with the limits of generativeAI, ethical concerns, or how to responsibly interpret results will vary widely. Companies are looking for simple ways to shield themselves from liability and push responsibility onto users.
For less technical or younger users, these caveats are simple ways to signal limitations to the product. In that regard, it‘s not much different than a warning label on your hair dryer telling you not to use it in the bath.
At the same time, these models are complex and rapidly evolving. It may be sufficient to smack a caveat on as a first step. Generally though, companies will need to find better ways to help users navigate the nuances of these models if they want to provide them to a mainstream audience.
Benefits:
Anti-patterns:
Doesn't protect companies from hallucinations or weird behavior
As models are tuned, there are still many unknown unknowns that could impact the experience a user has with the technology. AirCanada recently had to provide a refund to a customer, despite it being against their policy, due to their chatbot providing inaccurate information. A caveat is not sufficient to remove responsibility.
Companies could use these to skirt ethical responsiblity
In the race to get AI products to market, companies are not prioritizing crafting well-constructed experiences. Chatbots that quickly turn racist, image generators that ignore entire races, aggregated sources with inaccurate data–these are recent examples of this technology gone awry. Caveats are a bandaid, and not a suitable substitute to creating thoughtful experiences. There is an ethical imperative for companies to think about how to ensure responsible use and interpretation of the content they provide
