Introduction
Generative AI tools such as ChatGPT are freely available and easy to use. Their use creates opportunities but also risks. Most organisations have put policies in place to provide guardrails and guidance for individual staff members using generative AI tools in their work, but in reality, compliance is difficult to track and enforce. At the same time, generative AI is becoming more and more prevalent in the form of standalone tools or is built into existing software applications that organisations already use. To balance governance efforts and the need for staff to become familiar with the use of generative AI, Mercy Corps decided to build two in-house chatbots.
Background
Mercy Corps was approaching the project as an opportunity for the organisation to learn about generative AI – how to use it safely and responsibly, how to build and manage generative AI based solutions, and how to assess the benefits of it.
Realising there was no way of avoiding or preventing the use of generative AI tools, the organisation wanted to provide a safe alternative to using freely available tools, while providing commensurate user experience. “Safe” in this context meant that it would be in line with Mercy Corps’ responsible data policy, that the risk of data leakage by accidentally incorporating personally identifiable information as input into a public chatbot would be reduced, and that the parameters of the chatbots would be set to limit or mitigate bias and “hallucinations” – or the fabrication of false information – by limiting the chatbots source material to that of the Mercy Corps Digital Library as the primary source of information.
AI development
The first chat bot was built at the request of Mercy Corp’s Monitoring, Evaluation and Learning (MEL) team. It serves the use case of a small team wanting to use generative AI to interact with a custom set of documents specific to their area. Synthesising large volumes of documents to see relationships in them and generate new insight had been a long-standing challenge for MEL teams which the project hoped to help address.
The second chatbot is aimed at the overall organisation and will serve as a “front-end” to Mercy Corps’ Digital Library, which contains over twenty thousand documents. While the library has a search function and structure today, the chatbot aims to offer additional functionality to make the content more easily accessible. Users can ask a question in natural language, covering both practical and administrative topics such as how to file expenses, as well as more strategic topics to help with program design, and define the type of output they would like – whether it’s an executive summary, and email, or something else. As most of the content of the library is in English language, the ability of the chatbot to understand requests posed in the user’s own language, and to respond in the user’s language, adds to the enhanced accessibility of the library contents.
For both chatbots, Mercy Corps is leveraging off-the-shelf large language models (LLMs) in their own Microsoft Azure environment. Retrieval Augmented Generation (RAG) has been activated so the chatbots can, when needed, also access public resources on the internet.
Operationalising AI
User feedback on the MEL chatbot was very positive – users reported between 40% and 60% increase in efficiency, which meant time saved to read and process multiple documents and generate text summaries from that. On top of that, users also reported that they were discovering new insights in the summaries provided by the chatbot.
The Digital Library chatbot is currently being tested by forty staff members from across Mercy Corps’ countries and functions, who are evaluating whether the responses are accurate and usable for the intended purpose. Additionally, the project team is conducting adversarial testing or “red teaming,” meaning they are trying to see to what extent they can make the chatbot provide wrong or inappropriate responses, which enables them to correct and fine tune the model. Users can also see which documents the chatbots is using the generate the output, which helps to validate the results and make corrections where needed. So far, recognising that while an LLM will not always reference the best source of information and not always be perfect in its output, testers see that the time saved by not having to search and read through multiple documents to extract relevant bits of information is substantial and allows the user to focus on validating and refining the response given by the chatbot.
As part of the overall project, Mercy Corps’ Data Protection and Privacy team has launched an "Ethical AI" workstream, which ensures the integration of personal data protection principles into any AI use. The team has conducted research on emerging AI regulations and distilled a set of recommendations from them. By merging these recommendations with Mercy Corps' humanitarian principles, the team has started to create processes such as ethical AI assessments. Shadrock Roberts says:
“The most exciting part of the process has been developing technology that helps people do their jobs. Working in Data Protection and Privacy can sometimes feel demoralising, because you’re often treated as a roadblock. However, our team is dedicated to creating tools and products that simplify data protection, and it is incredibly rewarding when these solutions are adopted by staff.”
The Data Protection and Privacy team is leading the chatbot projects in close collaboration with Mercy Corps’ IT teams, and with the help of external consultants to provide technical expertise to understand the options the technology offers and what the best infrastructure setup would be for Mercy Corps’ purposes.
Learnings
“I would say technically I've been surprised at how well these work”, says Shadrock Roberts about the LLM technology used. While adaptations through fine tuning and system prompting are still needed, the overall accuracy of the LLMs trained on the organisation’s own documents so far is seen as good. He does stress the importance of having experts in the respective fields, such as MEL, working closely with technology experts to get the chatbot output right.
Another key point is the importance of developing policies and tools for the safe and responsible use of AI tools alongside building the technology solution. That way, the organisation and its users are reassured that they can trust the tools, and when new AI solutions are requested by users, there is a standardised approach to evaluating those.
Roberts also mentions that organisations must think about how they are going to retain AI expertise in the organisation, given that this is a very sought-after capability in the market right now. Whether it is through building expertise in-house or contracting with service providers, to scale up and sustain AI across an organisation, a solid capacity foundation is needed.
Plans for the future
Mercy Corps is planning to roll out the Digital Library chatbot broadly across the organisation as a Beta version so that users can start utilising the solution and thus become familiar with how they can employ generative AI to assist them in their work. Initially English, French, and Spanish language interaction will be available. As the chatbot is run in-house on Mercy Corp’s documents, scaling it up to more users will increase infrastructure cost over time but with up to five thousand potential users, it will still be more cost efficient than acquiring licenses for publicly available solutions. With the roll out, the organisation is expecting to gain a better understanding on how users will interact with generative AI, and what other use cases there might be, eventually even for use in programmatic areas where tools might interact with individuals and communities that Mercy Corps serve.