September 2023, Melbourne
Introducing Chuffed
In today’s world, we’re up against vast social and environmental challenges. Grassroots groups offering passion and adaptability are pivotal in tackling these issues. But these organisations need funding, and this is where Chuffed comes in.
Chuffed is a ground-breaking not-for-profit whose goal is to make charity fundraising simple. Through their innovative online crowdfunding platform, they've empowered hundreds of charities to change lives. At the time of writing Chuffed has helped rally more than $50m via 750k ethical donors.
The Problem
Chuffed's clients—especially budding charities—often grapple with writing compelling campaigns and emails. While training resources are available, they're not always the quickest fix. Chuffed’s founder, Prashan Paramanathan, hoped that an AI solution could simplify this process for its rapidly expanding user base.
The video below demonstrates the current Chuffed onboarding process where users write their campaign page content.
Chuffed onboarding process
Engagement With GDI
Prashan tapped into the expertise of the Good Data Institute for a cutting-edge solution. Through brainstorming workshops with Chuffed, we narrowed in on the idea of an AI co-writing tool. This tool would leverage Chuffed’s writing expertise to generate drafts, nudging users in the right direction.
Our team featured 3 of GDI’s volunteer data scientists: Dave, Eleanor, and Akriti. Akriti took the lead in managing the engagement, while Dave and Eleanor focussed on experimentation, development, and delivery.
Our Methodology
Our first task was to investigate off-the-shelf OpenAI models, and refresh our knowledge on ‘prompt engineering’. In writing this article I asked ChatGPT for a witty definition of prompt engineering. It replied, “Prompt engineering: Getting AI to spill the beans just right!” In slightly more detail, it’s the practice of carefully designing what you put into the model (prompts) to elicit the desired output. In our case, this turned out to be an interesting balancing act: we needed to provide enough detail to get the tone right, but not so much that the model started to hallucinate or replicate content verbatim. Through trial and error, we found prompts that produced consistently good results; no need to retrain a model.
Diving into the detail We had OpenAI’s gpt-3.5-turbo and davinci-003 models shortlisted for this task (GPT 4 was not available at the time) given their qualitative prowess over other Large Language Model (LLM) alternatives. For this use case gpt-3.5-turbo had the best results, using a combination of both ‘chat completions’ and ‘completions’. This was an engagement with limited time—retraining a model was out of the question. Our research thus focussed on prompt engineering to imbue Chuffed’s content-writing knowledge as ‘meta prompts’, and to guide the output into the required format as much as possible. We were startled to find that we only needed 374 characters in the meta prompt to go from questions about the charity to campaign page content that was at a par similar to those most successful on Chuffed’s platform. We didn’t need to submit a single real example as a meta prompt. For emails it took more, with our best results coming from a meta prompt of 2,133 characters. We experimented with the model’s API parameters to get the right balance of consistency and creativity. The ‘temperature’ parameter had the largest impact. At a setting of 0.2 we received content that was valid, but too bland. At a setting of 1.8 we received a cornucopia of bizarre hallucinations. We settled for a mix of 0.7-1.0 in our final application (the default is 1).
With a working model in hand and an understanding of Chuffed’s operations, we realised a fully integrated application was a tall order for the given timeframe. So, we pivoted, and decided to build a prototype web app which we’ve affectionately dubbed Chuffed-GPT.
The Solution
Chuffed-GPT is a configurable flask app designed to transform user inputs into engaging campaign pages and kick-off emails. Picture this: you're a charity trying to launch a new campaign. Instead of staring down a blank page, you answer straightforward questions like, “Who are the people you’re trying to help?” and “What are you asking contributors for?”. With the click of a button, the app produces a complete first draft of your campaign page via OpenAI’s magic.
Why the Q&A approach? It mirrors Chuffed's existing onboarding process (video above), making the transition a breeze. And once the content's whipped up users can review, refine, and rock on to the next step – writing that essential first email. Check it out below!
The solution built by GDI for Chuffed
Prashan advised he was very interested to experiment with this workflow in-house, and to test results for different series of questions. We externalised the meta-prompts and questions into simple text files for this purpose. Chuffed's team can tweak, rebuild, and see those changes come to life on the fly.
Demonstration of the configuration of Chuffed-GPT
Outcome
The engagement was successful!
We supplied Chuffed with:
A final report containing our analysis on model usage for their use case.
The complete Chuffed-GPT code repository with instructions on how to configure, use, and host the application.
Suggestions on how the project can be extended in the future.
With the delivered prototype, we’ve provided Chuffed the first version of their AI co-writing tool. The standalone app serves as a nifty sandbox, allowing for rapid user testing and iteration before the solution is integrated into their existing tech stack. Prashan, Chuffed’s founder, has already envisioned other internal uses for the tool.
Stay tuned! We're excited about the potential for a second phase of the project.
Author:
Dave Cole (GDI Volunteer)
About GDI:
The Good Data Institute (established 2019) is a registered not-for-profit organisation (ABN: 6664087941) that aims to give not-for-profits access to data analytics (D&A) support & tools. Our mission is to be the bridge between the not-for-profit world and the world of data analytics practitioners wishing to do social good. Using D&A, we identify, share, and help implement the most effective means for growing NFP people, organisations, and their impact.