Introduction
In early 2019, GDI began its journey as a provider of pro-bono data and analytics consulting services to the not-for-profit (NFP) sector, by connecting mission-driven organisations with data and analytics (D&A) professionals.
The GDI team quickly learned how nuanced the needs of NFP organisations are, and that all are on a very different stage of their D&A journey.
To understand how data professionals could most effectively help these organisations to achieve their full potential, GDI worked with 180 Degrees Consulting and conducted a survey of leading NFPs across Australia and New Zealand, supplemented by in-depth interviews and learnings from partnerships to date.
In this paper, we begin by analysing the survey to characterise the current state of D&A in the Australian NFP sector. We look at how data is collected, how it is processed and analysed, and the role that D&A plays in day-to-day operations. We then discuss the key D&A challenges currently faced by NFPs, and explore additional benefits that enhanced D&A capabilities are able to offer. Lastly, we present the journey of an NFP that we recently partnered with to implement a robust D&A strategy, outlining the key steps involved in the strategy design and implementation process.
GDI surveyed 27 NFP organisations operating in Australia, fielded through GDI’s existing partnerships, their immediate connections, and the network of 180 Degrees Consulting. The respondent organisations ranged in size, from volunteer-run to more than 100 paid staff, and focus, with missions including youth support, first nations communities, refugees, and mobility impairment. The survey consisted of 20 questions split across four sections, and the typical response was 20 minutes. We conducted seven half-hour follow-up interviews to get further insight into individual responses.
Section 1: The Current State Of D&A In NFPs
What Are The Primary Functions Of D&A In NFPs Today?
Our survey found that the most common purpose of Data and Analytics (D&A) within surveyed NFPs is to generate statistics that articulate the organisation’s impact, to track the progress of projects under operation, and to generate internally and externally-facing reports (Figure 1).
What Does D&A Look Like At NFPs Today?
D&A is most frequently used by NFPs to generate impact statistics and visualisations. Through our interviews we found that there is a growing appetite from donors for information and visualisations that quantify the impact of their chosen project. Charts that may have been more common in annual reports are now increasingly present in organisation’s regular marketing and communications updates, as well as in their social media channels. Metrics such as ‘meals delivered’, ‘courses taught’, and ‘families supported’, are being shared to drive engagement within donor groups.
We found the second key function of D&A to be project tracking. Detailed and reliable data from live projects helps teams to make informed decisions about program strategy and resource allocation.
Lastly, 48% of NFPs indicated report writing to be a primary purpose of D&A in their organisation. We found that most of the reports written by NFPs are externally facing, typically either for donor communications, regulatory and financial reporting, or to support grant applications. The internally facing reports are often to assist with longer term operational planning. Interviewees expect the role of D&A in report writing to increase in importance over the next few years, as government has been placing a higher burden on NFPs to show the impact of their programs to receive funding.
We found that most data collection is conducted by NFP employees, rather than automated tools; less than a fifth of organisations cited data collection tools as their primary data collection method.
The manual nature of data collection today is typically a result of either a lack of expertise, or of limited resources to dedicate to automating processes. Less than 20% of respondents employed a data or information specialist within their organisation (Figure 2).
The organisations that did have such roles were usually larger and more established, with either more than 20 paid staff or a volunteer network of more than 100 people. Organisations of this size are in the minority of Australia’s NFPs.
We found that a lack of expertise and insufficient resourcing often shifts the responsibility of preparing reports onto members of senior management. For most organisations surveyed, the primary collectors of data in their organisation were paid staff (Figure 3), and in our interviews we found that these staff were typically at the director level.
The lack of D&A resourcing is reflected in the analytical tools found to be used within NFPs. While D&A professionals will typically tools and codes such as Tableau, Python, and R, we found that NFPs are more commonly using Microsoft Excel, followed by a series of more niche, task-specific programs (e.g., imis, Information Leader).
What Are The Key D&A Challenges Faced By NFPs Today?
Our research highlighted two problems commonly faced by NFPs when performing D&A tasks: effectively measuring the achievement of their programs, and losing time to report generation.
We found many NFPs struggle to use their data to produce a clear view of how well they are achieving their stated purpose. This is not a new issue - Australian Institute of Company Directors presented it in their 2014 survey, when they showed that 50% of NFPs feel their achievement measurement to be ineffective, and 61% wanted more information on their achievement of purpose.
One interviewee, a chairperson of a large international NFP, found it difficult to drive the board to create reliable measures of performance, largely because much of the data collected by their program managers was qualitative and difficult to aggregate.
This problem is partially explained by the nature of NFPs, which unlike traditional corporations, do not rely on conventional financial metrics to track their performance. However, program managers at NFPs would have a wealth of relevant program data at their fingertips, ripe for performance measurement, with the right D&A functions in place.
The second issue we noticed is that a significant portion of leadership time is sunk into producing reports using unreproducible, ad hoc processes.
Our research found that report writing is a crucial function for many NFPs in Australia and NZ, with interviewees suggesting that it can often represent 15-20% of a program manager’s working week. Interviewees stated a large portion of the report writing task is spent gathering and synthesising data from projects and internal operations, and our survey showed that this can take up to 9 hours per week. As we will discuss, much of this time can be freed from report generation through the implementation of D&A practices. Change can be driven from the top-down; survey respondents indicated that senior management would benefit the most from D&A capabilities.
Section 2: The Benefits Of Enhanced D&A Capabilities
What Benefits Does Enhanced D&A Have To Offer?
A 2019 survey by Kearney showed that corporations with leading D&A strategies generate as much as 83% more profit than those lagging. The nature of D&A in the NFP world is less advanced, and less well documented, but the potential benefits are just as tangible. Through our research and experience to date we have found that enhanced D&A capabilities can advance NFPs across three dimensions: time savings from report writing, improved performance and impact measurement, and operational improvements.
As our survey showed, many of the D&A tools used by NFPs today are ill-equipped to run the automated and bespoke analyses required to simplify report writing. Furthermore, many NFPs do not have the dedicated D&A-trained staff to implement such procedures. Improved D&A tools and dedicated D&A staffing can greatly simplify data collection and analysis, enabling manager time to be redirected from report generation toward more impactful and rewarding tasks. Last year, GDI volunteers worked with an NFP program manager who spent 2-3 days each month preparing a standard report. Together with the manager, the team restructured the NFP’s database and created an automation tool that could produce the report data in under a minute. This situation is not unique; GDI and 180DC have each worked with many organisations with latent time savings that could be unlocked by D&A improvements.
Many organisations that struggle to track and articulate their impact are limited by D&A capabilities, rather than a lack of appropriate statistics. Our interviewees explained that it is common for data capture and storage to be overlooked when a program is first commenced. This is typically either because it isn’t a priority, the platforms do not exist or are cumbersome, or it is unclear what the key metrics should be. Once the program has started, it is difficult to implement the appropriate processes and instigate the behaviours to ensure the program data is captured going forward. There are a wide range of cloud-based tools that can greatly simplify data capture and storage, reducing the burden on program staff and eliminating a key barrier to collecting powerful data. Examples range from common tools like Google Forms/Sheets, Dropbox and Microsoft 365, to more sophisticated tools such as SurveyCTO and AWS S3. AWS S3, is a secure storage facility provided by Amazon Web Services and is often referred to as a “data lake”. Such facilities can act as a central repository for various types of datasets, both structured and unstructured. Registered NFPs can also receive significant discounts to many of these solutions from initiatives such as TechSoup.
A 2019 survey by Kearney showed that corporations with leading D&A strategies generate as much as 83% more profit than those lagging. The nature of D&A in the NFP world is less advanced, and less well documented, but the potential benefits are just as tangible. Through our research and experience to date we have found that enhanced D&A capabilities can advance NFPs across three dimensions: time savings from report writing, improved performance and impact measurement, and operational improvements.
As our survey showed, many of the D&A tools used by NFPs today are ill-equipped to run the automated and bespoke analyses required to simplify report writing. Furthermore, many NFPs do not have the dedicated D&A-trained staff to implement such procedures. Improved D&A tools and dedicated D&A staffing can greatly simplify data collection and analysis, enabling manager time to be redirected from report generation toward more impactful and rewarding tasks. Last year, GDI volunteers worked with an NFP program manager who spent 2-3 days each month preparing a standard report. Together with the manager, the team restructured the NFP’s database and created an automation tool that could produce the report data in under a minute. This situation is not unique; GDI and 180DC have each worked with many organisations with latent time savings that could be unlocked by D&A improvements.
Many organisations that struggle to track and articulate their impact are limited by D&A capabilities, rather than a lack of appropriate statistics. Our interviewees explained that it is common for data capture and storage to be overlooked when a program is first commenced. This is typically either because it isn’t a priority, the platforms do not exist or are cumbersome, or it is unclear what the key metrics should be. Once the program has started, it is difficult to implement the appropriate processes and instigate the behaviours to ensure the program data is captured going forward. There are a wide range of cloud-based tools that can greatly simplify data capture and storage, reducing the burden on program staff and eliminating a key barrier to collecting powerful data. Examples range from common tools like Google Forms/Sheets, Dropbox and Microsoft 365, to more sophisticated tools such as SurveyCTO and AWS S3. AWS S3, is a secure storage facility provided by Amazon Web Services and is often referred to as a “data lake”. Such facilities can act as a central repository for various types of datasets, both structured and unstructured. Registered NFPs can also receive significant discounts to many of these solutions from initiatives such as TechSoup.
Communications and marketing campaigns present a variety of challenging questions: which donor clusters to target, how frequently, or what call to action to include. The challenges can be due to limited information on donor demographics, limited tracking of prior campaign success, or not knowing what analysis to run on the data available. Implementing a D&A strategy can ensure that all relevant communications and donor information is tracked from the start, and that the data is structured so that the right questions can be asked to inform future campaigns. To better understand how D&A can optimise NFP resource allocation, GDI interviewed a program manager of a not-for-profit healthcare provider that had been experiencing challenges with patient medication allocation. The provider responded by investing in D&A processes to ensure that detailed tracking information was logged at each stage of the medication distribution journey. The team was able to quickly identify the source of the errors as incorrect medication labelling, and the medication allocation issues reduced by 60%.
D&A can also be used to create new NFP propositions which would not be possible otherwise. An example organisation is Inclusive America, a U.S.-based NFP that is developing a database of qualified diverse, bi-partisan candidates for political office in the U.S. It would not be possible to create such a platform without access to D&A skills and technology.
Section 3: Implementing A D&A Strategy
NFPs that choose to implement improved D&A practices can realise benefits across the organisation, but with so many different tools available, and limited resources to draw from, it can be difficult to know where to begin. Drawing from prior experience, we have synthesised a framework to assist NFPs navigating the change to getting more out of their data.
The Five Pillars Of A D&A Strategy
We define a D&A strategy as a set of processes for collecting, storing, analysing, and reporting data to enable data-driven decisions. We consider there to be five steps for developing a data strategy:
To help explain this framework we have used a case study of a D&A project undertaken by the Good Data Institute for Rape Prevention Education (RPE), a New Zealand based NFP.
GDI partnered with RPE to establish automated processes for three of its major operations: board reporting, rostering, and survey data analysis. In this case study we will present our approach to automating the survey data analysis.
Step 1: Purpose
Before developing the automation process, GDI worked with RPE to establish the objectives of automating their survey analysis. It is critical to establish these objectives up front to ensure that the strategy is fit-for-purpose.
RPE runs regular educational sessions relating to sexual violence and harassment issues. RPE wanted to understand how they could regularly tailor these presentations to reflect the changing attitudes of their audience as their understanding and awareness of the issues evolved.
To inform the session design, RPE wanted to use data to better understand the impact that each session was having on its audience’s awareness and understanding of these issues over time.
Step 2: Collection
There are several factors to consider when determining the data collection method. What resources would it require? Is it easy to replicate? Are there any inherent biases in the collection method? Is it error-prone? Will it be easy to store?
The most important factor for the RPE team was mitigating the risk of bias in its survey. Avoiding bias in surveys always presents challenges; for RPE we focused on ensuring two key points: that the respondents were truly reflective of the target sample (by maximising each audience’s response rate), and that the questions were clear and neutrally-phrased.
RPE decided to present each audience with a paper survey, and to transcribe the responses into Excel afterward.
The clear outstanding issue with this approach is the labour-intensive and error-prone nature of manually transcribing each survey. However, we opted for this approach because the main digital alternatives also present their own challenges: the costs involved in procuring tablets for a resource-constrained organisation, and the limited response rate achieved by sharing online survey links. The most appropriate data collection approach is not necessarily that with the most automation.
Step 3: Storage
Data storage approach is an important consideration for two key reasons:
Security Data security is often overlooked, but as data breaches and online threats become more prevalent, it is important to take active measures to safeguard critical systems and sensitive information.
Secure storage and regular backups help protect sensitive information from system failures, improper usage, or cyber threats. But an increasingly complex online environment means data must also be protected from unauthorised access, whether by accidental breach, someone within the organisation, or by a hacker.
There are several questions to focus on when addressing this issue:
a) Is my organisation collecting data about people that is identifying, revealing, or otherwise sensitive, if it fell into the wrong hands? This could include information on donors, clientele, financial records, or confidential agreements.
b) Who has access to what datasets and where are those datasets stored?
c) Who is managing access privileges within the organisation?
d) Is my organisation adhering to best-practice regarding authentication and passwords?
e) Are staff regularly informed of best-practises regarding data security?
f) We strongly advise speaking with an expert in data security processes and systems before implementing a data storage system. There is also an excellent guide for Australian SMEs written by the Australian Cyber Security Centre which has insights easily applicable to small-medium sized NFPs.
Ease of use
The data has limited value if the relevant stakeholders cannot easily access it, or if it is in an unmanageable format. A platform that is user-friendly and cloud-based will help ensure usability.
For RPE, the data was stored in a secure web-based system (AWS S3, an access-controlled data lake service managed by AWS), which took in the raw spreadsheets RPE produced and mapped them to csv files. The GDI team then developed a web application in R (using the R Shiny package) which cleaned and manipulated the data in order to simplify analysis.
Step 4: Analysis
There should be two components to an analysis process: the exploratory phase and the production phase. Exploratory phase
The exploratory phase is where you dissect the data by drilling into it and visualising it across a variety of dimensions. The aim is to identify the data points most suited to your question, either for immediate extraction to insert in a one-off report, or for recurring analyses to include in future reports, which will then link into the production phase.
Programs with dashboard features such as Tableau, Microsoft Power BI or Google Data Studio, are helpful for this phase, because they allow you to quickly visualise your data across a variety of dimensions.
Production phase
Once you have identified the most relevant data points, you move to the the production phase to focus on producing only the exact analysis or visualisations required to answer your questions.
If you are producing a recurring report, it is helpful to create a streamlined system for generating the data points and key visualisations on a regular basis.
Programming languages like R or Python are well suited for this phase, as they allow the analysis to be automated.
After exploring the data with the RPE team to determine what was most insightful, the GDI team used R to create a dashboard with each of these analyses clearly visualised, and regularly updated. This is accessible from within the web application created to upload and store the data, greatly simplifying retrieval for the RPE team.
Note - reproducibility
Steps should be taken during the analysis phase to ensure that whatever process is chosen is reproducible. Reproducible tasks are those that can be easily replicated and understood throughout an organisation, not just by those responsible for creating it. Doing so is important to ensure that the system can be adjusted and maintained over time, while enabling greater automation. Reproducibility was emphasised in the design process of the GDI project for RPE, and is why, where resources allow for it, GDI encourages organisations to script their own solutions for analysis, rather than using programs such as Microsoft Excel.
Step 5: Reporting
Reporting is ultimately about compiling and organising the insights drawn from analysis and communicating implications to the decision makers.
It is important to consider your audience when choosing how to present the information; without digestible charts, they won’t be any closer to making a data-driven decision.
Reports are often thought of as written documents, but they don’t always need to be. They can be in the form of dashboards (e.g,. Tableau, Google Data Studio), emails, or any other format that is easily consumed by the end-user.
In the case of RPE, reporting and analysis was both conducted via the dashboard constructed by GDI. The dashboard consisted of data visualisations summarising the feedback from those who attended their programs. This was ground-breaking for RPE; they had not previously used any of this data. RPE’s users were also able to break down each visualisation by demographic and geography. Using R, GDI could analyse unstructured, free-text data and summarise it by sentiment and word-frequency.
The dashboard was built using "Shiny", a web-application framework for R, and was deployed with authentication; RPE’s key decision makers could log in with a simple URL and easily click through the dashboard.
Closing Remarks
We would like to thank each of the not for profit members who dedicated their time toward helping us with this study. We would also like to thank the 180 Degrees Consulting team of Yan Zhai, Rachel Wong, Chi Mao, Arani Sivakumar, and Adam Stanley for writing, fielding, and analysing the survey. Lastly, we would like to thank members of the GDI leadership team for their contributions, including Luke Mills and Penny Talalak.
About GDI
In late 2018, the Good Data Institute (GDI) was co-founded as ANZ's first dedicated organisation to give not-for-profits access to pro-bono data analytics support and tools, via a network of socially-minded data analytics volunteers. The launch of the organisation was driven by the realization that the same data and analytics processes that the private sector is using to compete in the marketplace can also be leveraged by mission-driven organisations to accomplish their goals and enact change. GDI has since supported over 10 NFPs to understand issues or needs more deeply, to improve operational effectiveness (i.e., in support functions such as fundraising), or to visualise and quantify results and impact. Through our work, GDI aims to become the number one pro-bono data analytics partner for the NFP sector in Australia and New Zealand, ultimately extending our ability to identify, share, and help implement the most effective means for growing NFP people, organisations, and their impact.
If you are interested in learning more about we can work together to help your organisation achieve its full D&A potential, please get in touch through our website: https://www.gooddatainstitute.com/join-us
Alternatively, if you are a D&A professional looking to use their skills for good, we are always on the look-out for capable and motivated volunteers to join the network. Please send us a message! https://www.gooddatainstitute.com/volunteers
About 180DC
180DC is the world’s largest student-run consulting group for non-profit organisations and social enterprises. Our mission is to provide high quality, affordable consulting services to help organisations develop innovative, practical and sustainable solutions to whatever challenges they are facing. This whitepaper was done with the 180DC branch at the University of New South Wales. To see an example of the kind of work we have done, see the case study below:
Example work:
One example is a project done by 180 Degrees Consulting UNSW for the peak body of an Australian transport charity, where 180DC developed a solution using the Google Suite.
The purpose of this project was similar to RPE; the client wanted to know how they could best tailor their services to the needs of their users, and they needed a way to understand who was using their services and what they were using them for.
The team created a system that regularly collected user behaviour data (using Google Forms), transferred the data into a consolidated spreadsheet (using Google sheets), and which was processed, analysed, and reported using a Google Data Studio dashboard.
Through this solution, the NFP was able to understand the needs and behaviour of its users in real-time to better tailor their services to the needs of their community, using tools that cost them no money, and took limited resources to maintain.