AI Sentiment and Category Analysis
The Client
Creating Communities Australia (CCA) is a prominent organisation committed to fostering vibrant and sustainable communities across Australia, with a mission to build flourishing people and communities.
Over the past 30 years CCA have developed a range of IP which they are looking to productise using AI to improve community wellness, equity, and liveability.
Specialising in community engagement, CCA routinely conducts extensive surveys to gather community insights. However, their challenge was the laborious task of manually categorising survey responses into thematic categories, which was both time-consuming and costly.
The Goal
The goal was to use CCA models and data to build a bespoke survey and analytics tool to process large quantities of data and provide insight into the community experience to inform a client’s decision-making.
CCA urgently needed to modernise and streamline their survey analysis process by replacing their manual categorisation method with an automated system to save time and reduce costs.The longer-term hope is that this solution can be adapted to a variety of residential communities including local government areas, FIFO villages, prisons, and refugee camps amongst others.
The Journey
Enlighten Designs embarked on a project with CCA to develop a cutting-edge solution. The initiative began with in-depth research into machine learning models for text classification and OpenAI's advanced language models for automated text segmentation and sentiment analysis.
The Solution
The Enlighten Designs team set out to create a solution that not only harnesses the power of AI but also puts user-friendliness at the forefront. Here's what we designed:
• Intuitive Survey Upload Interface: We developed a straightforward web application where users can easily upload their survey files. This platform is designed with simplicity in mind, ensuring that users of all technical backgrounds can navigate it without any hassle.
• Seamless Backend Processing: Leveraging Azure Functions, we've enabled a serverless backend that operates smoothly behind the scenes. This means quicker response times and less waiting for the users when they upload their survey data.
• Smart AI-Powered Analysis: Utilising OpenAI's cutting-edge GPT model, our system can intelligently parse through the survey responses, breaking down and assessing sentiments with remarkable accuracy. It's like having a keenly intuitive assistant who reads between the lines, recognises emotions, and understands the context - all to elevate the insight you gain from your community's feedback.
• Custom-Tailored Categorisation: With a sophisticated machine learning model that we trained using synthetic data from OpenAI, the tool can sort responses into 27 distinct categories. This custom-built categorisation not only saves users countless hours of manual sorting but also enhances consistency in how responses are interpreted.
• Interactive Data Visualisation: To make sense of the data, we've integrated an interactive Power BI report that allows users to visualise and analyse the results dynamically. As soon as new data is processed, the report updates in real-time, providing an engaging and easily digestible way to view community insights.
Our aim throughout the development of this solution was to make complex technology accessible and useful for everyday tasks. We wanted to empower CCA staff and stakeholders with a tool that streamlines their workflow, liberates them from the drudgery of manual data categorisation, and provides insights that were previously buried in unstructured survey data. With this user-centric solution, CCA can now tap into the full potential of their community engagement efforts with ease and precision.
Continuous Collaboration and Refinement
Throughout project development, Enlighten Designs maintained close collaboration with CCA, fine-tuning the solution through rigorous testing and prompt engineering. This led to a highly accurate system for both sentiment analysis and category classification.
The Results
The final system delivered impressive results:
The language model excelled in segmenting thoughts, summarising sentiments, and contextually interpreting user comments, including the translation of non-English remarks, understanding of Australian slang, and identifying central ideas.
Through rigorous testing in collaboration with CCA, the machine learning model has demonstrated high accuracy in categorising survey responses.
The tool enables faster analysis, helping Creating Communities gain insights more quickly while reducing time and costs.
"Enlighten Designs has helped CCA to deliver a powerful, precise solution for analyzing our survey data. The insights we’re uncovering are highly valuable, and we're excited to see how further refinements will enhance our processes."