We are constantly looking for novel and interesting ways to use AI to help on client projects or develop in completely new products that everyone in the agency can benefit from. The part I always found the most interesting was the data used behind the models. Paid media clients deal with large data sets, and my view was we could train a model on this data and use it for quick analysis and report building.  

We named it PACE (performance analysis conversation engine). 

This had mixed results. The hurdles we needed to jump appeared too high, and we were not entirely sure we could jump them at all. The main issue was ‘hallucination’ and QA. When prompted with these large custom datasets, we could not trust the model to not make up an answer to clear the question it was asked. Checking if every answer was correct then became a problem as the checks themselves would take time. The other problem was blandness. When PACE gave us the right answer, it never managed to go the next step and provide something ‘new’. If we take part of a PACE answer as an example: 

Prompt for PACE: 

Analyze the Microsoft Ads data across markets and provide a summary of each market’s performance. Identify key weaknesses for each market and provide steps to address those weaknesses. 

Answer from PACE:

DK: Low CTR, low CVR, and high CPC indicating non-compelling ads, ineffective landing page, and possible overpayment per click. To address these weaknesses, the following steps can be taken: 

• Optimize ad copy and creative to increase the relevance and appeal, potentially improving CTR. 
• Review and improve the user experience on landing pages to help improve CVR. 
• Analyze bidding strategies to ensure a cost-effective CPC and CPM. 
• Segment campaigns more granularly to target higher-intent users, potentially reducing CPA. 

We can see that the answer is technically ‘correct’ but really provides little detail from the data it has and talks in generalities. We ultimately decided to park this idea, and look elsewhere. 

I still believed that synthesising data in certain ways could yield some good AI applications, and an opportunity to test this came during a workshop on search behaviours of a clients target personas. The idea was simple: could we turn an AI model into a customer persona by feeding it data, and then ask it questions?  

This tool was called ENACT. 

We refined several prompts to keep the model within its remit, and collected a variety of data points to use, including survey data, customer service data and demographic data. The prompt would be split into several structured areas: 

1. Introduction: Explaining what the model has to work with and the context 
2. Specific Query: Instruction on what to do. An example here would be “Assume the role of a persona in this data (small business founders) and tell me some interesting things about you” 
3. Data reference: Rules the model has to follow 
4. Verification: A sub-set of rules that help us QA responses 

This idea proved much more successful. 

Its task in that first client workshop was to ideate on new search topics that target personas might be interested in. It proved highly effective at suggesting a range of topics which we could then test for scale using a keyword planner. These answers were justified by specific topics raised in transcripts and survey data.  

Many other areas can be explored with ENACT.  

For example, we can ask the tool to assume the role of multiple personas at the same time, and then compare these against each other on key questions such as where they spend time online, and how this leads to differences in media planning for marketing campaigns to each persona. 

The two AI tools described here show our range of ideas for how to apply the technology to challenges we encounter in paid media. It also shows our recognition of hype turning into pragmatism as we have become more accustomed to the (current) limits of AI models, and where they might go in the future.