Six months have passed since we rolled out the Luceena Prediction Feature. At the time, we referred to the old dilemma attributed to Henry Ford, who is said to have quipped that 50% of his marketing was wasted, he just didn’t know which 50%.
We all know that marketing and communication don’t always work the way we plan and design them. Sometimes, there are simply unknown variables that throw a wrench in our plans. Of course, there are already performance-based solutions, like Google AI campaigns, A/B testing, or the more traditional focus groups that review and assess entire campaigns.
With Luceena, we wanted to take a different approach. One that meets the demands of brand building and performance while also reflecting the speed of digital communication. That’s why we conceptualized our Prediction Feature last year and launched it in January.
We don’t want to go too much into detail here, as it is, of course, one of Luceena’s features that gives us a market advantage. Still, we can disclose this much: the model, in its final expansion stage, includes 11 different evaluation clusters that contribute to the success prediction of a post. These range from “Image Appeal” to the tone of the caption, the posting time, or even the weather.
We trained the current model with just over 80,000 posts from Instagram and LinkedIn, focusing on the industries that correspond to Luceena’s largest user accounts. While this might not seem like much compared to AI models like GPT-4, which are rumored to be trained on around 100 trillion data points, after more than six months and just over 2,000 posts that Luceena has analyzed and assessed, we can confidently say that our current model is already impressive.
In the two industries on which most of the training data is based, we have a prediction accuracy of ±10% for the actual performance (engagement, reach) of a post. When we step outside of these industries, we still maintain a very good accuracy of ±20%.
The good thing about the model is that it learns continuously. Every post submitted through Luceena, for which we are allowed to pull performance data after posting (there are some legal aspects here), improves the model.
An interesting point from our data is the prediction performance over the course of the year. As expected, it becomes slightly less accurate with better weather. Although our training data includes historical information covering an entire year, we probably couldn’t predict the summer accurately due to the limited dataset. The European Championship (June 14 – July 14) also had negative effects on the accuracy. We’ll definitely keep an eye on that.
We can only recommend every social media manager to give Luceena a try. A quick note, though: a prediction will only be displayed after a few posts, as the first predictions are not valid enough. After all, each channel works a bit differently. By the way, you can sign up here. And the best part? Luceena is still free!