Marketing Mix Modeling – will Robyn by Meta automate MMM?

Robyn is an open source project by Meta that aims (at least to some extent) to automate Marketing Mix Modeling. According to the authors, its goal is to, among other things, exclude human error and analyst subjectivity. In this article, I would like to describe its most important functions and share my own opinion on whether Robyn is able to completely automate the Marketing Mix Modeling process. However, before I get down to describing the tool, first let’s understand the term Marketing Mix Modeling.

Marketing Mix Modeling

MMM is an analysis using econometric techniques that, using historical data (including sales, media or seasonal data), explains what and to what extent affected, for example, sales of a given product/brand. As a result, those responsible for planning marketing activities are able to use the media budget more effectively in the future. As for Robyn itself, it contains a lot of interesting elements. Below I will mention those that I think are the most important.

Ridge regression

Robyn decomposes the dependent variable using ridge regression, which is a variant of linear regression that includes the L2 regularization norm. It differs from the classical linear model in that it imposes a “penalty” on insignificant variables. It was used because variables in MMM models are sometimes highly correlated with each other, which means that we cannot draw statistically significant conclusions from the model.

 I think that the use of ridge regression is a very good solution, especially when we aim to automate the modeling process. In this case, a user unfamiliar with statistical methods gets the opportunity to build a model without detailed data analysis (which is nevertheless recommended). However, this can lead to a situation in which the algorithm imposes a penalty on a medium that the user cares a lot about testing. In this situation, it will not be included in the decomposition, which can prove problematic from a business perspective.

Ad-Stock Effect and Effectiveness Curves

To perform modeling in Robyn, we need to assign hyperparameters to all media variables (e.g., TV spending). We specify them in ranges (e.g., from 0.4 to 0.7 for TV), and then during modeling Robyn selects them using the Nevergrad library (which will be presented later). Sample intervals for different media have been recommended by Meta, but the user can change them as deemed appropriate (e.g., reduce them when modeling on monthly data). We use the hyperparameters to calculate Ad-Stock effects and Effectiveness Curves.

The Ad-Stock effect for media variables captures the fact that a consumer, after seeing an ad, does not forget it immediately, but over time. The higher the Ad-Stock effect, the longer the consumer remembers the ad. This is what a sample graph of these effects looks like in Robyn:

Effectiveness curves are used to visualize the non-linear impact of spending on media efficiency. They inform us below which threshold it is not worth investing, above which further investment is ineffective, and where the so-called “sweet spot” is located, which is the optimal amount to invest in a given medium.

I think the solution is very interesting and certainly makes it easier to determine the parameters of the effectiveness curves and ad-stocks. By determining ranges rather than specific values, a user who does not specialize in Marketing Mix Modeling will be able to achieve satisfactory results.

Prophet

To determine variables over time like trend, seasonality or holidays, Robyn uses the Prophet tool (created by Facebook). It allows the model to select holidays according to the country for which you are building a model, and automatically determines trend and seasonality.

While the idea of automating the effect of holidays is a very good one, I believe that more intervention is required from the person performing the modeling process in order to get the correct results. For some products, some holidays increase sales, while others decrease them. Therefore, treating all holidays in the country as a whole may distort the actual impact of specific holidays in the model.

This is how sample graphs of variables generated through Prophet look like. The trend and season were created based on the dependent variable.

Nevergrad

This library (also created by Meta) allows you to efficiently optimize hyperparameters and metrics for assessing model quality, which include:

• NRMSE – the lower it is, the lower the variance of models residuals – which is better!
• DECOMP.RSSD – the lower it is, the better the model matches decomposition to the distribution of media spending

Based on these, Nevergrad chooses a set of Pareto optimal models. But where does it choose them from? Well, Nevergrad performs a user-selected number of iterations (the documentation recommends no less than 2,000) that create multiple models. It then minimizes the selected metrics and selects Pareto optimal models from among them.

These have been marked with a red line. As you can see, these are the ones that minimize both NRMSE and DECOMP.RSSD.

In my opinion, Nevergrad is an amazing optimization library and works perfectly in this case. It selects models very sensibly, works quickly and bases its results on natural selection.

Budget allocator

The final, but also very important feature of Robyn described in this article is the budget allocator. Robyn, for our chosen model, compares the historical budget allocation with the algorithm’s recommendation, and also shows the predicted changes in media impact after applying them.

The light color indicates the empirical portion of the budget spent on the medium, and the darker color indicates the portion suggested by Robyn.

In my opinion, this is a great solution that can significantly help planning the future budget so that all media are used as efficiently as possible. It clearly shows which media have been underinvested and on which too much money has been spent.

Conclusions

I think Robyn is a great tool for a general check of what influences, for example, our brand sales. However, there are a few issues that, in my opinion, won’t make Robyn a full replacement of a classic Marketing Mix Modeling.

First of all, with Robyn we will not be able to get results at a high level of detail. Due to the use of ridge regression, we are not able to use media data with a very detailed breakdown in the model. The algorithm will simply consider them too insignificant, impose large penalties and consequently not include the variable in the decomposition.

In addition, Prophet variables (season, trend, holidays) are effective in terms of automating the entire process, however, without manual modification it is difficult to draw meaningful conclusions. In the classic approach to Marketing Mix Modeling, we are able to examine which elements of the “season” affect the dependent variable, and which exact holidays affect it positively or negatively.

I would recommend Robyn to people who want to examine at the general level the elements that influence, for example,  the sales of a product or brand. Although it is not able to give very in-depth conclusions, it works well for a general view of the determinants of sales.

Ultimately, I believe that Robyn can streamline the work of analysts, however, this tool alone cannot automate Marketing Mix modeling. Despite features that streamline some processes, a lot of analyst input is still needed to achieve satisfactory results.

References

1. Robyn’s documentation
2. Nevergrad’s documentation
3. Prophet’s documentation
4. Photo by Campaign Creators on Unsplash

Would you like to publish an article on Data Science Hacker, as Igor did? We are open to cooperation and would be happy to promote your article on social media.

Contact us!

---