Background

In recent years there have been a series of changes in the environment of the digital marketing industry focusing on ensuring the strictest control of
users’ privacy. From the launch of the new GDPR data protection legislation in 2018 to the recent confirmation by Google of the banning of the use of third-party cookies on Chrome from 2023, the possibility of accessing personal identifying data for measuring and optimising the impact of digital advertising campaigns is increasingly limited.

In this regard, Meta, the company that owns some of the most widely used apps in the world, such as Facebook and Instagram, developed Robyn.

Robyn is a transparent open-source code for Marketing Mix Modeling analysis that is available for data scientists, analysts or modellers at any company, institution or body. This is undoubtedly an important step in democratising access to MMM, a tool for measuring ROI on marketing activity that consists of the use of advanced statistical models for modelling time series (weekly, daily, etc.) in order to obtain the most influential variables in the evolution of such series. These models are performed on Key Performance Indicators, which are all the variables identified as necessary for assessing the effectiveness of a marketing strategy (sales, website visits, etc.), and do not use any personal identifying data for the analysis.

The main aim of MMM is to be able to explain the change over time in these KPIs based on the various drivers that affect the business. Irrespective of the model chosen, the result obtained is an equation capable of estimating the behaviour of the variable analysed. The closer this estimate is to reality, the more robust the model is, and the better the decisions taken on the basis of the results will be.

At the beginning of 2022, Meta engaged Deloitte to conduct a study, using Robyn, to measure the impact of advertising on direct bookings on the websites of four Spanish hotel chains that have had different media strategies. The project had two specific objectives: (1) perform a diagnosis of Robyn’s potential based advertising expenditure per hotel per medium, identifying optimisation opportunities. It should be noted that, in order to conduct the analysis, weekly data from October 2018 to September 2021 were used, including multiple variables in the analysis.

The main data sources used are as follows:

• SimilarWeb and SemRush: information on web traffic and internet bookings of the four hotel chains subject to analysis, as well as the estimated spend on Paid Search.
• Meta: metrics of advertising expenditure and impressions on Facebook and Instagram.
• Google: mobility indicator used to explain lockdown and de-escalation in Spain (Google Mobility) and relevant search information (Google Trends).
• Kantar and Infoadex: information on GRPs on TV and estimated spend by medium.
• Eurostat and INE: macroeconomic variables (unemployment, economic situation, travellers, etc.)
  

Methodology and diagnosis of Robyn

We commenced the study with an exploratory analysis of all the information available that helps us to understand the data gathered and ascertain the situation in the hotel industry in Spain over the three years of available historical data on website bookings and advertising expenditure.

Once the relationship between the various indicators under analysis was understood, we could commence the modelling process. As has been mentioned above, we used the Robyn packaged. Robyn is a group of codes in R that is based on ridge regressions of time series that permits the automatic selection of hyperparameters and the calibration of results through other measures, such as experiments. Through a real exercise performed in the hotel industry, we have available a diagnosis of the main advantages of Robyn over other more traditional modelling approaches:
 

It reduces human bias:

1. It automates the seasonality extraction plus adstock & saturation selection using a combination of algorithms like Fourier Series and multi-objective optimisation from ML-packages Prophet and Nevergrad, increasing the interpretability and precision of the model.
2. It uses regression ridges to resolve multi-collinearity.
3. It standardises the method used for measuring the efficiency of marketing, enabling analysts to follow a standardised modelling process to assess each variable appropriately.

It facilitates optimum adaptation to the specific nature of each case:

1.  It is fully customisable to the metrics that are of interest to an advertiser (notoriety, website visits, sales, etc.).
2.  It is adaptable to the behaviour of each variable, giving, for example, the possibility of measuring a different advertising recall for each advertising medium.
3.  It allows calibration and validation through experiments (A/B testing).

It speeds up decision-making:

1. It is structured in an automated code, which makes it
   significantly faster to execute than traditional models.
2. It includes an integrated marketing budget optimiser with the possibility of applying restrictions.
3. It reduces modelling times, enabling campaigns to be optimised more quickly.

Key findings of the Business Case performed

As well as putting Robyn to the test, the results provided Meta with full visibility of the impact of advertising campaigns on their platforms (Facebook & Instagram) in the real context of advertising media planning.

Following the application of Robyn open-source code to measure advertising efficiency, the following significant learnings were obtained:

• Robyn enables robust MMM models to be developed. The model fit (R2) obtained for total internet bookings was 81%.
• Advertising on Meta, in both video and non-video format, has significant importance in generating website bookings. In 2021 16.8% of internet bookings were generated through Meta (video + non-video).

• Adstock is measured for all the advertising media analysed. The audiovisual media (Meta Video and TV) obtained the largest ad-stock effect, confirming their great power to generate advertising recall. On the other hand, Paid Search was the medium with the lowest advertising recall.
• From the analysis of the aggregate Meta data for the four hotel chains, it can be seen that the expenditure in Meta has not yet reached saturation. The contribution to internet bookings at hotel level is correlated on a linear basis to the real level of expenditure made.
• Paid Search accounts for the largest percentage contribution to bookings due to the high level of expenditure made. However, advertising in Meta achieves the best ROI, i.e., the largest number of internet bookings per euro spent. If we translate the results to cost per acquisition (CPA), Meta obtains a result of less than EUR 30 for each booking.
• The optimisation of media suggests the expenditure on Meta advertising media, especially in video, should be increased to harness the recall effect generated by this medium. If the media strategy is optimised, the total volume of internet bookings could be increased by 8.8%.

In short, Deloitte has been able to evaluate the Robyn open-source code in a real case in the tourism industry and has concluded that it is a very useful option for the robust and efficient development of MMM. In developing models on the basis of the Robyn code, we continue to obtain the main results historically provided by measures using MMM (including optimum levels of expenditure for all media under analysis), and also see improvements in precision and flexibility, facilitating better adaptation to the real context of digital advertising.

In addition, this project has enabled Deloitte to begin the development of Robyn AccelarAItor, an easy-to-use cloud- based platform that will make the execution and selection of the best model even simpler, taking advantage of all the functionalities offered by Robyn and enabling profiles with less technical knowledge to apply this type of methodology.

Contact us

Carlos Real

Senior Manager - Artificial Intelligence & Data

creal@deloitte.es