The solution to resolve Looker Studio's connection to GA4
November 30, 2022 | Thomas Danniau, Solution Lead
What is the problem?
GA4 API quotas
Looker Studio is using an API to request data from GA4. API or Application Programming Interface is a commonly used protocol for two applications to communicate with each other. However, there are a couple of limitations with this approach. One of them is that you are highly dependent on the API of the data source you wish to connect with.
When Google updated the quotas for the GA4 DATA API in November 2022, a lot of the Looker Studio users received the following error: “Looker Studio cannot connect to your data set.” So, the actual problem is not directly related to Looker Studio but rather to the GA4 API. More information about these quotas can be found here.
At first glance, the quotas seem reasonable. However, you can easily reach the quota limitation when
- You have multiple Looker Studio reports,
- The reports are consumed by multiple people
- You are using a lot of filters within the report.
This issue can be extra annoying when you are developing or updating a dashboard and you frequently need to refresh the data. Imagine that you receive this error when you have been working on it for a couple of days?
In the short term, you could reduce the consumption of the dashboard (e.g. by making it simpler or by sharing it with less people) or you could hope that Google will update its quotas. Obviously, this is not a sustainable solution!
More importantly, it should be a trigger to rethink how you approach reporting and analytics. This issue is not a GA4 API issue only. If you connect directly from a visualization tool to any data source like GA, Facebook, LinkedIn, a Marketing Automation tool, CRM etc., you will eventually reach the limitations of this setup:
- Looker Studio is a visualization tool, not an ETL (Extract, Transform and Load) or data storage technology.
- When you extend the amount of data sources, the performance of the dashboard will be low because you depend on the speed of the API connection of those data sources. The tipping point is unpredictable, but eventually you will have an annoyingly slow dashboard.
- The data transformation capabilities in Looker Studio are limited and most calculations will impact the performance of the dashboard. E.g., If you wish to filter or aggregate data from different websites, you will be slowing the reports significantly.
- You depend on the existing connectors and APIs. The approach is not flexible and is a potential risk for future data sources. The GA4 API update is a good example but there are plenty of other similar updates from other vendors.
- User access can be a mess in Looker Studio. When you want access or edit rights to a dashboard, you also need to give access to the data sources. E.g., when you are onboarding or offboarding new people, things can get complicated.
What is the solution?
(Marketing) Data Warehouse
We recommend a layered approach with a Data warehouse in between. The principle is that you have a separation of concerns. So, this means no direct connection between the sources and the visualization. Instead, you prepare the data in a data warehouse.
There are different approaches and solutions to extract the data from your sources to your (marketing) data warehouse. A visualization tool (like Looker Studio) can connect to a clean pre-prepped dataset in BigQuery or another preferred data warehouse solution.
This DWH approach is not a quick fix. You need to set up the environment and you need to reconnect the sources of an existing dashboard. But eventually you will have a scalable setup with future proof capabilities.
- All marketing data in one central database
- Ownership of your data
- Collect and store historical (third) party data
- Aggregate and compare data from multiple data sources
- Visualize data in a secure and fast way
- Advanced analytics (custom attribution modelling, machine learning on raw data, …)
- Pave the way to translate data into value via AI
Composable approach without vendor lock-in
The beauty of a composable architecture is that you do not have to stick with one Vendor. Because there is a separation of concern, you can potentially replace BigQuery with Azure SQL server, Looker with PowerBI and so on. It is also perfectly feasible to combine custom code with the ease of use of a SaaS solution. This hybrid way of working gives you the required flexibility with a reasonable total cost of ownership.
Depending on the requirements and capabilities, we can recommend different solutions for ETL, data warehouse and Data visualization.
What is your digital challenge for 2023?
Have a chat with Christophe, one of our business developers, over a nice cup of coffee to discuss your digital challenges and where we can help you.
Keep reading with these insights
Google Analytics 4: Why you need to switch now
No way you can avoid it: the sunset of Universal Analytics (UA) is approaching and before you know it, Google Analytics 4 (GA4) will be high in the sky. Yet, the lion's share of companies is not prepared for this big change. Are you?
Digital suite or composable technology stack: which one to choose?
While digital experience suites have their advantages, a new approach is challenging them in terms of flexibility and efficiency. Is it time to trade in the digital suite for a composable technology stack?
Working with customer data in 2022: challenging, but not impossible
As the data landscape is changing and consumers regain control of their data, it is becoming increasingly difficult for organizations and digital marketers to manage customer data in a compliant way. However, while challenging, it is not impossible. The key: transparency and trust.