A complete guide to tracking events with GA4 & BigQuery

12/01/2023
A guide to using the new Google Analytics 4 API & Google BigQuery to track events from any kind of client, not just web, without using firebase or gtag.

Useful but not often used, the new GA4 measurement protocol offers a programatic method of tracking events using Google Analytics.

Let’s first have a look at the diagram provided by google, explaining the GA4 measurement protocol:

You can start by adding a new GA4 property to your Google Analytics dashboard. Go to the property settings, “Data Streams”, click on “Add stream” and choose “Web”.

Now choose a URL and Name. You can disable enhanced measurement.

After you created the stream, write down the “MEASUREMENT ID”. After that, go to API Secrets, create a secret and write down the secret value.

After that, you are all set to continue to the implementation. I will be using Node.js for my implementation, but you are free to use any programming language you like.

const https = require('https')

const apiSecret = process.env.GA_MEASUREMENT_PROTOCOL_SECRET
const measurementId = process.env.GA_MEASUREMENT_ID

// store this somewere and reuse it
const clientId = 'a unique client id string to identify this device/client'

// this identifies the specific user
const userId = 'the id of the user'

// generate a new session id for every new session
// you can also collect the session id from the "_ga" cookie if your events are coming from a web client
// using the "_ga" cookie allows for more insights like cross-device tracking for example
const sessionId = 'the id of the session' 

const collectionUrl = 'https://www.google-analytics.com/mp/collect' // or use /debug/mp/collect to validate events

const postReq = https.request(`${collectionUrl}?measurement_id=${measurementId}&api_secret=${apiSecret}`, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'User-Agent': 'My user agent string'
  }
}, response => {
  if (response.statusCode === 200 || response.statusCode === 204) {
    res.json({ success: true, error: false })
  } else {
    console.error('Backend reporting error, status = ' + response.statusCode)
    res.json({ success: false, error: { message: 'Error while reporting', code: 'backend_reporting_error' } })
  }
})

const events = [
  {
    name: 'clicked_on_the_red_button',
    user_properties: {
      // https://developers.google.com/analytics/devguides/collection/protocol/ga4/user-properties
      customer_tier: {
        value: 'level 3'
      }
    }
    params: {
      // values should not be more than a 100 characters long
      session_id: sessionId,
      timestamp_micros: Date.now() * 1000 // important to include
      engagement_time_msec: 100 // this value must be set

      event_category: 'Interaction',
      event_label: 'Red button click',

      my_custom_property: 'some value',
      my_second_custom_property: 42,
    }
  }
]

postReq.write(JSON.stringify({
  client_id: clientId,
  user_id: userId,
  events: events
}))

postReq.end()

First, start by declaring the “apiSecret” and the “measurementId”. Make sure that the api secret is not publically accessible. The client_id, session_id and user_id should be defined for every request.

The URL endpoint for GA4 is: https://www.google-analytics.com/mp/collect or /debug/mp/collect if you want to receive error messages.

Make sure that the events follow the required JSON format. The most important requirements are that the events need. to have a name, a session_id, a timestamp_micros property and an engagement_time_msec property.

The events also need to adhere to the following additional requirements:

  • Requests can have a maximum of 25 events.
  • Events can have a maximum of 25 parameters.
  • Events can have a maximum of 25 user properties.
  • User property names must be 24 characters or fewer.
  • User property values must be 36 characters or fewer.
  • Event names must be 40 characters or fewer, may only contain alpha-numeric characters and underscores, and must start with an alphabetic character.
  • Parameter names (including item parameters) must be 40 characters or fewer, may only contain alpha-numeric characters and underscores, and must start with an alphabetic character.
  • Parameter values (including item parameter values) must be 100 characters or fewer.
  • Item parameters can have a maximum of 10 custom parameters.
  • The post body must be smaller than 130kB.

You can test your events using the Google Analytics Events Builder.

Also check out the this reference for a comprehensive list of possible event types with custom properties. The list also includes information on forbidden and required parameters.

You can now test your code and verify it works by checking the “Real-Time” tab. inside GA dashboard. 

 

Using BigQuery for data analysis

 

Next, we will be streaming the events to Google BigQuery which allows us to do large scale analysis on our data.

Log into your Google API Console and create a new project. Go to API’s and services and enable BigQuery. Make sure that billing is enabled for your account or that you have a free trail.

Now go back to your Google Analytics property, in the admin view, scroll down to “Product links” and click on “BigQuery” to add a link to BigQuery. You can choose either “Streaming” or “Daily” exports.

That’s it, you can now deploy your app in the wild using the GA4 measurement protocol!

Thank you for reading!