Menu
the REFERENCE

Using AR in cue sports app with firebase

This blogpost explains how we used Augmented Reality (AR) in the Snooker Coach application to project the ball positions (in real-time) on the actual snooker table.

About the SnookerCoach app

To stay in touch with the latest iOS technologies, and to contribute something to my sport domain, I have been developing the SnookerCoach app for several years now. I consider this to be a self service app that enables amateur snooker players to increase their level, and for professional snooker players to keep track of their results.

Check it out here:  https://snookercoaches.com

I always try to include the latest technologies; one eye catcher is the support for AR, but other interesting items include full integration with Firebase, QR scanning, intelligent sharing... In this blogpost I will give some insight in the usage of AR in the application.

Useful AR, and it's not a game!

A practice routine card for the exercise "half ball green". Notice the button "View in AR" at the bottom

A practice routine card for the exercise "half ball green". Notice the button "View in AR" at the bottom

 

One of the primary features of the SnookerCoach app is to present the player "practice routines" that are used to practice a certain skill. Practice routines are shown as cards which can be easily swiped.

Each practice routine has a specific table setup where the snooker balls need to be placed on very specific locations on the table. The accompanying practice routine description then gives a detailed explanation about what to do and how to do it.

In traditional training manuals, the table setup is shown via a static picture, depicting the table and position of the balls. This poses a real challenge for more complicated practice routines: it's nearly impossible to put the snooker balls on the exact same place. This is however crucial for achieving consistent training results. 

This is where AR comes to the rescue: in addition to the static picture the SnookerCoach app allows the player to scan the table and then visually project the ball on the table. The player can simply take a real ball and put it exactly where it is supposed to be!

Let's see it in action, a user's point of view

 

  • The user browses though the different practice routine cards
  • The user decides on a specific practice routing and reads description and looks a the picture
  • He/she presses the "view in AR button"
  • The AR mode is activated
  • The app asks the user to scan the bed of the snooker table
  • The app asks the user to mark the black spot (this is the place where black ball is placed, its a standard in the snooker game)
  • The app asks the user to mark the pink spot (same here)
  • The app now projects the practice routine in AR on the table
  • It's that easy!

On a technical note

Firebase

 

The app heavily integrates with Google's Firebase platform. For the AR functionality 2 modules are of interest:

  1. Firebase Realtime Database is used to store the different data regarding practice routines: title, description... but also whether or not this particular routine actually has AR.
    Some simple routines don't need AR and thus don't have AR modelling, for cost reasons. Depending on wether AR is supported, the "view in AR" button is shown or hidden.
  2. Firebase Storage is used to store the 3D model information for each practice routine. When the user selects to view in AR, the app downloads the model in realtime from Firebase and then renders it on screen. This means that the actual shown scene is completely dynamic and not embedded in the app.

Models

 

AR requires 3D models that are placed using the AR technology in the "real world". Having these 3d models is a real challenge. Luckily in our case the task was greatly simplified because of the following:

  1. We don't need the complete snooker table, just the bed of the table (otherwise the "pockets" would be a real challenge )
  2. Dimensions of a snooker table are fixed (part of the rules of the game)
  3. Radius of all snooker balls is the same, all colours are standardised also as part of the rules of the game.

Putting it together, we used sketchapp (certainly not the most powerful 3D modelling tool, but easy to use and it works fast for our template approach). In sketchapp we defined template models for all coloured balls and snooker table to the exact dimensions.

Special interest is going to "place markers" and "ball-running-lines" because these objects are going to be manipulated by the app at runtime. This requires that these can be identified by the app; we used naming conventions inside sketchapp to achieve this.

When this was done, using the template, we created for each practice routine the 3D model, exported it to the required format (for loading into the mobile app) and uploaded them to Firebase.

Mapping the model into the real-world table

 

  1. Ask the user to "scan the bed of the table". This allows the AR technology to detect the horizontal plane of the snooker table. Once detected, this is then visualised by a checkerboard pattern.
  2. Ask the user to select the black and pink spot.
  3. The app calculates the distance between the black and pink spot and scales the 3D model accordingly. This accommodates for small differences in scale between the ideaal snooker table and the actual snooker table.

    The app uses the black and pink spot to figure out the real-world orientation of the snooker table then rotates the 3D model accordingly.

Mapping the 3D model exactly on the real-world table is a real challenge. We want to keep the app simple to use and we need a high degree of accuracy.

The solution that was devised is both elegant and easy to understand :

Dynamic elements in the 3D models

The app automatically detects 2 types of dynamic elements in the model:

  • "place markers". Displayed as a billboard hovering above some balls. The app animates these so that they always point to the user (when walking around the table) and thus are easily readable.
  • "ball-running-lines". Displayed as dashed lines, but animated with a fade-in-fade-out animation to make them more apparent to the user.

To find these elements, the app crawls through the 3D model, and finds them based on naming conventions of the objects inside.

Get more tech news and updates from The Reference delivered to your inbox by signing up for our monthly newsletter.

SIGN UP TO OUR NEWSLETTER

Don't miss out

top
It's more than digital, it's your business
The Reference is nothing without its customers. Melexis is the stock market-listed global player in the semi-conductor and sensors industry for whom we facilitated future company growth by updating the brand, building the completely new corporate website and giving shape to the use of online channels. Read more about this client.