Although technology is becoming an increasingly important aspect of our lives, for many of us there’s a clear line between the physical world and the virtual world. Yet this is changing rapidly thanks to technologies like Augmented Reality, and Mixed Reality that are blurring the line between digital and physical at breakneck pace. As a technology enthusiast, I have been following these technologies and looking for opportunities to use them in a project of my own for quite some time now. Therefore, when I had the idea to use these technologies to take OSC’s yearbook to the next level at the start of this year, I immediately pitched the idea to Ms. Mora – the yearbook’s editor-in-chief – and joined the yearbook team to make my vision a reality.

The Concept

The OSC Yearbook is an annual print publication aimed at members of the OSC community that records and celebrates events that take place during the academic year. Likewise, Gecko Network is a service group at OSC that creates digital content like websites and videos to celebrate the very same events. Whilst both GeckoNet and the Yearbook creates content to meet the same purpose, the yearbook lives in the physical world whilst GeckoNet’s content live primarily in the digital world. The concept behind this project aims to use AR technology in order to combine the Yearbook’s physical content with GeckoNet’s digital content and in turn greatly improve the end user experience. Furthermore, in doing so especially GeckoNet will reach a much larger audience since the Yearbook is consumed by almost every single OSC community member at the end of the year.

The idea was to build a mobile app that would allow users to scan pages from the yearbook. Then a GeckoNet video would be played based on the information on the scanned page. For example, if a user scanned the page about “Welcome Back Picnic” on the Yearbook, then the app would redirect them to a video about “Welcome Back Picnic” that was produced by GeckoNet.

Development

Although this app may seem very simple and intuitive from an end-user’s perspective, developing it is quite complicated since delivering on this vision requires various pieces of hardware and software working together. Furthermore the development process would be much harder since I would have to build the apps natively for iOS and Android using Apple’s ARKit and Google’s ARCore since AR is still relatively novel and lacks a solid cross platform framework. Realizing this along with the fact that this was my first experience working with AR, I decided to first build a demo application that could scan and recognise the content on a page from last year’s yearbook. This would allow me to understand the limits of the technology, and clearly show whether this project would be feasible.

I got the demo application up and running quickly, but unfortunately the results were not good enough. Although the app would recognize pages themselves, it could not consistently distinguish between different pages very well perhaps because every yearbook page looks quite similar. Furthermore, I learnt that both ARKit and ARCore were only compatible with relatively new devices, which would make the AR yearbook inaccessible to quite a few people. Therefore, I decided to go with an alternative technology called QR codes that isn’t as new as AR, but is extremely reliable and would give users the same experience. The only difference would be that users would have to scan the QR code on the page using the app instead of scanning the entire page in order to view the digital or augmented content related to that page.

Although I could have just printed QR codes with the links to the videos on the yearbook, I decided to build my own app that would scan custom QR codes in order to tailor the experience as I saw fit and also to give the yearbook team the ability to change the video associated with each QR code even after the Yearbook was printed. So I got to work, first by building a very light Backend that included a database that ties each QR code to a video link and description. Thereafter, I built a simple yet elegant mobile app that reads the QR code and retrieves the video link/description by sending the information within the QR code to the Backend. Building this didn’t take too long, but I quickly ran into a roadblock yet again when trying to deploy these mobile apps.

The Android app was published on the Google Play store without a hitch, but the iOS app was quickly denied entry to the App Store since the app’s target audience was too limited according to Apple’s guidelines because the app was only applicable for people at OSC who bought a Yearbook. Despite this however, due to the versatility of the QR code technology I still had an option. Instead of storing a unique ID on the QR code that would tie it to a video, I stored a URL that went to a web app of my own and provided this unique ID as a parameter. A few lines of Javascript at the web app’s endpoint sent the parameter to the backend, retrieved the URL to the video and simply redirected the user to that video link. Therefore, iOS users could simply use the QR code scanner built into their Camera Apps in order to get the same experience and my problem was solved! Although any off-the-shelf QR code scanner would have worked, I decided to retain the Android App since it had already been built and gives users a better experience compared to these alternative Android QR code apps that often have ads, malware, etc. on them.

Putting it all together…

Once the technical aspects were complete, all I had to do was generate the QR codes that would be placed in the yearbook. Before I generated them, I came up with the design below so that the QR codes would match this year’s “vintage” theme and fit in with the other content on the Yearbook.

Now that all the development and preparation is complete, the yearbook will be going in for printing in the next few weeks. The response for this augmented feature will only be seen once the Yearbook gets into the readers’ hands.