Meet Benjamin, API Engineering Team Lead at SQUARS

What does AR mean to you and what made you decide to work in this field?

In my opinion, AR is one of the most cutting-edge technologies to date when it comes to merging the real with the digital world. In recent years, we’ve been able to observe the trend of everything, including ourselves, drifting away from reality and moving toward a fully virtual space. I strongly believe that with AR, this gap that has arisen between the virtual and the real can be closed once again, which will make us feel less detached from the actual world that surrounds us.

What makes AR particularly interesting to me as a researcher is the fact that the technology is still developing and there’s still massive room for improvement. In fact, much of the progress made within the field of the AR user experience is dependent on a basic trial and error scheme, which, admittedly, can be quite time-consuming, but also extremely rewarding every time an achievement is made.

Sometimes I catch myself getting a bit ahead of time and having these lofty ideas about all the beautiful things AR can potentially do in the future, but the reality is that in order to get there, much research still needs to be done on how to enhance the tools that are currently available to us.

Merging the real with the virtual world is also a very challenging thing to do: while the virtual world is an artificially created space that’s easy to alter and control, reality is, and always will be, unpredictable.

What is your personal history with AR?

Interestingly enough, my personal gateway into the whole world of AR was VR. During my studies, I mostly focused on VR and spent much time being fully immersed in the virtual world. However, I couldn’t shake off the feeling that the hardware I was using for my VR experiences was isolating me from the real world and what actually mattered. Moving people and experiences from all across the world over into one virtual space may sound fascinating on paper, but to me personally, VR seems to be rather antisocial in its application, as you would always need to wear a headset or some other sort of hardware that detaches you from the room you’re in and the people around you.

With AR, on the other hand, this is a lot different. We can now share the same virtual experience in the same physical space by projecting an immersive 3D visualization onto the real world. Compared to VR, AR is way more accessible as all the end-user needs to access the experience is their mobile phone. This means the hardware is already there.

Where do you still see challenges for AR at the moment?

Even though VR is unquestionably able to provide a more immersive experience than AR, the developments so far in the AR hardware sector have already been astonishing, particularly in terms of quality and pricing. Still, some aspects remain challenging: increasing the field of view (FOV) of the device so it matches the natural FOV of the human eye, and reducing the latency of the virtual experience so that when you move your head, the resulting changes are displayed immediately. Allow me to give you some technical facts: should there be a latency of 20 ms or above - which, if you think about it, really isn’t a lot - this will already cause the wearer of the device to feel nauseous even though their brain may not actively register the delay.

Is this also something you’re working on at SQUARS, or what exactly are your areas of responsibility?

Indirectly, yes. I work on optimizing the processing speed of our framework and on reducing the overall latency so we can ensure a smooth application of our product and provide a great user experience.

Another part of my work consists in making sure that SQUARS is compatible with other platforms and hardware such as different phone models, smart glasses, and all the nice hardware that’s currently in development and will enter the market in the years to come.

Even though the performance quality of many of today’s smart phones is already excellent, mobile devices still have very limited computational power compared to desktop PCs, which can potentially hinder a good AR experience. At SQUARS, we are trying to optimize our services for both high-end and low-end hardware and to include as many phone models as possible.

Which technical problems are you trying to solve with your work?

As said in the beginning, the high level problem we’re trying to solve is figuring out how to connect the virtual with the real world.

To be more specific, what we’re focusing on is object-centric target tracking. For this to work, we need to take clever approaches in order to efficiently describe to our algorithms what an object is. The first gap we need to close in this regard is the differentiation between the 3D world and the 2D image. As humans, we have the ability to view the world in 3D. This 3D effect is only possible because we have 2 eyes. The fact that low-end devices have only 1 camera at their disposal complicates things for us AR engineers.

To put it in a nutshell: The greatest challenge we need to overcome is closing the gap between a 3D world and a 2D image.

What do you particularly enjoy about your work?

What I really like about my work is how multifaceted and hands-on it can be. Sometimes I need to step out of the coding environment and interact with physical objects in my surrounding, like a box on my desk, to then use them for my work and bridge the gap between the virtual and the real world.

What else would you like to add?

Technology-wise, there’re a lot of possibilities that lie in front of us, some of which we might not even be fully aware of yet. AR is a journey, and overcoming all the challenges along that journey won’t be easy, but that’s exactly what makes it fun for me, and why I can encourage everyone to work in the field of AR - especially here at SQUARS. Right now, SQUARS is only at the beginning of a gigantic ecosystem that is slowly but steadily emerging in the backdrop.

If we look into the future, there’s no turning back from where we’re right now in terms of integrating the digital space into our lives. Soon we’ll even be at a point where it’s our lives that are being integrated into the digital sphere, which is basically a whole shift of paradigm. We’re already breaking out of the boundaries of our screens, we’re escaping the limitations of our interface devices - in other words, we’re more and more removing all the hardware that, so far, has been preventing us from fully merging the digital with the real. That’s a development I really enjoy seeing unfold in front of my eyes, and I can’t wait to see where it’ll ultimately lead us.