What is the future of augmented reality for consumer market?

Augmented reality fidget spinners, a fading Pokemon Go craze, and user misconceptions. How AR startups are doing today and what awaits one of the most promising industries after 25 years of development? Emerline team tried to find out.

Great expectations

Each time a new technology comes along, new designers make the same horrible mistakes as their predecessors. Technologists are not noted for learning from the errors of the past. They look forward, not behind, so they repeat the same problems over and over again... The most egregious failures always come from the developers of the most recent technologies.

Donald Norman, The Design of Everyday Things

AR and VR investment-wise has grown 140% in 2016. It’s a phenomenal value if you consider that in absolute numbers it means a leap from $680M to $1800M. Investors and analytics eagerly await the end of this year to see whether the explosive growth of the market continues.

And we think it will. Despite a very visual lack of mass-market consumer devices that could reveal the whole potential of augmented reality right now, people still wait with a mounting impatience for affordable AR that doesn’t come.

It didn’t come in 2013 when Google Glass was released. It won’t come in 2017. When, then?

The next big thing around the corner

Tech media proclaims AR as the next big thing. Sure, but after 6 or 7 years of listening to the same message the general public is tired of waiting.

Don’t get us wrong. We want AR and VR to succeed. We all do. The sentiment is fairly uniform right now. Just compare the relatively recent attendance figures of the major VR and AR forums and conferences. These events attract massive crowds. They attract congregations so large it’s hard to believe there’s actually very little in terms of real, consumer-ready products out there that could justify the interest.

We don’t say the whole industry is living a huge pipe dream. Not at all. It’s just… too excited for its own sake. The progress we have been promised and the products we were shown years ago are still crude and rather limited.

VR headsets are bulky and dizzying, AR glasses are slow to come to market and don’t offer a lot in terms of functionality. Most of them are not available to buy.

Automotive-based AR is one of the few places where things actually happen.

The “Wow!” effect is spent

After 25 years AR technology is still extremely early in development. And in terms of consumer-ready application it seems to be in the cradle right now - since the absolute majority of AR products are either in early prototype phase or offer limited and, frankly, most of the time gimmicky functionality.

With a lot of half-baked ideas on the table it’s understandable why a regular consumer may not be as excited about AR and VR future as it used to be in early 2016. So is AR a bit too early in development to find practical use in the world?

In an effort to find reassurance in AR progress and real-life application we’ve interviewed 3 industry experts leading the teams behind successful AR products.

Visible and tangible sound, through AR

We talked to CEO of Noiseless Acoustics, Kai Saksela, who leads a Finnish startup team that now enters construction and power grid industries with a product ready to help engineers make buildings and electric lines safer and... quieter.

A solution without a problem

Kai, an expert acoustician, together with his cofounder, Jonas, have invented a portable acoustic camera. The first one in the world.

In 2015, after 2 long years of work in his parents’ garage, Kai managed to build a working prototype. Next year, in 2016, his invention got noticed at a local Finnish startup event and shortly after that media attention followed.

Although the new acoustic camera was really impressive, it remained for some time just an interesting device with a very limited purpose. The camera could separate and visualize any sound, even some that we as humans can’t hear. But putting it to good use turned out to be a challenge.

It’s been a continuous challenge for us to find base use cases for our products. Most of our use cases were found by accident. Take for example the coil whine story.

My cofounder Junas lives close to the forest where there’s a bunch of wires going through. One day he was walking to work with the acoustic camera in hand and decided to point it to the wires when he heard some rattling noises from up there.

It worked really well, the source of the noise was clearly visible on the screen. And then gradually as we were using our camera more and more, we discovered that we could visualize a lot of sounds from places where you couldn’t hear anything.

We went through a lot of trial and error territory before we found most of our applications.
Kai Saksela, CEO of Noiseless Acoustics

Today Noiseless Acoustics’ camera uses powerful analysis to separate sounds from one another and categorize them: gas leakages, partial discharges, and poorly insulated areas in buildings.

This new functionality means that any person with basic training can use the camera in a professional setting to easily locate gas tank leaks, partial electric discharges in the grid infrastructure, and poorly insulated areas in buildings.

Augmented reality as an industry-enabling technology

The ability to fully understand sound is the next huge step for Noiseless Acoustics team in improving their product. Kai hopes that their AR-powered camera will allow non-acousticians to clearly see what each sound means, which sound is important and which is not, and what they need to do about it.

The team is ready to implement the solution into AR goggles. But Kai underlines that they are not an AR company in the traditional sense and only use AR to present a strong solution for the set of problems they see with the sound today.

A tourist app developed by scientists

We also talked to Piero Fraternali, full professor of web technologies in the Polytechnic University of Milan, the largest technical university in Italy.

His team is involved in research of water shortage and regional water availability in different European countries: during summers most countries in Southern Europe are experiencing a tremendous drought. Piero says Europe is now experiencing extreme summer drought conditions, induced by climate change.

One of the biggest challenges of Piero’s research team is lack of large amounts of high quality data, with sufficient spatial and temporal coverage.

His team eventually got access to over 2,000 webcams mounted all over the Alpine slopes. Alpine region is serving more than 100 million people with potable water. Even though the cameras are installed there for touristic purposes they frame the mountains in such a way that scientists can easily analyze how much snow is covering the peaks.

This was Piero’s research team starting point. They could now analyze very simple content and extract from it useful information that scientists could use in concert with more sophisticated data from satellites and other traditional means of monitoring nature, such as ground stations.

Piero’s next step was to find a way to entice people to take pictures of mountains and share them with the research team. This would allow them to monitor more environmental aspects -- for example, the availability of snow in certain critical places or the status of glaciers.

“What is that mountain in front of me?”

We knew that our application had to bring true value to the users. The idea lied on the surface. When a researcher goes trekking they would stand in front of a really beautiful panorama often wondering “What is that mountain in front of me?”

So the idea was on the surface! Everybody with an interest in an outdoor activity would like to know more about what they are looking at. And we built an app that helps people identify the contents of an image they are seeing right now.

This is how PeakLens was born.

Piero Fraternali, head of web technologies research group at Milan University

But there are hundreds of apps that identify mountains. So PeakLens had to be extraordinary to gain a large user base. That’s why the team of developers had to use a unique approach to achieve that.

A space-powered AR app

Other apps take the phone’s GPS location and compass value and then try to guess on a static camera shot all the peaks that are present in the picture.

PeakLens analyzes peaks in real time. It takes the phone’s GPS location, analyzes the compass reading, and uses the accelerometer and gyroscope in order to understand what you are pointing your device to.

Then it takes a camera frame -- and here lies PeakLens' big difference. It compares each frame with the virtual skyline PeakLens extracts from a commonly available server that stores a 3D model of the Earth, called a DEM (Digital Elevation Model).

All these services come from multiple sources, including the exploration missions that the Shuttle has done. PeakLens compares in real time what you see with a virtual representation of what you would see from that position at that angle computed from the DEM data.

All this creates a large combination of sensor readings that go into building a digital skyline of the scenery you point at. Then PeakLens uses the AI and computer vision components to identify what's in front of your phone. In real time.

Augmented reality as a research driver

PeakLens is unique in using true AR technology to involve regular users in citizen science-powered project demonstrating true potential of augmented reality as an enabler of new research opportunities.

Night starry sky

We met with San Shepherd, cofounder of Escapist Games and one of the creators of the Star Chart app, to learn more about how they applied AR to education and entertainment.

Star Chart boasts over 30 million active users throughout 15 platforms, offering both AR and VR experience.

Since 2010 the app has come a long way. 7 years ago, when San Shepherd and Chris Walley left Electronic Arts to pursue the career of independent game developers they couldn’t dream that Star Chart would become the best AR-powered educational app for students studying astronomy and the night sky, with over 1 million downloads by educational institutions.

Add artwork to taste

Despite good quality and high educational value, the app did poorly at the release. It sold only a couple of copies a day. The surge in downloads and the following media attention happened only when Escapist Games decided to add the constellation artwork.

The sales snowballed. In the summertime, 6 months later, Star Chart was selling 1000 copies a day.

This fact convinced the team that users pay a lot of attention to the visuals and are happy to see them even in a purely educational setting. Looking at Star Chart today we can see that the Escapist Games took this lesson to heart. The app is full of beautiful pictures and takes every possibility to show it off.

A fragmented solution

Building Star Chart for 15 augmented and virtual reality platforms, San’s team encountered one of the most common problems that plagues modern devices today -- fragmentation. Both Android and VR devices require a lot of time and resources for continued development.

To deal with fragmentation issue efficiently, the team had to switch from Objective-C to Unity engine. San admits that it’s too expensive to keep so many versions updated, especially with the rate at which devices enter the market and then go obsolete.

Virtual and augmented reality as an educational tool

Star Chart has been expanding throughout these years from an AR-based app into a VR product. Partly because, San explains, he views VR as a more perspective technology in the short term.

One of the reasons – and it’s a common complaint we’ve heard from AR developers – is that there is no good hardware that could make AR convenient to use. Smartphones can’t rival even first-generation VR headsets in ease of use.

San believes AR potential will be stalled until a better hardware platform opens new possibilities for software developers. He added that instead of mistakenly studying Pokemon Go as a staple of AR success people should take a closer look at real augmented reality products that leverage full potential of the technology.

Conclusion

Even after so many years of development augmented reality remains a complex technology to use effectively in consumer products. Good consumer-ready AR products are rare and far between. It doesn't help that the general public sees Pokemon Go as the biggest success of AR in the recent time, either.

This leads to an interesting situation where the most successful AR products are those that have found the right balance between user expectations and the limitations of AR hardware.

And as we can see from talking to our interviewees sometimes the best way to find the right product/market fit for an AR product is thinking outside the box.

In case with Noiseless Acoustics and their AR acoustic camera it was a random discovery that cable whine and other sounds inaudible to human ears can lead to a valuable application.

In case of PeakLens the creators found an interesting possibility to involve people in citizen science by offering tangible value through a seemingly unrelated product.

When we look at Star Chart, one of the main reasons for their success and popularity among similar apps has come through creating engaging visuals that present a more appealing look of the night sky, delivering value both to astronomy students and regular users.

Despite that optimism, we’ve also outlined a number of challenges that seem the most prominent today in consumer-oriented AR:

• mediocre sensors lead to poor user experience;
• users lack understanding of what AR really is and how it works;
• being trailblazers in a new industry is hard and involves a lot of experimentation, trial and error;
• device fragmentation significantly slows down development and reduces user experience;
• a multitude of bad products on the market alienates some users from all AR tech.

We believe that some of these problems will disappear once augmented reality matures past a certain point, but right now even some VCs don’t necessarily understand what AR is and how it can be used best in the business world.

Even the people that write reports for investors are citing Pokemon Go as the biggest technology advance in the last 10 years. Those people are smoking crack, the real technology advances are going to be coming in the next 5 years and they are going to be of a completely different nature.

Teaching consumers what to expect from AR, what to hope for, what to look for -- and simultaneously trying to find the right hardware that delivers that experience for the right price and with the right features – is going to be the biggest challenge for AR.

I think that potentially we will be going through the trough of disillusionment of AR just as we have with VR and with some other things. But at the end of it we will be seeing a huge growth and revenue potential during the next 5-15 years.
San Shepherd, cofounder of Escapist Games