When GPS was invented in 1978 and made available to the public in the following decade, navigation changed profoundly. As it guides automobiles, airplanes, and ships at sea, GPS has had a tremendous impact on how we get around. However, there are still challenges to be overcome. How can augmented reality be used to guide users in indoor and outdoor environments?
Augmented reality allows for powerful and intuitive solutions in navigation. By showing virtual guides in physical space through the view of a smartphone or headset, users can be directed point to point more naturally than by comparing a map to their immediate surroundings. Due to this great benefit, AR navigation can help in both indoor and outdoor environments.
Augmented reality development is easy to get into, but difficult to master. With the demand growing for more complex and higher quality AR software products, many organizations are shifting toward more specialized solutions to suit their needs. AR indoor and outdoor navigation are tasks that require a more specialized solution based on the environment at hand, requiring special hardware, technology, and expertise to implement successfully.
It’s important for entrepreneurs, business owners, project managers, CTOs, and CIOs to understand the breadth of this industry and how it can best accomplish their specific objectives. In this guide, we will show you exactly how augmented reality navigation systems work and how you can put these systems into action for your users.
Overview of Navigation Technology
Before we can get into augmented reality navigation, let’s talk first about some important technologies in navigation. Each technology has benefits and drawbacks depending on whether it is used for indoor or outdoor navigation.
WHY GPS ISN’T ENOUGH
The global positioning system (GPS) may have fundamentally changed how we traverse the Earth, but it has serious limitations for certain use-cases. Even though the United States Department of Defense allowed increased precision of GPS in 2000, GPS still is not 100% precise in all situations, especially indoors.
GPS is made possible with satellites and clever exploitation of the Doppler effect. Ground stations use radio signals to determine where GPS satellites are in orbit around the Earth. Your cell phone or car detects signals from these satellites and determines the distance of four or more satellites.
Although the precision that this technology offers is incredible, it has limitations. If for any reason these radio signals are blocked, pinpointing your location is significantly more difficult or impossible. Water, dirt, walls, tall city buildings, and other obstructions can disrupt GPS reception significantly. Accuracy on its own can also be limited by a number of factors and errors. If we need to determine someone’s position inside a building or in an exact position in a dense downtown area, GPS can only do so much to support AR navigation.
BLE BEACONS: THE IOT APPROACH TO NAVIGATION
Bluetooth low energy (BLE) beacons function similarly to lighthouses. These IoT devices, often used in smart city solutions, broadcast unique radio signals to BLE-enabled devices like smartphones. These beacons do not necessarily know what devices receive this information and cannot communicate with those devices.
The packets of data that these beacons transmit may contain information other than its ID. It may also contain sensor data that the beacon is programmed to measure. When this information is received by a smartphone or other device, it can take some action based on the information from the beacon.
BLE beacons present a variety of applications such as identifying points of interest to users and tracking moving objects like vehicles and cargo. However, some promising applications are indoor positioning systems and real time locating systems (RTLS). Indoor positioning system beacons help devices locate their positions within buildings by measuring signals from these beacons. Conversely, RTLS works by placing beacons on moving objects and having them transmit signals to localizing edge devices.
BLE beacons offer much more precision for AR-based navigation solutions, especially AR indoor navigation technology. However, this infrastructure does need to be present in order for it to succeed.
VISUAL POSITIONING SYSTEM (VPS) – NAVIGATION POWERED BY AI
This innovative technology mimics how we determine our location in the real world. Just as we search for landmarks with our eyes, visual positioning systems (VPS) can use your smartphone’s camera to analyze your surroundings and determine your location.
One of the most notable applications of VPS is used in Google Maps. By comparing camera data with extensive Street View databases, Google can determine your location and give you on-screen directions. This is particularly useful in dense urban areas where skyscrapers obstruct GPS signals. This is all powered by artificial intelligence analysis and machine learning. However, unlike Apple ARKit’s location anchors, the APIs for this solution are not yet available. They may become available in the future.
There are a few other technologies for AR-based navigation that we will talk about next.
AR Navigation: How It Works
Augmented reality navigation is an innovative solution that incorporates the above technologies for indoor and outdoor solutions. The primary goal of this technology is to provide directions to users on screen overlaid on top of real environments seen through the camera of a device like a smartphone or headset. This alleviates the challenge of comparing the real world against a reference like a map which is more difficult for users to use while navigating.
Thus, there are two parts to AR navigation, the first being the actual navigation and localization, and then the display of AR directions as text, arrows, and paths on the screen. This second step is actually the easiest part of the whole process, with the more challenging element being identifying the user’s position.
In outdoor settings this is generally simple unless there are interfering obstacles as described earlier. However, indoor environments generally require more powerful infrastructure like BLE beacons.
There are two development platforms primarily used for AR development on mobile devices: ARKit for iOS and ARCore for Android. These development environments are responsible for translating device movement and sensor analysis into information to show a renderer know how to draw virtual objects on the screen so that they appear to reside in the real world.
ARKit: AR Navigation on iPhone and iPad
ARKit is often regarded as the more powerful of these two powerful mobile AR platforms. However, both perform scene analysis using almost identical methods. If that’s so, why is ARKit more powerful for AR navigation solutions?
ARKit and ARCore may use the same methods, but ARKit is backed up by much more reliable hardware and software. Apple has full control over the production and design of its iPhone and iPad hardware as well as the OS software. Because of this, ARKit’s performance is more optimized. Its performance is more reliable because there is very little diversity in performance among iPhones and iPads.
Meanwhile, ARCore devices on Android phones are much more inconsistent because there is a great deal of hardware and software diversity between one Android phone to another. Since there are multiple manufacturers, it is more challenging for Google to come up with a consistent, reliable experience.
One of the key features that puts iPhones above Android devices in the AR arena is the LiDAR sensor. This hardware can make AR navigation easier due to its superior depth sensing capabilities, enabling ARKit to analyze depth in a scene at unprecedented speeds. This allows for quicker processing of AR experiences with higher accuracy.
ARKIT LOCATION ANCHORS
ARKit 4 introduced location anchors in 2020. This technology utilizes VPS to identify landmarks such as buildings and compares them against Apple Maps Look Around images which are similar to Google Street View pictures. This feature allows developers to affix objects to specific locations rather than in arbitrary locations. This can be helpful for helping users identify landmarks around them with AR navigation and operates with higher accuracy than simply using GPS data on its own. Most interestingly, location matching occurs locally which is good news for users concerned about privacy.
Location anchors can make it easier for users to navigate dense downtown areas where GPS may be obstructed by skyscrapers. In the future, VPS-based technologies can be used to improve indoor navigation accuracy in conjunction with BLE beacons.
It’s important to note that the API for ARKit’s location anchors is available while Google’s AR visual navigation API is not. Although Google may provide these APIs eventually for ARCore navigation software development, ARKit is the best option for now.
ARCore: AR Navigation on Android Smartphones
Although ARCore may not be as powerful as ARKit due to current hardware limitations and inconsistencies, it is still an incredibly valuable platform. In 2020 there were 129.1 million Android users in the USA, roughly a third of the population. Developing AR navigation solutions for this audience is critical for remaining competitive.
As a more specific example of what can be accomplished with ARCore (and ARKit as well) is an application in indoor navigation. Our ARCore indoor navigation demo video shows how users can be navigated throughout a building using on-screen AR instructions.
Overcoming Challenges of Indoor Positioning
AR solutions powered by indoor positioning systems (IPS) have a lot of potential to make waves in various industries. However, unlike outdoor navigation systems, they face many barriers to acceptance. The most challenging part about IPS is determining the user’s location. Let’s talk about a few examples and solutions to get around these issues.
VISUAL MARKERS & APP PERFORMANCE
Visual markers are helpful for ensuring IPS accuracy. However, each marker has its own unique ID. When a marker is seen, it is compared against the entire list of markers to find a match. Too many markers can cause a decrease in performance, as this comparison stage can take longer and longer with more and more markers.
This is especially the case if there are thousands of markers spread across multiple locations that use the same app for navigation. For example, a supermarket chain attempting to implement an indoor navigation solution might use markers to help their customers find products in their stores. When thousands of markers exist across multiple different stores, it can slow down the performance of their app. However, GPS can be used to filter markers down to only search through the markers at the store the customer is shopping at.
However, there is another unavoidable problem with the use of visual markers. Business owners may not approve of the way these markers affect the appearance of their interior design. With many businesses having strict interior design standards, visual markers may not be the best solution in all cases for AR indoor navigation.
WI-FI RTT
A powerful alternative to visual markers for AR indoor navigation, Wi-Fi Round-trip time (RTT) is effectively very similar to BLE beacon technology. However, it can use existing Wi-Fi router infrastructure to determine a device’s location with an accuracy potentially within one meter. This is important for AR indoor navigation especially because with the visual marker method the user must keep their camera active for the entire session to maintain the best accuracy possible.
Wi-Fi RTT is much more effective than traditional Wi-Fi positioning systems (WPS). Traditional methods that create Wi-Fi fingerprints of the building are accurate between 5 and 15 meters. However, Wi-Fi RTT technology, so long as three or more routers are involved, uses a multilateration algorithm. This enables the position to be accurate between 1 and 2 meters.
ULTRA WIDEBAND (UWB)
This technology has great promise to improve indoor positioning and AR navigation apps. Thanks to ultra wideband’s ability to transmit large amounts of data at low power, UWB can be an efficient IPS solution. By using time difference of arrival (TODA), UWB transmitters on items to be tracked can localize their location.
This technology is ideal for tracking items like packages in warehouses, but it also holds promise to be useful for tracking the location of smartphones and other devices to help users find their way around buildings with AR applications. However, despite its incredible accuracy, UWB technology cannot penetrate walls and other obstacles.
The APIs for UWB are not yet available, but we expect that they will become available in the future. This will result in a dramatic increase in accuracy for indoor positioning systems if used correctly.
How to Develop an AR Navigation App
As we’ve discussed, the challenge of creating AR navigation apps is mainly with understanding the current position of the device. Routing and rendering AR directions on a user’s screen is the easy part. Let’s talk about how to develop an AR navigation app with these elements in mind.
UNDERSTAND PROJECT FLOW
Before we can get into the details of what separates indoor and outdoor AR navigation app development, we need to understand how project flow works. You must make certain that the team that you choose to construct the app has iOS/Android developers with experience working with ARKit and ARCore. You will also need UI/UX designers, a project manager, and a QA engineer.
During the pre-contract stage, you will explain to the software development company what your requirements are for the project. This is where brainstorming solutions happen and where a technical strategy will emerge.
Once this is complete, the business and technical analysis stage will determine a vision for the AR navigation app. This will consider constraints of the project such as deadlines, budget, technology, and corporate environment. While the business analysts and project managers work with stakeholders to create this vision, technical analysts will take stock of the available tools for development in order to narrow down the technology stack.
The design stage is where the AR app takes visual form. This typically deals with sketching, wireframing, and mock-ups. 3D object creation and mapping is also part of this stage.
The development stage begins with a proof of concept (PoC), a minimum viable product that will test the project with minimum resources and time. This might include a hard-coded (or ‘mocked’) route and custom features like voice recognition. Developers may choose to build this PoC on only one development environment. Between ARKit and ARCore, ARKit may be the best at this stage due to its higher performance and consistency.
DEVELOPING OUTDOOR AR NAVIGATION APPS: GPS AND VPS
Outdoor AR navigation, depending on certain conditions, is generally easier to accomplish than indoor AR navigation. As discussed previously, more reliable technologies are available outdoors in most conditions like GPS. However, VPS systems like Apple’s ARKit location anchors and Google’s AR VPS solution for Google Maps take it a step further by analyzing visual content around the user.
VPS may not work as expected in rural environments due to the lack of consistent artificial features like buildings. It may be impossible to use in off road environments where Google Street View or Apple Maps Look Around data is not available.
If for some reason VPS and GPS are not reliable or available in a desired location, similar tactics that are used in indoor AR navigation systems may be helpful, such as BLE beacons.
Developing Indoor AR Navigation Apps
One of the challenges of indoor navigation is that it does not have access to the extensive satellite mapping data that outdoor navigation has with Google Maps and Apple Maps. Because of this, custom maps must be created for each building and indoor space. Quality and scalability are important factors that can determine the success of a map.
Once we have a map of the indoor space we want to navigate users through, we need to add metadata to define corridors and rooms to create a graph. Once this is done, we can analyze the surroundings to add visual markers. We can also use beacons, or Wi-Fi routers for Wi-Fi RTT.
This technology is only expected to improve. With ultra-wide band technologies set to be available soon, indoor navigation will become more precise and reliable than ever.
It’s important to ensure that the map matches the real world. If you are using visual markers, they need to be placed at specific points marked on the map. If they do not perfectly match, then positioning may be inaccurate.
Markers have to be first placed on the planned map, then place them in the real world. The AR app can scan the marker. The map will then have to be shifted to fine tune its accuracy. Finally, once it has been calibrated, the precise position of the marker can be uploaded to the cloud.
Fortunately, there are now many more alternatives to visual markers than there were several years ago. With Wi-FI RTT, UWB and BLE beacons available to developers, positions indoors can be tracked with a level of accuracy never seen before. For indoor environments, VPS need not be the primary form of AR navigation.
DRAWING ROUTES FOR INDOOR NAVIGATION
Rendering the route is the easy part, relatively speaking. However, there are some important considerations. Using graph theory to create directions with lines or arrows, users can be directed to their destination.
One of the main obstacles to overcome is occlusion. AR 3D layer content is placed on top of the real world on the user’s screen in most contexts. However, this can be confusing to look at if the lines are drawn on top of or through walls. It’s important to take into account occlusion so that users can be guided to their destinations more naturally.
Understanding size and distance is also a vital part of the process. Objects that are farther away appear smaller from the perspective of the user. The most important virtual object for this process is the destination pin. When the user can see the destination pin on their screen, it should appear larger when near and smaller when farther away.
VISUAL MARKERS: CONSIDERATIONS FOR INDOOR AR NAVIGATION
If you are using visual markers for your indoor navigation system, there are some important considerations to take into account:
- The more visual complexity an AR navigation marker has, the easier the program will be able to recognize it.
- Every marker must be unique, otherwise the program may mistake the marker for another.
- Asymmetry will help the program understand which side of the marker it is on.
- It’s important to create a database of the markers so that the development kit can understand the markers to be tracked.
Future of Indoor Navigation: Increasing Accuracy and Detail
GPS’s profound effects on navigation on our planet were only the beginning of the revolution. With new technologies on the rise such as BLE beacons, Wi-Fi RTT and UWB increasing location accuracy to levels never seen before, users can navigate areas with greater precision than GPS has ever had for the general public.
VPS solutions are also making it easier for AR navigation apps to render content in specific locations. With software solutions like ARKit location anchors and Google Maps VPS systems, users can navigate urban locations with on-screen directions with accuracy previously thought to be science fiction.
For indoor environments, Wi-Fi RTT, BLE beacons and UWB technology may mean that visual markers are no longer necessary. However, hybrid solutions between these technologies and visual markers may improve accuracy significantly. ARKit and ARCore have certain margins of error that a hybrid system may be able to overcome. This may revolutionize the way people find their way through crowded buildings, large supermarkets, and complex convention centers.
AR NAVIGATION: FINDING OUR WAY INTO THE FUTURE
Because of these technological improvements, AR navigation technology is set to experience tremendous growth, piggybacking off the success of growing technologies like ultra wideband connectivity.
Business owners, project managers, CTOs and CIOs face a lot of challenges when looking to implement AR navigation solutions into their businesses. However, understanding the possible use cases and technologies can be useful for determining ways in which AR navigation can integrate with existing systems and meet the needs of businesses and consumers. AR navigation is a fast growing market and remaining competitive is important for success.
Using AR navigation to help consumers find the products they need in a large supermarket or to help workers find specific items in large warehouses can be a gamechanger in improving sales and workplace efficiency. This technology also stands to improve overall customer and employee experience satisfaction. Taken together, AR navigation indoors and outdoors can improve the overall performance of a business.
Credit : Andrew Makarov
Head of Mobile Development