Summary of activities and deliverables

Activities 2022

  • Up-to-date analysis of the state of development in the field of navigation solutions for blind people
  • Experiments with selected devices and configuration and localization algorithms
  • User requirements analysis
  • System design
  • Dissemination

Deliverables 2022

  • Report on the state of development in the field and the selected equipment and algorithms
  • Report on experiments with preliminary implementations of configuration and navigation modules, on selected hardware
  • User requirements document (URD)
  • Architecture design document (ADD)
  • 1 scientific paper - review of space configuration solutions for augmented reality applications
  • 1 scientific paper - review of sonification solutions in assistive systems for the blind

Summary of the scientific and technical achievements of the phase

The stage of development of the field and selected equipment and algorithms

In this activity, the latest assistive solutions for the blind were analyzed. For these, sonification methods were investigated, in order to choose the best solutions for transmitting navigation information. Considering the very important part of configuring spaces, during this activity we also analyzed the existing methods of configuring spaces in augmented reality applications.

Requirements Specification

This activity defined the functional and non-functional requirements of the VIP SpaceNav system. A configuration application intended for location administrators will allow defining or uploading a 2D/3D model of the building, configuring features for localisation and defining points of interest, and establishing navigation paths towards points of interest. Setting up locations will be done through a mobile app and a desktop app. The mobile application used by visually impaired people will run a localisation service and an obstacle detection module, providing navigation instructions from the current position to a selected point of interest, or facilitating free walking inside a building.

System Design

As part of the architectural design activity, the architecture was defined through a distribution diagram. The main components are:

  • Location map definition GUI (running on the administrator's desktop): allows space configuration through the following sub-components: 2D/3D map definition (location map definition), POIs definition - desktop GUI (points of interest definition in the desktop application), Pathfinding dataset definition - GUI (definition of the navigation graph to points of interest inside the building)
  • Location mapping (running on the administrator's mobile device): allows the configuration of the space through the sub-components: Position calibration (defining a link between the position in real space and the position on the map), Scanning & position calculation (calculating the phone's position and saving the location characteristics) , Anchor definition (definition of an anchor), POIs definition - mobile GUI (definition of points of interest in the mobile application).
  • The components on the CMS (server):
    • Localisation dataset (allows storage of anchor information for localisation)
    • Location map: allows storing information about the building map (2D/3D map component), points of interest (POIs database) and navigation paths (Pathfinding dataset)
  • The components running on the VIP user's mobile phone
    • Computer Vision: Calculates the user's position and detects obstacles in the environment
    • Guiding: Determines the minimum path from the user's current location to a point of interest, and generates guidance encodings to the destination
    • Alerts: Computes obstacle properties and generates audio/haptic encodings for the VIP
    • Menu: component that allows the VIP user to select a point of interest to be guided to, as well as settings regarding preferred audio/haptic encodings.
    • Audio-haptic rendering: the component that synthesizes the audio and haptic signals that will be rendered to the user through the headset/speaker and the haptic device (optional).
System architecture

Experiments with preliminary implementations

Following the state of the art analysis, a possible solution was selected, the use of cloud anchors (Google ARCore). Thus, a series of experiments were carried out to evaluate the localisation accuracy and ease of use of a system based on Cloud anchors.

In the first experiment, a single cloud anchor was placed in the scene and the localisation accuracy was tested when the anchor was not in the field of view of the user's mobile phone camera. In the second experiment, the possibility of using several cloud anchors was tested, in order to enable the localisation for a space with several rooms. In the third experiment, the possibility of making a connection between the real position of the user and the position in the virtual map of the building was tested, in order to guide the user from the current position to a point of interest.

After the experiments it was found that the use of cloud anchors has an acceptable accuracy (errors less than 10 cm).

Summary of dissemination activities

Publications:

  • Maria Anca Băluțoiu, Anca Morar, Andrei Lăpușteanu, Alin Moldoveanu, Florica Moldoveanu, “A Survey of Environment Configuration Solutions for Augmented Reality Applications”, submitted to PRESENCE: Virtual and Augmented Reality, ISSN: 1054-7460 - in review
  • Andrei Lapusteanu, Alin Moldoveanu, Anca Morar, Maria Anca Balutoiu, Florica Moldoveanu, “Sonification solutions in navigation systems for visually impaired people”, submitted to Displays, ISSN: 0141-9382 - in review