Change language:English
English
Português
- image: HeritageCARE project
The European project HeritageCARE is part of the Interreg-SUDOE initiative and aim at implementing a sustainable and integrated methodology to preserve and maintain our historical and cultural heritage. Part of this initiative, we are investigating to find what virtual and augmented reality can do for us.
Let us focus first on virtual reality (VR) and understand what it is. VR is an artificial environment that is created with software and presented to the user in such a way that he suspends belief and accepts it as a real environment. The last part is the most important; immersion is the key to define VR. This immersion factor is what leads VR to be such a good medium when it comes to training, education or interactive story. Therefore, we can build digital copy of buildings to allow anyone to see it from any place at any time. This could lead to a new dimension to share our heritage.
Therefore, we can build digital copy of buildings to allow anyone to see it from any place at any time. This could lead to a new dimension to share our heritage.
On top of the recreational aspect of being able to visit those monuments from the comfort of our home there is also preservation at stake. Indeed historical places are destroyed on a regular basis due to natural causes such as time, climate change, floods, earthquakes but also human activity like war, arson and urban sprawls. VR will not stop those unfortunate actions from happening but will allow visiting those buildings even after such fate happened. Companies like CyArk[1] have been created towards this unique goal of creating buildings’ digital copy before they receive any severe injury.
The principle to generate 3D model of a building is a mature process and lead to satisfactory result. First, a point cloud acquisition of the building is performed usually using a combination of time of flight scanner, robust at long distance and to detect plan, and photogrammetry, good at detecting edges and corner. From this point cloud a mesh as to be created using triangulation technics to connect the dots. Then texture is added to the model by superimposing photo of the building over the mesh[2]. Finally, the 3D object can be integrated to a VR engine, which add virtual lights, movement and interactivity to complete the VR experience.
image 2: process used to digitalize buildings
Now, let us shift our interest towards augmented reality (AR) and more generally towards mixed reality (MR). AR is the ability to display additional contextualized information on top of the real environment. However, the experience can be pushed further by allowing real world interaction to affect virtual object this is why the term mixed reality started to emerge since there is a cohabitation between the real and virtual environment.
image 3: Reality–virtuality continuum
For the HeritageCARE project, Microsoft HoloLens will be used to generate MR experience, which is the top players in its category. For a long time MR where mostly used through cameras of our portable devices such as phones and tablets but true MR is to be experienced through glasses where your vision of the world is not a video and your hands remain free. On top of that, the HoloLens bring voice and simple gesture recognition, which makes it an ideal hardware for our needs. However, it is important to remember we are in presence of the first development version. Therefore, limitations are to take into consideration, such as small field of view, limited battery life and impossibility to see virtual object close up. All of that will undoubtedly getting better in the future generations.
image 4. HoloLens: Microsoft Press tool pictures.
MR will allow the user to superimpose the virtual mesh to the real building, allowing direct interaction with each parametric object of the BIM on the fly…
Main use case of the HoloLens for the HeritageCARE project is to ease and increase efficiency of building inspectors. Especially at the service management level 3 where the Building Information Modelling (BIM) will be used. MR will allow the user to superimpose the virtual mesh to the real building, allowing direct interaction with each parametric object of the BIM on the fly, accessing past inspection data and saving the new one. In addition, if small maintenance tasks has to be performed, AR assistance can be develop for simple task and in case additional knowledge is required, AR remote assistance may be used. This will allow an experimented user to connect to the HoloLens to see inspectors’ vision and display contextual information to help them completing their task.
All of this is exciting and I am sure we will discover new way of helping inspectors and owners in our quest of preserving and maintaining our historical and cultural heritage. If you are interested in the HeritageCARE initiative, do not hesitate to follow and share our Facebook page or visit our website (under construction) to follow our progress.
Adrien Fonnet
Adrien Fonnet is graduated from the French engineering school, Arts et Metiers, and a master research in 3d visualization. Currently working as part of the CCG/cVIG team, in particular on the HeritageCARE initiative. In parallel, he is conducting a thesis on Augmented Reality Smart Assistance part of the MAPi doctoral program in computer science. His interest focus on augmented reality, virtual reality and human computer interaction.
[1] http://www.cyark.org/
[2] MURPHY, Maurice, MCGOVERN, Eugene, et PAVIA, Sara. Historic building information modelling (HBIM). Structural Survey, 2009, vol. 27, no 4, p. 311-327.
