Mixed Reality for the Construction Industry

Extending Trimble’s Product Capabilities with Microsoft HoloLens

What is mixed reality?

Mixed Reality technology blends real world objects with digital content, interactively, and in real time. Together with holographic technology, it bring the models out of the screen and helps peolpe efficiently interpret physical and digital information, and the spatial relations between them. This article describes the benefits of integrating Microsoft HoloLens’ Mixed Reality technology with Trimble products, during key stages of the design- build - operate workflow.

About Microsoft HoloLens

Microsoft HoloLens is the first fully wireless, holographic computer that enables high-definition holograms to integrate with your world. The device features a see-through, holographic display and advanced sensors that map the physical environment. HoloLens allows users to place and interact with 3D holograms in the physical environment.
The projected holograms can appear life-like, and can move, be shaped, and change according to interaction with the user or the physical environment. Gestures, gaze and voice commands can be used to navigate and control the presented content.

Between reality and virtuality

Mixed Reality (MR) spans the purely virtual and purely real environments. In the context of the construction industry, this is the phase in which digital and real content co-exist, where architectural design collides with reality, and where construction teams transform digital content into physical objects.

In today’s practice, the interpretation of digital content and its translation to real world objects, heavily depend on the user’s spatial understanding even if advanced 3D modeling tools are used. Interpretation errors are common during the design and construction stages, and often result in poor quality, cost overruns and schedule delays.

Visualizing digital content as holograms in the context of the physical world bridges the gap between virtual and real, and eliminates some of the current workflow’s inefficiencies. Moreover, while our physical world is finite, Mixed Reality presents the opportunity for an infinite environment in which data such as schedule, specs, and simulation can be overlaid onto the world, creating a hyper-reality environment.

Holographic 3D visualization

The construction industry is spatial by definition. The last few years’ transition from 2D documents to Building Information Models was a natural evolution, which improved team communication and coordination. BIM solutions such as Tekla Structures are being widely used today for detailed design and fabrication, but still, the interaction with the volumetric data is being done on 2D screens.

Mixed Reality and holographic technology bring the models out of the screen and provide users the ability to engage and interact with design data more intuitively. Moreover, unleashing the 3D model democratizes the data by offering a natural way to experience and understand the design. While years of education and practice might train architects and structural engineers to visualize their designs in 3D, other stakeholders have a hard time deciphering them. Using Mixed Reality, even those stakeholders who do not have professional expertise in construction can walk around and explore the design in real 3D without the need for an expert to guide them and dictate their point of view.

The freedom to move along the reality - virtuality continuum is additional advantage of Mixed Reality, with a major impact on the efficiency along the design-build-operate workflow. For some decisions or specific tasks, a real “unmodelled” world is required as visual input. For other tasks, a mixed reality – a “world partially modeled” – or even a completely immersive Virtual Reality mode is ideal. With Mixed Reality the user controls the “mix” and adjusts it according to his task requirements.

Moving along the reality - virtuality continuum: in this proof of concept, a Trimble SketchUp model is displayed using Microsoft HoloLens. To best explore and experience the design, stakeholders can switch between a tabletop holographic view and an immersive view mode.

Mixed Reality and holographic technology also improve remote collaboration. The ability to share 3D holograms with remote stakeholders, including participants’ position and gaze, improves communication and collaboration efficiency.

Remote collaboration: in this proof of concept, Trimble Connect data is shared with remote users. Using Microsoft HoloLens’ holographic technology, the remote teams experience the ‘spatial presence’ of the design model, share views, position, and gaze information, and communicate over real 3D holograms.

Between mixed reality and digital mockups

The use of digital mocksups for evaluation of complex structures is a common practice today. Compared to physical mockups the digital mockup provides a faster, cheaper and more flexible solution. Using digital mockups stakeholders can effectively analyze multiple variation of the design ensuring quality and constructability. Mixed reality extends the capabilities of the digital mockup by enabling real 3D evaluation at scale and in context. 
Using devices such as Microosft HoloLens in combination with modeling and analysis applications, users will be able to visualize the structural design model in real space, apply forces and loads and observe the behavior of the structure under this action in fully perceived three-dimension. 

Seamless translation

In every construction project there is a decisive process, in which design is transformed into reality, abstract ideas are translated into physical objects. As almost every project is unique, site-specific and labor-based, intensive and accurate translation of the design information is required. Currently, turning design documents into reality is a constant struggle. It is a complex cognitive process in which wrong interpretation of data often leads to costly errors, quality issues, and rework. Digital 3D models reduce the level of abstraction, and help clarify the design intent, but, at the execution stage there is a clear boundary between documentation and the real world.
Mixed Reality technology blurs this line. With Mixed Reality, the information is superimposed on the physical environment. This minimizes the need for this translation and reduces subjective interpretation. The user visualizes the design in context, and can better understand, interact with and execute the required actions in real time. The immediate visual feedback also supports a more effective production control process. Overlaying design on as-build structures reveals any deviation and provides immediate feedback.

Extending the Digital Thread

Integration of business workflows and improved communication are clear goals in today’s construction industry. The silo structure and the need to re-create data - when moving from one stage to the other, are still sources for major inefficiencies. Project delivery methods like IPD and technology changes such as BIM support this trend and have proven to positively impact project quality, cost and schedule.
Mixed Reality technology supports integration and collaboration along three main axes:

The Project Stage Axis: Design to Build (and Operate):

Moving along this axis, Mixed Reality brings 3D data to life and puts information in the user’s hands without the need to change or adjust the data format. A sign-off BIM model can be projected in context to guide construction teams on-site, and later, by facility management teams when evaluating required changes or maintenance work.

The Digital to Physical Axis:

Holographic display is not limited to 3D models. In fact, using Microsoft HoloLens, multi-layered datasets can be overlaid as holograms on the real world. A partial list includes 2D documents, energy analysis, light simulation, acoustics, layout data, and equipment metadata. The abilities to integrate digital and physical content and present data in context improve communication and leverage confidence in decision-making.

Office to Field Axis: Design to Build, Build to Design

Sharing up-to-date design data with on-site teams is crucial in the dynamic construction environment. Using the Mixed Reality technology, construction, teams benefit from access to geo-located, context-based data, which can be projected and anchored to the physical environment around them. The ability to map the as-build physical environment supports communication from the field to the office and completes the Build to Design – Design to Build loop.

Microsoft HoloLens and Trimble

Trimble collaborates with Microsoft on the development of holographic construction-realted solutions.  The two companies developed a proof of concept to demonstrate the potential of the technology. Development of applications and extensions to current products, which will enable designers and construction teams to utilize mixed reality is in in progress.

The future is mixed

Mixed Reality will have a significant impact on the construction industry over the next few years. The technology addresses some of the industry’s major inefficiencies during the design, construction and operation stages. Integrated with Trimble products and with Building Information Models as a main data sources, Mixed Reality improves communication, tightens workflow integration, and enables real time collaboration with remote teams.

While it is possible to describe the benefits of Mixed Reality in words, one must experience with it first hand to appreciate its power and realize how it can transform AEC industry.

Aviad Almagor is the Director of the Mixed Reality Program at Trimble.  His focus areas are mixed reality technologies implementation and strategies. Aviad joined Trimble as part of the Vico Software acquisition where he was the Director of Product Design. Prior to Vico Aviad served in various positions with leading AEC software companies, and as a practicing Architect. Aviad holds an MBA degree from Heriot-Watt University, Scotland, and a B.Arch degree from the Israeli Institute of Technology.


Back to main page