Test.jpg

Build AR is a cloud-based platform for small business interior design construction which improves the efficiency of construction and communication while connecting architects, construction team and vendors.

 

It eliminates the gap between architects and contractors by visualizing technical information of the project and keeping each stakeholder on the same page and avoiding meaningless iteration caused by insufficient communication.

Build AR

00 Background Statement

Mainly contributed to Research & Product Design. Lead concept definition & ideation by using Rhino, Unity 3D, Vuforia & C#

Team:With Siyu Zhao

From a long time ago, construction is coordinated by 2D drawings. Architect produces 2D drawings from a 3D concept, General Contractor reads these drawings and interprets those drawings by actually building it.

 

During this process, the architect would come to the site from time to time in order to correct anything installed in the wrong way or discuss technical issue happened on-site due to the limitation of 2D drawings.

By designing a platform where architects provide the digital model of the project as a frame for reference, vendors who provide the product like plumbing, fixture, or furniture could add their product data to this frame. Consequently, by using AR technology, on-site construction workers and General Contractor could quickly reference the related information once they run into any difficulties during the construction process.

01  Research & Solution

01.1 Observation & Insight

In many small budget projects, General contractors and on-site construction workers often have limited skills to both comprehend and realize architect's drawings, it often requires constant site visits for Architect to make sure that every detail in construction goes within plans. But such coordination takes an enormous amount of time and effort. On the other hand, general contractors and construction workers actually comprehend how to build detail from their experience, they just have difficulties understand the drawings. 

1. GC with Worker.png

​Architect on-site with GC

2. Architect with GC.png

GC on-site with Workers

3. BIM.png

Current BIM System

However, such a cloud-based platform like Autodesk BIM 360 is mostly benefiting the coordination between architects and engineers. It hasn't been largely adopted in the coordination of construction.

01.2 Persona

The problems on small-scale interior construction sites are the inefficient communication between General contractors(refered as GC), Construction Worker, Vendors, and Architect.

Persona

The problems on small-scale interior construction sites are the inefficient communication between General contractors(refered as GC), Construction Worker, Vendors, and Architect.

 

GC and on-site construction workers often have limited skills to both comprehend and realize architect's drawings, it often requires constant site visits for Architect to make sure that every detail in construction goes within plans. But such coordination takes an enormous amount of time and effort. On the other hand, GC and construction workers actually comprehend how to build detail from their experience, they just have difficulties understand the drawings. 

01.3 Construction Process & Correspondence Issue

2.Arrangement.jpg
3.Original Process.png

According to interviews with architects and my own experience with a handful of interior projects, below is an illustration of the communication paradigm between our targeted personas when projects progress. 

As observed in the Communication Scenario, we could conclude that in conventional communication patterns during construction, the architect has to go through everything and discuss everything using 2D drawing eventually, which multiplies the time and effort in communication.

Current Communication.png

Conventional Communication Pattern

Revised Communication

Optimized Communication Pattern

However, if we could change this paradigm referencing BIM, we could simplify the communication process by a 3D model provided by the architect which organizes all the information from different products and drawings (Product Information, Architectural Plan, Texture, etc.), serving as a virtual consultant onsite connecting architects and workers.

01.4 Feasibility

This research solely focuses on two parts of the question, the affordability and the usability of our proposed product. 

Affordable AR application in Constructio

Affordable AR application in Construction

After some quick research, what we could assume is that one of the toughest issues of augmented reality, indoor positioning is possible to be efficiently executed by consumer-level technology given that some of the esteemed institutes have funded similar projects with promising progress.

Eye Tracking Implementation in HMD.png

Eye Tracking Implementation in HMD

Problems of HMD like causing nauseous or fail to provide immersive experience could be solved by a built-in eye-tracking device. It gives the possibilities of high-res real-time representation brought by foveated rendering or enhanced depth of focus.

Combining these two discoveries, we can conclude that the idea we are proposing is not only a game-changing concept for the field but to some degree, economically and technologically feasible.

03 Design & Prototype

03.1 Hardware

Just like Alan Kay put it, people who are really serious about software should make their own hardware. To accommodate the full interaction, it is necessary to come up with a set of hardware designs.

HMD

The head-mounted device is designed to imitate a particular face shield at a construction site.

1. HMD Device.jpg
1. HMD Device.jpg
1. HMD Device.jpg

RGB camera and depth camera are located in the forehead panel for potential first-person view recording. Augmented reality content is projected on the transparent face shield. By adding a secondary depth camera and integrating an eye tracker, the device could achieve an enhanced experience with better image quality and more precise calibration.

 

Additionally, this device also has headphones for voice command and instructions.

Image Target

The term “Image Target” is borrowed from the Vuforia Engine. It represents 2d images that our device could detect and track. It enhances the calibration process when the depth camera is insufficient to provide precise indoor positioning, motion tracking, and object detection especially in the extreme condition at a construction site.

2. Image Target.jpg
3.Indoor Positioning.jpg
4. Object Detection.jpg

​Indoor Positioning

Image Target on the surface of the construction could be an alternative to orient the architectural digital model and overlay all the information to the existing site.

Object Detection

Image target on the object could be a secondary source for detecting and recognizing product from different manufacturers and trigger an AR scene.

5. Motion Tracking.jpg

Motion Tracking

Although most interactions are achieved by gesture in an AR application. Image Target on the working gloves help recognition and configuration of the camera.

Labor Glove with Image Target

6. Glove with Image Target.jpg

The special design gloves do not consist of any electronic components. It is a simple labor glove with image target printed on the back of hands, palms, and fingertips. Since for most of the construction sites, it is mandatory to wear a labor glove, designing the labor glove to implement the design is self-evident.

7. Back of Hand, Control Panel.jpg

Back​ of Hand: Control Panel

The image target on the back of the hand could trigger the control panel, which is the main way to interact with the onsite hologram.

8. Palm Control & Display.jpg
9. Finger Zoom In.jpg

Palm: Control and Display

Finger: Mirco Gesture

The image target on your palm could trigger both control panel or 2D documents. It is a conventional way to check cutsheets of products.

The image target on the finger could help motion tracking and gesture recognition simiulating similar experience of operating a tablet or smartphone.

03.2 Key Function

The key Function of this application is in response to the communication problem we discussed. This application provides a detailed installation tutorial for onsite workers, provides cloud-based onsite inspection for architect and GC, and provides data support for the GC and Client.

Provide Detailed Installation Tutorial

1. Provide Detailed Installation Tutoria

It became easier to integrate the full model in the installation process with the combination of augmented reality. The product on the construction site will be detected and recognized. When running into difficulties in the installation process, onsite workers could quickly reference either the installation video or the interactive 3D model instead of going through complicated text or confusing illustration.

Provide Cloud-Based Onsite Inspection

2. Provide Cloud-Based Onsite Inspection

Made possible by real-time indoor positioning, it became possible to overlay information in a physical space. Therefore, without physically being on the site, the architect could use their modeling tools to easily and quickly discuss general problems with onsite workers and GC according to the photos and videos.

Provide Data Support

3. Provide Data Support.gif

Since all the information has been gathered all together and embedded in the digital model, we could visualize that information for GC/Client’s inspection. Although numbers and charts may provide sufficient information in terms of texture, budget, and lighting, an onsite visualization could really provide more intuitive illustration sometimes even help clients and even architect to make decisions regarding their budget and choice of material.

03.3 Operation Guideline

Group 97.png

To fully understand the operations and navigation of AR devices, we conduct a series of in-depth investigation regarding the interaction logic of some of the AR device. After the investigation, we speculate a similar operation manual for us that is both efficient and intuitive.

You can see the full manual 

04 Reflection

For all the project, Build AR is the most relevant, but the hardest.

 

It is relevant because it is an actual problem that exists that could be potentially solved based on my understanding of the field. It is the hardest, simply because as an AR application based on an HMD, you have to think about how you would interact with your design before you even have a design.

 

Obviously, an application of this scale and importance could be a self-fulfilling prophecy, many of the major companies are already working on it, it is still fun to have our own perspective.

 

However, we didn’t try to put yourself in a place, presumably hundreds of years from now, where the relevant technology is smooth and peachy. We try to understand the innovations and limitations of our current technology and imagine what could be really happening in the next few years. Our design may look clumsy and even a bit stupid, but it is based on our research and understanding.

 

For our next step, we may try to test the prototype on a real HMD and try to use a more comprehensive wireframe to explore the full scope of this application.