Build AR is a cloud-based platform for small business interior design construction which improves the efficiency of construction and communication while connecting architects, construction team and vendors.
It eliminates the gap between architects and contractors by visualizing technical information of the project and keeping each stakeholder on the same page and avoiding meaningless iteration caused by insufficient communication.
Build AR
00 Background Statement
Mainly contributed to Research & Product Design. Lead concept definition & ideation by using Rhino, Unity 3D, Vuforia & C#
Team：With Siyu Zhao
From a long time ago, construction is coordinated by 2D drawings. Architect produces 2D drawings from a 3D concept, General Contractor reads these drawings and interprets those drawings by actually building it.
During this process, the architect would come to the site from time to time in order to correct anything installed in the wrong way or discuss technical issue happened on-site due to the limitation of 2D drawings.
By designing a platform where architects provide the digital model of the project as a frame for reference, vendors who provide the product like plumbing, fixture, or furniture could add their product data to this frame. Consequently, by using AR technology, on-site construction workers and General Contractor could quickly reference the related information once they run into any difficulties during the construction process.
01 Research & Solution
01.1 Observation & Insight
In many small budget projects, General contractors and on-site construction workers often have limited skills to both comprehend and realize architect's drawings, it often requires constant site visits for Architect to make sure that every detail in construction goes within plans. But such coordination takes an enormous amount of time and effort. On the other hand, general contractors and construction workers actually comprehend how to build detail from their experience, they just have difficulties understand the drawings.
Architect on-site with GC
GC on-site with Workers
Current BIM System
However, such a cloud-based platform like Autodesk BIM 360 is mostly benefiting the coordination between architects and engineers. It hasn't been largely adopted in the coordination of construction.
01.2 Persona
The problems on small-scale interior construction sites are the inefficient communication between General contractors（refered as GC）, Construction Worker, Vendors, and Architect.
The problems on small-scale interior construction sites are the inefficient communication between General contractors（refered as GC）, Construction Worker, Vendors, and Architect.
GC and on-site construction workers often have limited skills to both comprehend and realize architect's drawings, it often requires constant site visits for Architect to make sure that every detail in construction goes within plans. But such coordination takes an enormous amount of time and effort. On the other hand, GC and construction workers actually comprehend how to build detail from their experience, they just have difficulties understand the drawings.
01.3 Construction Process & Correspondence Issue
According to interviews with architects and my own experience with a handful of interior projects, below is an illustration of the communication paradigm between our targeted personas when projects progress.
As observed in the Communication Scenario, we could conclude that in conventional communication patterns during construction, the architect has to go through everything and discuss everything using 2D drawing eventually, which multiplies the time and effort in communication.
Conventional Communication Pattern
Optimized Communication Pattern
However, if we could change this paradigm referencing BIM, we could simplify the communication process by a 3D model provided by the architect which organizes all the information from different products and drawings (Product Information, Architectural Plan, Texture, etc.), serving as a virtual consultant onsite connecting architects and workers.
01.4 Feasibility
This research solely focuses on two parts of the question, the affordability and the usability of our proposed product.
Affordable AR application in Construction
After some quick research, what we could assume is that one of the toughest issues of augmented reality, indoor positioning is possible to be efficiently executed by consumer-level technology given that some of the esteemed institutes have funded similar projects with promising progress.
Eye Tracking Implementation in HMD
Problems of HMD like causing nauseous or fail to provide immersive experience could be solved by a built-in eye-tracking device. It gives the possibilities of high-res real-time representation brought by foveated rendering or enhanced depth of focus.
Combining these two discoveries, we can conclude that the idea we are proposing is not only a game-changing concept for the field but to some degree, economically and technologically feasible.
03 Design & Prototype
03.1 Hardware
Just like Alan Kay put it, people who are really serious about software should make their own hardware. To accommodate the full interaction, it is necessary to come up with a set of hardware designs.
HMD
The head-mounted device is designed to imitate a particular face shield at a construction site.
RGB camera and depth camera are located in the forehead panel for potential first-person view recording. Augmented reality content is projected on the transparent face shield. By adding a secondary depth camera and integrating an eye tracker, the device could achieve an enhanced experience with better image quality and more precise calibration.
Additionally, this device also has headphones for voice command and instructions.
Image Target
The term “Image Target” is borrowed from the Vuforia Engine. It represents 2d images that our device could detect and track. It enhances the calibration process when the depth camera is insufficient to provide precise indoor positioning, motion tracking, and object detection especially in the extreme condition at a construction site.
Indoor Positioning
Image Target on the surface of the construction could be an alternative to orient the architectural digital model and overlay all the information to the existing site.
Object Detection
Image target on the object could be a secondary source for detecting and recognizing product from different manufacturers and trigger an AR scene.
Motion Tracking
Although most interactions are achieved by gesture in an AR application. Image Target on the working gloves help recognition and configuration of the camera.
Labor Glove with Image Target
The special design gloves do not consist of any electronic components. It is a simple labor glove with image target printed on the back of hands, palms, and fingertips. Since for most of the construction sites, it is mandatory to wear a labor glove, designing the labor glove to implement the design is self-evident.
Back of Hand: Control Panel
The image target on the back of the hand could trigger the control panel, which is the main way to interact with the onsite hologram.
Palm: Control and Display
Finger: Mirco Gesture
The image target on your palm could trigger both control panel or 2D documents. It is a conventional way to check cutsheets of products.
The image target on the finger could help motion tracking and gesture recognition simiulating similar experience of operating a tablet or smartphone.
03.2 Key Function
The key Function of this application is in response to the communication problem we discussed. This application provides a detailed installation tutorial for onsite workers, provides cloud-based onsite inspection for architect and GC, and provides data support for the GC and Client.
Provide Detailed Installation Tutorial
It became easier to integrate the full model in the installation process with the combination of augmented reality. The product on the construction site will be detected and recognized. When running into difficulties in the installation process, onsite workers could quickly reference either the installation video or the interactive 3D model instead of going through complicated text or confusing illustration.
Provide Cloud-Based Onsite Inspection
Made possible by real-time indoor positioning, it became possible to overlay information in a physical space. Therefore, without physically being on the site, the architect could use their modeling tools to easily and quickly discuss general problems with onsite workers and GC according to the photos and videos.
Provide Data Support
Since all the information has been gathered all together and embedded in the digital model, we could visualize that information for GC/Client’s inspection. Although numbers and charts may provide sufficient information in terms of texture, budget, and lighting, an onsite visualization could really provide more intuitive illustration sometimes even help clients and even architect to make decisions regarding their budget and choice of material.
03.3 Operation Guideline
To fully understand the operations and navigation of AR devices, we conduct a series of in-depth investigation regarding the interaction logic of some of the AR device. After the investigation, we speculate a similar operation manual for us that is both efficient and intuitive.
You can see the full manual
04 Reflection
For all the project, Build AR is the most relevant, but the hardest.
It is relevant because it is an actual problem that exists that could be potentially solved based on my understanding of the field. It is the hardest, simply because as an AR application based on an HMD, you have to think about how you would interact with your design before you even have a design.
Obviously, an application of this scale and importance could be a self-fulfilling prophecy, many of the major companies are already working on it, it is still fun to have our own perspective.
However, we didn’t try to put yourself in a place, presumably hundreds of years from now, where the relevant technology is smooth and peachy. We try to understand the innovations and limitations of our current technology and imagine what could be really happening in the next few years. Our design may look clumsy and even a bit stupid, but it is based on our research and understanding.
For our next step, we may try to test the prototype on a real HMD and try to use a more comprehensive wireframe to explore the full scope of this application.