Augmented Reality Registration and Alignment - BC-945

Project type: Innovation
Desired discipline(s): Engineering - computer / electrical, Engineering, Computer science, Mathematical Sciences
Company: Eterio Realities Inc.
Project Length: 6 months to 1 year
Preferred start date: 06/01/2025
Language requirement: English
Location(s): Victoria, BC, Canada; Kelowna, BC, Canada
No. of positions: 1
Desired education level: PhDPostdoctoral fellow
Open to applicants registered at an institution outside of Canada: No

About the company: 

Eterio Realities Inc. are the creators of the production software solution called FabStation (www.fabstation.com).  FabStation provides workers with visual guidance to build or inspect projects in metal manufacturing. The product overlays a digital twin onto a physical workpiece which can be viewed through a Microsoft HoloLens 2 or iPad Pro (with Lidar) utilizing Augmented Reality to either identify discrepancies between what was built and the intended design or give visual guidance to build the assembly in the first place.

Describe the project.: 

Currently, aligning virtual 3D models with large real-world objects in AR typically takes up to 10 minutes and requires trained users to place a QR marker to register and bring in the model, then manually adjust the model’s position via 6 DOF controls. This process is time-consuming, inconsistent, and unsuitable for scalable applications.

We seek to dramatically improve this process using AI, advanced CV techniques and other relevant technologies to improve this experience for users.

Challenges:
• Our customers work on large assemblies that are 3 to 30 m in length. Therefore, practically, only a partial view of the workpiece is able to be in the frame of view of the camera of the target device at any time.
• Known object recognition, 6 DOF segmentation and edge detection models require the whole object to remain in the field of view to work reliably.

Requirement:
• To develop an “object recognition” system that will recognize a partial view of an object and overlay its corresponding 3D model over top to within 1.5 mm
• Considerations for base materials that do not conform 100% to their design should be made
• The registration and alignment procedure should be understandable by a common first-line worker
• Alignment procedure should be able to be completed in 3 minutes or less

Our current research has lead us to develop a “4 point system” whereby we establish a triangular plane by selecting 3 points on the model and then place 3 corresponding points on the workpiece in 3D space to complete registration and X,Y, Z alignment. We then place a chosen 4th point at the other end of the workpiece to establish the rotational axis, pitch and yaw, and scale. We believe there is potential to leverage this new approach in this research project.

Required expertise/skills: 

Artificial Intelligence, Computer Vision and Augmented Reality. Specializing in :
• 6 DOF segmentation
• Edge detection
• Unity 3D
• AI models and technologies for computer understanding of objects in an industrial environment.