William Zickler, “The Development of Augmented Reality Software for Controlling and Observing Robotics”
Mentor: Jerald Thomas, Computer Science
Poster #14
Augmented Reality (AR) is the combination of virtual reality’s computer generated graphics layered on top of what can be seen in the real world, this idea has potential to be useful in many different applications and industries. One area in particular where AR can make a significant impact is in manufacturing and production lines where our goal is to create a software application that can not only take advantage of the new augmented reality tools but also discover new creative ways to interact with robotics. Our idea is to build software so that a user equipped with a Microsoft HoloLens 2 will be able to communicate with one or more FANUC ER-4iA Robots and create new ways to observe, analyze, and manipulate the robot. The implementation of both hand and eye tracking based controls (also called gaze) will be important for the future of augmented reality, and combining both together will maximize the interactions AR can deliver. While there is formal study currently in the works, the plan is to build our program in a way that best suits the user by allowing for a variety of different toggleable interaction and viewing techniques, so any user that may potentially use the application will find the best natural way to use it. The implications of this project may have a great effect on manufacturing line workers, machine inspectors, and engineers who need quick and clear representations of data or control of the robot. As AR technology becomes cheaper, lightweight, and more user friendly, it will begin to be adopted in more and more environments, eventually replacing the static machine-human interfaces that we have today. Thus development of its tools are important for not just this application, but many different industries going forward.