Steps of control

Autonomous gestion & Robot Control

We have decided to separate the management of the robot in two distinct parts. The first one is the management of the robot on a computer and the second one where the user will have to go in an immersive bubble in order to control the robot with precision

Robot Gestion

Research & Ergonomy

Since the user will be mainly in front of a computer to manage the robots, it is essential that he is in a comfortable position that allows him to work for several hours without physical problems. The user will have at his disposal a double screen allowing him to have a fast visualization of the various elements essential to his work

Robot Gestion

Dual screen

The use of a dual screen is essential in our use case. The user will have to manage both live information on a map and the different notifications and information related to the robot’s actions

Robot Gestion

Left Screen · Map Gestion

The screen on the left, concerns the map and levels of an offshore platform. We have 3 distinct parts

The main one, the map where we can follow the movement of each robot in live, the mission it is carrying out and its next mission. A legend is available to facilitate the understanding of the elements displayed on the screen

The choice of platform levels. With a simple click we can access the other levels but also visualized at which level the user is currently. The indication of the zone that is being visualized. Indeed, some levels can be too big to be well visualized. If the user zooms in on an area, he will be able to visualize its location at a glance.

Robot Gestion

Right Screen · Dashboard Gestion

The right screen is used to receive notifications and important information about the robots

Notifications. Here, it will receive requests for support from the robots in case a robot encounters a problem or needs the assistance of a human

The robots management, presents the conditions of the different robots, if they are performing a task, if all its components are functional, its battery level and the possibility to cancel a task or to take its control

When he selects a robot, he can see its schedule and an approximation of the time it will take to complete its autonomous mission. The robot’s cameras will be useful to visualize what it is doing to verify that it is not making mistakes in its autonomous steps

Robot Gestion

Emergency Case

In case of emergency, the user must be accompanied in a stressful situation but also be able to operate quickly and efficiently. For this, we have designed an intuitive and efficient user path. Important information will be highlighted while the user will receive instructions on the steps to follow

Robot Gestion

Left Screen · Indication Level and Position Problem

At a glance, the user will see the location of the problem. Moreover, if he clicks on the notification he will receive on the notification part of the right screen. The map will update and display the area in question and show the nearest robot that can be controlled to act as quickly as possible

Robot Gestion

Right Screen · Notification & Going to the bubble control

The screen on the right will receive notifications about the emergency cases that need to be dealt with in priority. The user will know if one of his colleagues is already solving one of the current problems. To avoid having several people working on the same emergency. An “Inspect map” button allows the user to quickly visualize where the emergency is located in order to quickly know the affected area. Once the user has taken control of a robot, a notification tells the user in which immersive bubble he will have to go. Moreover, the screen is blocked until the user clicks on one of the buttons of the notification to avoid any risk of error

Robot Control

Immersive bubble Overview

Robot Control

Research & Ergonomy

In order to propose an immersive bubble which will be used to control the robot without being disturbed by the external environment but also to act quickly and to have the least possible movement to make to take the control of the robot and then to operate as quickly as possible on the places of emergencies

Following a request from our teachers, the use of a virtual reality headset will not be proposed in this proposal, under the justification that a headset is not comfortable enough for a purpose that could last a certain period of time

Robot Control

Tests & Experiments

Before starting to produce the 3D and elements that will serve as a proposal for this subject. We challenged our ideas. By gathering all the accessories at our disposal in the office, we created a prototype of the device we imagined. During this experiment, we pointed out the constraining points of this solution. By taking the time to understand the problems that the user will encounter, we were able to anticipate and design relevant solutions

Robot Control

Immersive Bubble

Once our experiments were completed, we made an immersion bubble (picture below without the isolating part) which answers all the problems we faced

It has three curved screens that display the robot's cameras. They are mobile and move to be aligned and adapt to the size of all people who want to use it. The external screens will be either useful to have a vision scope with a degree of 180° or to be able to view with precision the movements of the robot's arms

A control pad offers the possibility to switch from one robot to another with a simple click without having to go back to the computer

The controllers are supported by exoskeletons arms that will allow to make precise movements but also to keep a fixed position if needed

The seat as well as the screens are adjustable and allow the user to sit in many different ways in order to use the controller in a multitude of combinations. Adjustable elbow rests will be useful to rest the user's arms but also to gain precision during certain movements

Robot Control

Controller

The controllers are the most complex part of our project. We made sure that they have an intuitive handling. They have two different modes, each with its own purpose. The first one is to take direct control of the robot’s movements but also of the arms manipulation. The second one is to manage the robot in order to access to more complex manipulation or even to access to difficult areas even for humans

We were inspired by virtual reality controllers because they not only allow to translate the hand movement but also through the use of buttons to offer a scope of functionality much more important than a simple video game controller or a keyboard/mouse

The user will be able to view on the pad the instruction of the controllers presented below to remember the use of the different buttons

Robot Control

Mode 1 · Robot Control

Like its name suggests, the first mode will have the function of controlling the robot. The figure below shows the functionalities of the different buttons and their uses

For example, the joysticks will be used to move the robot while the arm movement will be used to take control of the arms. In the same way, if the user wants to move the arms, he must keep his fingers pressed on the back of the joysticks. This will not disturb him during the manipulations and will also help him not to make mistakes in the manipulation

The force feedback on the triggers is used to feel the power that the robot uses during its manipulations. A vibration that is increasingly strong is added to this to make the user understand that he is involving too much force in his movement and that he must release the pressure to avoid breaking what he manipulates

The switch mode button allows you to switch to Mode 2 with a single click or, if you keep it pressed, to switch to Driving mode. This function has been implemented because it allows to block the arms of the robot as close as possible to its body in order to move on the platform without the risk of colliding with its environment

Robot Control

Mode 2 · Robot Management

When the user switch the mode, the functions of the buttons will also change at the same time

This second mode will be used to position and adjust the position, the rotation angle of the arms and the display of the cameras

All this will offer the user a precise management of the angle of view and the use of the arms. He will be able to see angles of view that are difficult to access for a human but also to reach with his arms objects that are difficult to access

Another feature that is related to reverse kinematic and the “Lock/Unlock rotation angle arm” feature will be presented below in more detail

Robot Control

New Robot

In parallel of the controllers and the immersion bubble. We have imagined a robot that is capable of performing simple tasks in an autonomous way via artificial intelligence, but also of recharging itself in a station provided for this purpose

It has an arm for the central camera that represents its head and therefore its main vision. To complete its view and have a better visibility of its environment, a 360° camera has been added to have a complete view, which will avoid the user or the robot itself to collide with obstacles that he would not have seen

It also has two arms that are adjustable either on the angle of rotation or in length. In this way it can access any type of object around it, whether it is high or lower than its center of gravity. It will be possible for him to reach objects difficult to access even for a human

Clamps at the end of his two arms are used to grip the objects or handles that he will encounter during these missions. These clamps are connected to the force feedback of the joysticks. The user will have a direct feedback of the force he uses when using these clamps

Robot Control

Inverse Kinematics

The use of Reverse Kinematics makes sense for this project.

In order to understand, the Reverse Kinematics as on the two pictures below are used to control the arms of the robot in a natural way. If we move a part of the robot, the rest of its joints will follow at the same time

This is convenient for the main use, when the user controls the arms as well as the hands. But also when he will manage and adjust the rotation angles of the robot as explained in Mode 2.

Two examples are presented below, like the adjustment of the arms or a use case that shows an interesting functionality of the arm

Thanks to the adjustment of the arms, he can control the angle of each joint of the arms, block their positions and therefore have access to even more complex areas

The use case explains that the user is not limited to the size of his own arms but can extend the arm and have access to a longer distance, which will be very practical in some cases as presented in the video presentation

Robot Control

Video Presentation

To demonstrate the relevance of our robot, we made a video that presents a use case where the user following an emergency notification will take control of a robot and solve the problem manually. Each step for the realization of the mission is presented and demonstrates that its functionalities will be useful to the realization of their tasks