Hands-Free Camera Controller
Jeffrey Gould
Introduction
Background
Scope of Work
Design Criteria
Summary
Tasks Accomplished
Tasks Not Accomplished
Recommendations
Design
Objective
What Components Were Chosen and Why
Mapping Algorithms
Mapping of Yaw
Mapping of Tilt
Problems Encountered
Memory Constraint
Sensor
Sensitivity
Not Tilt Robust
Future Work
Change the Sensor
Change the Micro-Controller
Do Research
Add Functionality
Additional Operation Mode
Voice Recognition
Fully Wireless
Introduction
Background
In many teleoperation robotics applications a human being must control a manipulator or remote robot. In these cases the operator will need updated visual information of the environment to ensure safe and effective manipulator/robot control.
This visual information can be gathered by a video camera strategically placed in the remote environment or attached to the robot. The video signal can than be displayed to the operator to aid in control. In some cases it may be desired to modify the orientation of the camera in the remote environment to provide the operator a more useful perspective. There is an advantage to offering a mechanism for “hands-free” orientation adjustments. Consider a mobile robot that is being driven in a remote environment. When driving forward, the camera would be best oriented forward. But, when entering a turn the driver may want to look in the direction the robot will be turning. A hands-free mechanism could allow for the camera’s orientation adjustment without requiring the driver to release the driving controls. Another example of a useful application would be a robot used for remote inspection work. In this application the camera can be reoriented continually while the driver’s hands are free to control the robot’s motions.
Scope of Work
This project seeks to demonstrate a proof of concept of a hands-free controller for a remote pan and tilt video camera mount. It focuses on the selection and use of sensors to be used for determining two degrees of freedom of head tracking. The information from the sensors is used for the control of the camera’s orientation. When the sensors detect a head movement about the vertical axis (yaw) or the horizontal axis (pitch) a micro-controller commands an appropriate rotational response of the camera mount. This provides the user with a natural, intuitive method for reorienting the remote video camera.
Design Criteria
The mechanism is to be as self-contained as possible making it appropriate for use in robotics systems that are designed to be easily portable and expedient to set up in the field. Additionally, this system should not require an elaborate calibration procedure at each set up and the sensors should be integrated into other system components. Meeting these last two requirements will help to keep the mechanism transparent to the user (providing a more natural sensation) and will potentially increase the robustness of the overall system. The system’s robustness will be improved if it is not heavily dependent on calibration and also if it is not dependent on external devices for head motion tracking.
Summary
Tasks Accomplished
Sensors capable of measuring head rotation (yaw) and tilt (pitch) were selected. Software was written for a small micro-controller to read these sensors’ inputs and command the rotation of the camera mount about its two degrees of rotational freedom. The measurements that are gathered are transformed to servo commands using a mapping algorithm developed for this project. A special mount was purchased to attach one small servo to the output shaft of another. This mount allows two servos to be held so that the axes of their rotation are orthogonal to each other. All the necessary cables were fabricated and the sensors were attached to a piece of headgear to demonstrate the system.
Tasks Not Accomplished
It was originally desired to provide two separate control schemes for operation of the system. Due to complications in both the hardware and software, neither of these schemes was ultimately developed. A modified scheme was developed that allows demonstrating the hardware as a proof of concept only. The scheme allows the yaw and pitch control to be demonstrated separately, but not together.
Recommendations
It is recommended that the sensor for measuring rotation about the vertical axis (turning of head) be replaced with another technology. A recommended technology to investigate is gyroscopic rate transducers. Another recommendation is to consider moving the system to a different micro-controller platform. While the platform used was highly versatile in terms of hardware-interface compatibility, its memory limitations became a problem. Improved functionality and robustness will require larger software and memory constraints will likely preclude very many enhancements. The move to another micro-controller platform might be better to do sooner, before a large investment in platform specific software and hardware is made.
Design
Objective
The objective was to build a working system that demonstrates feasibility of the concept while providing some experience with using sensors.
What Components Were Chosen and Why
A complete list of components used is included in the appendix. I already owned some of these components because I had purchased them for other personal projects (for example, the Brain-Stem© micro-controller and the servo motors). The servos are compatible with the micro-controller and together their union made a solid basis for the rest of the system.
The Devantech magnetic compass was chosen as the sensor for measuring yaw for two main reasons. The first is its compatibility with the BrainStem. The second is its low cost as compared to the micro piezo gyro module (see appendix under “recommended device”). At the beginning of the project I believed the compass was a suitable sensor that would be easy to use. The fact that it gives out a reading of absolute heading is desirable in this application as it is not as subject to drift as some angular rate sensors can be.
The ADX202e accelerometer from Analog Devices was chosen because I wanted a chance to work with a MEMS (Micro-Electro-Mechanical-System) sensor. Equally important is the fact that I received this chip as a free sample from the manufacturer.
Mapping Algorithms
Mapping of Yaw
The mapping of the compass readings to a change in the camera’s yaw proved to be a very difficult problem. It was decided that the servo commands given would be absolute position commands rather than relative. The resolution of the servo is not very high (0-255 for 180 degrees), and the micro-controller works only with integers, truncating all floating-point values. The combination of these facts could potentially lead to an accumulation of position error if relative position commands were used. By specifying absolute positions, the error will not accumulate over time.
The servo is centered when sent the command of “127.” This should relate to the initial position of the user’s head at start up. When the system is initialized, a reference heading is taken and stored to be used for comparison to future readings.
The reference heading (and all future compass readings) can be an integer from 0 to 3599. What direction the user is facing when the system is initialized will determine the complexity of the mapping scheme. In the simplest case the reference heading can be subtracted from the new reading and the result can be scaled to a command value for the servo.
Two problems arise with this method. One is when the user turns left or right by more than 90 degrees (past the mechanical limits of the servo). And the other is when the compass is rotated through the 3599-0 point.
The first problem results in the possibility of sending the servo a position command that is either negative or one that is greater than 255. When this happens, the servo attempts to swing around to the opposite side, causing highly undesirable oscillations.
The second problem can also cause oscillations and a general breakdown of a simple mapping algorithm. To solve both of these problems, a four case algorithm was originally developed. Depending on which quadrant the user’s head starts in, a different algorithm is chosen. The program for this algorithm can be found in the appendix.
Unfortunately, the complex (but very robust) four-case algorithm was too math intensive which resulted in an application that was too large to fit on the chip. A reduced complexity algorithm was developed (also shown in the appendix) that fits on the chip but is not as robust.
Mapping of Tilt
The mapping of the yaw rotation became very difficult and time consuming. When it came time to map the tilt readings, an extremely simplified algorithm was used. The change in tilt readings is doubled and then added to the servos center position. This is essential a transducer gain of 2. This value was picked by trial and error by using a simple program written for this purpose. The program took a tilt reading and moved the servo. Trials were conducted by moving the tilt sensor and watching the servo to see when they seemed to move in tandem. The gain was adjusted until their motions were similar. The gain value of 2 is not optimal, but the micro-controller only uses integer values. One solution to this limitation would be to raise the desired gain by one order of magnitude and then divide by ten. For example if the gain is desired to be 2.5, the gain can be made to be 25 and then the result can be divided by 10. The micro-controller will truncate the result to an integer, but will do so after a more precise gain has already been applied.
Problems Encountered
Memory Constraint
The first problem encountered was the limitation of the size of the program that could be loaded onto the micro-controller. There are two slots available on the chip and each can hold an application of only 1kByte. Although the original algorithm was robust, it was too large to fit on the chip. The reduced complexity version of the mapping algorithm fits easily.
Sensor
After a great deal of work was put into working with the magnetic compass, it was determined that it really is not suitable for this application.
Sensitivity
The compass is far too sensitive to magnetic fields for this application. Many magnetic fields caused by electronic equipment etc. dominate the reading of the compass and make reading rotation based on the earth’s magnetic field difficult and inconsistent.
Not Tilt Robust
The compass is meant to be operated in a horizontal position. In this orientation (without other magnetic interference) it is very good at measuring rotations. However, when the compass is tilted, it reads the earth’s magnetic field differently. This appears as a change in readings which is interpreted as a change in heading. Because of this, unwanted rotations about the vertical axis can happen.
Efforts were made to map these erroneous changes and build compensation into the algorithm based on the reading from the tilt sensor. This proved to be difficult because it was very hard to gather reliable data. A special jig should be built ensure tilt of the sensor without rotation. This would isolate the error and allow a mapping. A further difficulty is that the error appears to be non-linear in relationship to both pitch and yaw (roll was not investigated). That is to say as the tilt changes, the rate of error also changes in a non-linear way. But this relationship is also not the same for every heading. For example when facing East, there is one relationship between tilt and yaw error, and when facing North there is another. The compensation algorithm would become very complex and possibly require several iterations using a numerical method (or lookup tables) to make a final adjustment. This is too complex for this micro-controller and not desirable for this application. Furthermore, if there is a strong magnetic field disturbance nearby, even this compensation algorithm will not help.
Future Work
Change the Sensor
If it is desired to make a more fully functioning proof of concept demonstration, it is recommended that the yaw sensor be changed. Acroname (the BrainStem supplier) also supplies a rate sensing module for use with the BrainStem (see appendix). This module is the current recommendation for immediate improvements to the prototype. In addition to learning how to interface the module and take readings, supporting software for the new sensor will also to be written.
Change the Micro-Controller
To avoid future memory constraints, the micro-controller used in this application will be changed. It is likely that the next platform will be an HC11 chip-based platform as I already own one.
Do Research
At this point I am satisfied that this concept can be made to work and that there is a need to have this type of mechanism. I think it is worth pursuing and what should be done now is what should have been done in the beginning, research. Even though I started this project without doing a literature review on the subject, I think it would be a mistake to continue the project without doing it now.
Add Functionality
Additional Operation Mode
The original concept for this device included two separate control schemes. One scheme moves the camera along with the head giving a one-to-one correspondence between head rotations and camera rotations. This type of camera control would be most effective when the user is wearing a HMD (Head Mounted Display). An HMD immerses the user in the remote environment and a video image that moves with the user’s head will feel more natural.
The second proposed control scheme, which was not developed for this project, is outlined in detail in the appendix under “alternative Control Mode.” This proposed scheme would be best suited for the operator who uses a video monitor to watch the images. In this situation it is desirable to let the operator always face the video monitor, even when the desired camera angle is not straight ahead. To accomplish this, it is proposed that a control scheme be developed that will hold the camera still while the head is facing towards the video monitor but will pan the camera whenever the user’s head moves away from the monitor. For example, suppose the operator wants to redirect the camera so it is pointing slightly upwards from its current orientation. By simply tilting the head slightly above the direction of the monitor, the camera will begin to scroll in that direction. The operator can continue to watch the screen because the head motion should not be so large as to prohibit this. When the camera is in the desired orientation, the operator again faces the monitor. This will stop the scrolling motion of the camera. The camera will stay in this orientation until the next time the head is pointed away from the monitor. For example, to “undo” the above change in tilt, the user would need to point the head downward until the camera’s orientation was agreeable.
To improve the usability of this second control scheme, a method of homing the camera should be included. That is, by giving a command, the camera will return to a neutral, straight-ahead, orientation. Another improvement would be a method for adjusting the speed of the scrolling.
Voice Recognition
One emerging technology for increasing hands-free control of devices is voice-recognition software. Particular uses in this application for voice-recognition software include the homing command and scroll speed control mentioned above in “Additional Operation Mode” section. By simply saying commands such as “Home” or “Low Speed,” the functionality mentioned above could be added and the hands-free operation of the device is preserved.
Fully Wireless
The video signal can be made wireless by adding a video signal transmitter/receiver. I have a set of these and have verified that I can transmit a signal from a small remote video camera. I have also rewired a HMD to display the incoming video signal. However, the system still depends on wires connecting the micro-controller to the sensors and the servos (see appendix under “Wiring Schematic”). Future work could look into ways of separating one of these connections and replacing it with a wireless communication technology. Since standard servos from radio controlled models are being used, one possibility is to use a 2-channel transmitter/receiver to make the servo connection wireless. This would require some more investigation. It is probably preferable to make the motor/servo control connection wireless rather than the sensor connection. Wireless servos connections are common and an existing technology can probably be adapted.
Appendix
Component List
Alternative Control Mode
Data Sheets
BrainStem micro-controller
Devantec Compass
ADX202e accelerometer
Futaba S-3004 Servos
Recommended Device
Micro Piezo Gyro