Robot Database Test Plan

This document describes the test procedures and results for the five database systems being evaluated. The goal of the tests is to get a basic understanding of the database system and its capabilities. These tests will help in the concept selection process of the project; the results of the tests will be used to rank the systems according to the parameters in the concept selection document.

List of Equipment

  • Computer with Internet access and the following database systems installed:
  • MySQL Cluster
  • Robot Operating System
  • Microsoft Robot Development Studio (MRDS)
  • Hierarchical Action Framework (HAF)
  • RTI Data Distribution Service

Test Procedure

To gain basic understanding of each database system and its capabilities, the system will be set up and used to perform a basic task. The tester will keep track of the amount of time spent setting up the system and performing the task. The task for each of the systems was chosen from a tutorial for that system. References to the tutorials and the task to be performed are provided below. After setting up and performing the task from the tutorial, answer the evaluation questions provided below. The answers to some of the questions will need to be found by the tester from the system’s documentation.

Tutorial tasks to be performed

  • MySQL Cluster: Perform all instructions in the MySQL Cluster Quick Start Guide – Windows available at:
  • Robot Operating System: Follow the cturtle tutorial found at
  • Microsoft Robot Development Studio (MRDS): Programmed the beginner’stutorials that show how to program in Visual Programming Language.
  • Hierarchical Action Framework (HAF): Run the example mission “box1.txt” by following the instructions in the files “README.txt” and “how to run.txt” that come with the software (Note: python2.5 will need to be installed)
  • RTI Data Distribution Service: Do examples described in sections 4.1-4.5 of the RTI Shapes Demo User’s Manual, which is a pdf that comes with the software

Evaluation Questions

  • Does the system provide a publish/subscribe communication system? If so, describe its capabilities. If not, could it be compatible with a publish/subscribe system?
  • Is the system real-time? If not, do you expect it will meet the MAGICC lab’s speed requirements? Why?
  • Are there any potential ways that messages or data being transferred over the system could be lost?
  • On a scale of 1-10, how easy would it be for new users to start up the system and make simple changes to it?
  • Is the system compatible with the software languages commonly used in the MAGICC lab (C++, C#)?
  • How long do you anticipate it would take someone to learn the ins and outs of the system, so that they could make major changes to it and supervise its use?
  • What operating systems is the database system under evaluation compatible with?
  • List the sources of help available for the system.
  • Is the system open source? Is the system free? Does the MAGICC lab need to obtain a license to continue using the system?
  • How much time was spent setting up the system and performing the tutorial task?

Results

MySQL:

  • MySQL does not provide a publish/subscribe system. MySQL is database and leaves it up to the user how to interact with it. Thus, MySQL could be compatible with a publish/subscribe system.
  • Some versions of MySQL are real-time, and those that aren’t should still meet speed requirements.
  • MySQL is highly reliable and messages or data should not be lost.
  • I give MySQL a 2 on its easiness for new users because it requires creating and modifying text files and running complicated commands from the command line.
  • MySQL is compatible with the software languages commonly used in the MAGICC lab
  • I anticipate it would take someone a week or so to learn MySQL’s system sufficiently to make major changes or supervise it.
  • MySQL is compatible with Windows, Linux, Mac OS X, and several other operating systems
  • MySQL is commonly used and has many sources of help available. Help sources include the company, tutorials on MySQL’s website, and many other websites and forums can help.
  • MySQL is open source and has both free and commercial editions. The free editions would fulfill the needs of the MAGICC lab, so no license would be necessary to continue using it.
  • 1 hour was spent on the tutorial task.

ROS:

  • ROS provides a publish/subscribe communication system. It has a master node that keeps track of who is publishing and who is subscribing and then routes the communications that are sent to the appropriate parties.
  • It is not a real time system but because it was make to have little overhead I expect it to be able to meet the MAGICC lab’s speed requirements.
  • Messages that are published or stored in a buffer for each subscriber and as long as the subscriber processes the data before the buffer runs out the data should not be lost.
  • The ease of making simple changes should be about a 7 for a new user.
  • It is compatible with C++.
  • I would expect that it would take about a week for someone to learn the ins and outs and able to make major changes.
  • ROS is only compatible with Ubuntu Linux.
  • The most comprehensive help is the tutorials which have a wide range of helps covering most topics.
  • The system is open source and free for download.
  • I spent about 4 hours to do the first five tutorials.

MRDS:

  • MRDS does have a publish/subscribe method that connects the publisher and the subscriber directly.
  • It is not a real-time system but I expect that it probably would not speed requirements for the MAGICC lab because it was not made to be connected over a network.
  • Data is written over every time the publisher publishes data and so a lot of data could be lost if the subscriber does not process the data at the same speed the publisher publishes data.
  • It would be incredible easy to make changes to the system so it would receive a 10.
  • MRDS and the Visual Programming Language are compatible with C#.
  • It should not take that long to learn the ins and outs of the system and I think it would take about a day to learn how to and make major changes to the system.
  • MRDS is compatible with Windows.
  • The resources available are in the form of tutorials at
  • This is not open source but it is a free download.
  • I spent about an hour going over the tutorials and setting up the system.

HAF:

  • HAF does provide a publish/subscribe system with built in emergency protocols if communication is lost.
  • HAF is not real-time but was developed for use with UUVs so should also be fast enough for the MAGICC lab’s UAVs
  • HAF was made to be highly reliable so that UUVs are not lost, so no data or messages should be lost except in rare instances.
  • I give HAF a 5 on its easiness for new users because it uses a different programming language and is ran through the command line.
  • HAF is not compatible with the software languages used by the MAGICC lab. It uses its own language called HAFscript.
  • I anticipate it would take someone a few weeks to learn the ins and outs of HAF sufficiently to make major changes or supervise it.
  • HAF is only compatible with Linux.
  • HAF’s sources of help are limited to the documentation that came with it and Sam Smith, one of its developers, who Dr. Beard knows.
  • Sam Smith has provided the MAGICC lab with HAF and will continue to support its use, so no licenses are required to use it.
  • 50 minutes were spent performing HAF’s tutorial task.

RTI DDS:

  • DDS has a top of the line publish/subscribe system with many quality of service parameters that the user can modify.
  • DDS is real-time and will definitely meet speed requirements.
  • DDS is highly reliable and should not lose and messages or data.
  • DDS has a lot of tools to assist users so I give it a 9 on its easiness for new users.
  • DDS is compatible with the software languages commonly used in the MAGICC lab.
  • I anticipate it would only take a couple days to learn DDS sufficiently to make major changes and supervise it.
  • DDS is compatible with Windows, Linux, Mac OS X, and several other operating systems.
  • Help can be obtained for DDS through the company and it has good documentation.
  • DDS is not open source and is usually not free. However, the company does provide free licenses to universities for educational purposes. The MAGICC lab would need to investigate what is required to obtain one of these licenses.
  • 40 minutes was spent doing the tutorial task.