RMX-RTEMS Conversion

RMX-RTEMS Conversion

Robert C. Sass (rcs)

Anthony E. Gromme (teg)

Revision History

Rcs / 21Apr2005 / Initial version.
Rcs / 24Apr2005 / Change 2.1 work list. Add socket call and specialty micros sections. Add near & far pointers to selectors section.
Rcs / 29Apr2005 / Make Design section and do CVS, C and Fortran includes.
Rcs / 3May-2005 / Start design of RMX->RTEMS system call migration.

RMX-RTEMS Conversion

1Introduction

2Project Management

3Work Items

3.1Boot RTEMS in an EPC Micro with a Test Application

3.2Mechanical Changes

3.2.1C, Fortran & PLM Discussion

3.2.2UNIX/Linux Directory Structure and Make Files

3.2.3C Include File Strategy

3.2.4Fortran Include File Strategy

3.2.5RMX System Calls

3.2.6Selectors & Pointers

3.2.7PLM -> C & Assembler

3.3Structural Changes

3.3.1Time Format

3.3.2Remove SLCNet Dependencies

3.3.3Task Initialization

3.3.4Debugging

3.3.5RMX Job/Task vs. RTEMS Task/Thread

3.3.6Interrupt Handlers

3.3.7NMI

3.3.8RTEMS User Extensions

3.3.9Selectors for MBCD

3.3.10Micro Boot

3.3.11Socket Call Differences

3.3.12Specialty Micros

4Design

4.1CVS Structure

4.2C Include File Strategy

4.3Fortran Include File Strategy

4.4RMX System Calls

4.4.1Semaphores & Regions

4.4.1.1Create Semaphore

4.4.1.2Obtain Semaphore

4.4.1.3Release Semaphore

4.4.1.4Regions

4.4.2Mailboxes

4.4.3RMX System Calls Conclusions

1Introduction

Making the switch from RMX to RTEMS for the SLC micros will be some work but should be much less than what has been expended so far trying to move forward with RMX. The conversion can be done in discrete pieces, keeping the existing micros running until the final switch is made. In addition to having a common operating system and development framework for all IOC/micro code, it also gets us away from dependence on the RMX VMS support tools which have long been grandfathered by Alex Hunter. Furthermore, RTEMS also has additional features that RMX lacks such as:

1In-memory file system.

2POSIX real-time extensions.

3Threads.

4CORBA support.

Who knows where all that could lead?

The following assumes running existing micro applications under RTEMS in EPC micros and doesn’t include any FECC work.

2Project Management

Since we are only two people working on this, I don’t think we need a database to track the work items. I would propose just using this document to update tasks as we go and mark them “done” as they are complete. Is this enough or do we need something more elaborate?

I would propose that we put this document in a convenient place on the Web and simply use email to reserve and update it. I suppose we could CVS it but again that seems like overkill for two people.

3Work Items

3.1Boot RTEMS in an EPC Micro with a Test Application

The EPC micro looks like anold PC with grafted on Multibus and SBX interfaces. As proof that there aren’t any hidden surprises we first need to configure an EPC to Ethernet boot RTEMS and check that the lower ¾ megabyte is not used by RTEMS and Multibus memory and the I/O ports are properly mapped. Except for interrupt handling which we must change, micro I/O is independent of the OS.

1. Boot RTEMS into an EPC micro. (done)
2. Peek/poke I/O registers to insure that I/o space and the Multibus is mapped and accessible.

3.2Mechanical Changes

Mechanical changes are things like include files, RMX specific code and of course PLM. Here we just accommodate the known differences between the RMX and RTEMS environment and don’t address the RTEMS runtime issues. They can be done pretty much in any order.

3.2.1C, Fortran & PLM Discussion

Since the RMX C compiler is an old ANSI standard I assume that the GNU PC cross compiler will be fine. In addition to RMX specific code, we also have a non-standard include format that needs to be made portable either by changing allincludes that we have defined and/or adding conditionals for the RTEMS compiles. This implies deciding on at least an include directory structure for the new development environment. It might be useful to set up the entire directory structure at this point for all micro source code.

Our Fortran is also an old standard that I dimly recall has some peculiarities. Hopefully the GCC Fortran cross-compiler won’t product unexpected surprises.

Of course all remaining PLM code that needs to run in the RTEMS environment must be converted to C.

Eventually all selectors must be converted to pointers except for the bowels of the Camac I/O. All higher level code should just deal with pointers.

3.2.2UNIX/Linux Directory Structure and Make Files

From the above discussion we at least need to have a UNIX/Linux directory structure for include files and to run test compiles on both Fortran and C. To that end, now would be a good time to define the development directory structure for all of the micro code. Since we need to do test compiles on a continuing basis as we modify the running micro RMX code, we might as well also set up at least a first pass for the Make files that will build the applications as we think they will run under RTEMS.

Greg’s work provides a template for managing version/release code under CVS and can be used as a starting place for the micro builds.

The following is list of tasks I can think of from the previous discussion.

1. Define CVS directory structure for all micro source code and include files.See 3.1 for design.
2. Make dummy main files for each of the micro jobs and CVS them.
3. Do make files to build an RTEMS image with the dummy tasks.
4. Build and boot RTEMS with dummy jobs.

3.2.3C Include File Strategy

Decide on whether to use conditional compiles, standard include syntax (possible?) or some combination to make our non-RMX include files acceptable in both environments. Once a strategy has been decided upon, then some simple tools/scripts can be written to automate the conversion process.

1. Experiment and define the C include file strategy. See 3.2 for design.
2. Modify & recompile all micro C source code and include files for the defined strategy.
3. Test resulting image and release all modified code and include files to CMS on VMS.

3.2.4Fortran Include File Strategy

I’m not sure what the issues are here. Can we make conditional compiles work in both environments? Use a standard or home-grown pre-processing tool?

3.2.5RMX System Calls

Given that we must change all of them, at this point we could hide all system dependencies in another layer as was done with EPICS, this making any future migration easier. Alternatively where system calls are very similar e.g. semaphores we could use conditional compiles/macros to do the job. Or perhaps we can write our own RTEMS extensions to mimix the RMX RQ calls? We should keep in mind that once the RTEMS migration is complete, all of the RMX code can disappear. This would seem to make a stronger case for a separate layer. Perhaps even POSIX compliance?

1. Define RMX->RTEMS systems call strategy. See 4.4 for design.

3.2.6Selectors & Pointers

This is a specific variant of RMX unique capabilities. Selectors must be converted to vanilla pointers and the use of near and far pointers eliminated. The selector/pointer business will probably be on-going and I’m not sure how extensive it will be so I’ll just make one item here that we can elaborate on as we proceed.

1. Convert all selectors, near and far pointers.
2. BPM job
3. Timing job

3.2.7PLM -> C & Assembler

I believe Alex already has a tool to do the brute force work. Not sure how much PLM and assembler is left that will need to run under RTEMS.

3.3Structural Changes

Here we need to modify the RMX jobs and task to run in the RTEMS environment. Since I’m not yet an RTEMS expert I’m probably missing lots here so this is just an overview of issues I can think of off the top of my head. I’m sure other issues exist. Many details need to be worked out. This is the hard part.

3.3.1Time Format

The existing code uses a VMS style format and RTEMS uses a UNIX style i.e. ticks since some year stored in 32 bits. I think we have our own time routines and can continue to use the VMS style which doesn’t have any rollover problems though the RTEMS time must still be maintained in its UNIX style format. We have code in the clock interrupt but I’ve forgotten the details. Mostly related to the debugger I think. Some assembly may be required.

3.3.2Remove SLCNet Dependencies

Are there any left besides booting and the debugger? Both of these should disappear with RTEMS leaving only Pnet. Do we need the SLCnet driver just for it?

3.3.3Task Initialization

After startup, RTEMS transfers control to a user initialization task which starts up user applications. This is similar to what happens now in RMX though I’m sure we’ll need to make changes in how jobs/tasks are started/synchronized. More design needed here.

3.3.4Debugging

RTEMS supports GDB over Ethernet so debugging shouldn’t be an issue. However we’ve put lots of special things in our debugging code.Exception detection with stack dump among other things.

3.3.5RMX Job/Task vs. RTEMS Task/Thread

We use the RMX idea of the job that has the process contest and many tasks that can execute under it in several places. It seems that RTEMS has tasks and threads. It’s not clear to me what all of the impacts and implications are.

3.3.6Interrupt Handlers

RMX has the concept of interrupt handler and interrupt task. Mostly I think because the X86 has only a single interrupt line into the processor so only a single handler can be active at any one time. Many interrupt tasks can be concurrently active to execute the more time-consuming tasks needed to complete the processing of the interrupt. It looks like RTEMS may have a similar concept though of course, further research is needed and there may be dragons here.

3.3.7NMI

Most of the code was moved to an MI for the EPC board so other than interrupt task protocol I think most of this should work OK. What happens with the 386 boards? Make them EPCs?

3.3.8RTEMS User Extensions

At first I thought that this was a well-defined way to add your own system services but it’s not. It’s actually just a way to do additional user-specific things when an existing system service is called.

3.3.9Selectors for MBCD

If the FECC is not included, then we need to manage selectors for MBCD I/O. If we have a linear address that is just the physical address then this is easy and not an issue. An option here is to make MBCD package pointers 32 bits.

3.3.10Micro Boot

Micro reset, individual micro boot and IPL ALL must be handled transparently from the SCP. We could use DHCP to do a broadcast boot. SCP changes must also be made so boot of RTEMS micros is just like before.

3.3.11Socket Call Differences

There may be API differences between the RTEMS socket library and the TenAsys library that will require some conversion.

3.3.12Specialty Micros

We have at least the MPG code that will need a separate effort. Are there other micros left that don’t run the standard suite of jobs?

4Design

As a general goal I think it best to modify the existing code on VMS that runs under RMX to make it as compatible as possible with the gcc compilers. In addition to include file and other syntactic changes that can be made with a script, we’ll likely be make many other changes compatible to both systems “by-hand”. These can be tested in RMX so we won’t have more surprises and debugging to do under RTEMS. We don’t want to fork the code and comit to CVS until we have the RMX source “cleaned up”.

Once the RMX code is as compatible as possible with RTEMS we can then copy all of the source code to UNIX and begin serious testing.

4.1CVS Structure

The obvious CVS root place to start is the ESD-maintained-software-repository.

The boot directory would contain the boot scripts.

The latest RTEMS code is maintained in /afs/slac/package/rtems. In looking at the EPICS CVS info for RTEMS in SLAC-EPICS-Releases/base/src/RTEMS/base I see five .c files relating to RTEMS configuration so I assume we’ll have something similar to configure RTEMS for our EPC boards. These would be in the config/RTEMS.

Do we need config files for anything else?

The “C” and Fortran include directories would have the micro-only include files. For C that would be the *.hm files. I think there’s a clever way to include the *.hc files from VMS but maybe not.

In general I see no reason not to mirror the approximate structure of the existing code. Slclib is linked by individual jobs and Reentrant is code shared by all jobs. Since the member card has no significance to gcc we should put all sources in their correct directories.

This does not include special micro images like MPG, FB88 & PT01.

Any need to build Basic or DBS images?

So here’s the CVS directory proposal:

ESD / /slcRTEMS / /Boot
/Config
/RTEMS
/Src
/Etherboot
/Include
C
Fortran
/Libs
/Reentrant
/Slclib
/Jobs
/Anlg
/Async
/Bcom
/Bpm
/Cam
/Crate
/Dbs
/Diagnostics
/Fbck
/Klys
/Mcom
/Mgnt
/Stat
/Test
/Time
/Verex

4.2C Include File Strategy

Include SLCRMXI and SLCTXT in the include path specified by ucmp and eliminate them from the source code. Thus except for the RMX dependent code, the “C” code should just work under RTEMS.

There is the problem of the .hc files that are shared between the micro and VMS code. We can fork it into UNIX and VMS copies but that always makes one nervous. A better approach is to use the wget facility. This can be used in the make files to insure that the latest version of a shared include file exists on UNIX. The source still lives only on VMS.

What about extension .c86? It may be possible to have gcc recognize .c86 as a valid extension but I didn’t see any obvious switch and trying it on flora failed so renaming the files to *.c may be necessary and not a big burden.

4.3Fortran Include File Strategy

RMX Fortran does includes of the form $ (file.inc) that is the old form for include files.Using single quotes as in ‘file.inc’ is the new form. Try as I might I couldn’t get gcc or g77 to swallow the () form even without the $ before the include statement.

Ftn386 seems equally rigid in its desire for the leading $ on include statements and complains about any attempt to use the ‘file.inc’ form so we’re stuck.

Perhaps we can use the C preprocessor to help here on Unix.

Directives using “$” are list/nolist, compact, large, & reentrant as well as include.

There are only a few nested Fortran includes and we should remove them.

At this point the only option I see is writing a Perl or other script on Unix to change the includes and any other syntactic problems we encounter. We should do this up front initially to see what other compiler surprises are waiting for us.

4.4RMX System Calls

At first glance I thought we could replace our RMX calls with their real-time Posix equivalents but it seems that not all are implemented in RTEMS. In particular the process creation calls are completely absent because RTEMS has no memory protection and is a single process multi-threaded model. Other groups of Posix calls are only partially implemented for much the same reasons. Furthermore, much of the Posix documentation is incomplete and gives the impression that it’s very much a “work-in-progress”. Because the RTEMS system calls in many cases closely mirror their RMX counterparts I’ll make the following choice up front:

We’ll use the RTEMS and not the Posix API.

What about the EPICs OSI layer? That would make us compatible with EPICS but at the cost of changing a great deal of code I think.

Another option is to write calls that that mimic the rq system services and change a very minimum of code. This only works if the existing RTEMS services can be closely mapped to the RMX calls. This could encompass an rmx.h include file for RTEMS that redefines TOKEN and a few other items so that most RMX system calls could be ported as-is.

Given all of that, let’s first summarize the RMX system calls that we use and then see where that leads us.

Semaphores
rqcreatesemaphore
rqdeletesemaphore
rqsendunits
rqreceiveunits
Regions
rqcreateregion
rq_send_control / Isolated to get_region & free_region
rq_receive_control / Isolated to get_region & free_region
rq_accept_control / Isolated to get_region & free_region
Mailboxes
rqcreatemailbox
rqdeletemailbox
rqsenddata
rqreceivedata
rqreceivemessage
Memory
rqcreatesegment
rqdeletesegment
rqecreatedescriptor
rqedeletedescriptor
rqegetaddress
rqegetpoolattrib
Task Mgmt.
rqcreatejob
rqcreatetask
rqsuspendtask
rqdeletetask
rqgettasktokens
rqgettype
rqsleep
rqgetexceptionhandler
rqsendcontrol
rqreceivecontrol
rqsuspendtask
rqesetmaxpriority
Object Mgmt.
rqcatalogobject
rqlookupobject
rquncatalogobject
Interrupt Mgmt.
rqsetinterrupt
rqresetinterrupt
rqsignalinterrupt
rqwaitinterrupt
rqexitinterrupt
rqgetlevel

The RMX system calls have not been well encapsulated in the existing code. For RMX regions which were implemented later, most RMX calls to send and free a region are already isolated into two local “C” modules though createregion is still done locally. There exists get and free_semaphore routines but they are not universally used.Segment and descriptor calls of course have no meaning on RTEMS. If we take the approach of using the EPICS OSI library routines then we’re also insulated for possible (but unlikely) future migrations at the cost of making a large number of code changes.