FHIR Transform Engine

A White Paper from Open Mapping Software Ltd

R.P. Worden

December 2016

Abstract: Adapting an application as a FHIR server involves several kinds of data transformation, to translate information between the application’s own internal data representations, and the FHIR representations. Developing these transformscan be time-consuming and error-prone. Using the FHIR Transform Engine (FTE)the transformscan be automatically generated from declarative mappings, rather than hand-coded.

With this mapping approach, an existing healthcare application can be enabled as a read-only FHIR server without writing any database-specific or resource-specific code –by defining database mappings and some other small configuration files. This takes a few days’ development effort per FHIR resource. This white paper describes the mapping approach, the development tools, and their potential to accelerate the integration of healthcare applications.

The Impact of FHIR

In the few years since its introduction, FHIR has become the mainstream approach for the integration of healthcare applications. For any healthcare integration project - local, regional or national - the question is no longer: "What healthcare data standards shall we use?", but: "Is there any good reason not to use FHIR". Usually there is not. And even if you choose not to use FHIR (for instance, if you need detailed clinical models in some more specialised representation), you will still need to integrate your application with FHIR.

Therefore FHIR is now the basis of most new healthcare applications. So there is an increasing need to interface existing healthcare applications - with their specialised, local or legacy data representations - with FHIR. The FHIR Transform Engine is designed to do this. It is ,we believe, the most cost-effective way to do so.

Using FHIR for Local Application Integration

When a healthcare provider adopts FHIR for local integration, what steps do they need to take? FHIR is compatible with many interoperability paradigms and architectures, including REST, messaging, and SoA, so no major new architecture or infrastructure re-set is required. There is no need to rip and replace core applications by new FHIR versions. The important steps are incremental, business-led and clinical-led. These steps are:

  1. Decide, on business and clinical grounds, which applications most urgently needimproved integration.
  2. Decide which FHIR resources these applications need to exchange in order to work better together.
  3. Find or make the appropriate FHIR profiles.
  4. Enable those applications as FHIR servers and clients, for the chosen resources
  5. Exchangeand compare resources between the applications , in FHIR data formats
  6. Learn the business, clinical and technical lessons; initiate required changes to business and clinical processes

This need not be a large ‘bet the farm'technology-led initiative; it is an iterative, business-led and clinically-led initiative, with the modest technical effort focused around step (D). The rest of this paper addresses step (D) – enabling existing applications as FHIR clients and servers –anddescribes how it can be done with modest cost, risk and elapsed time, using the FHIR Transform Engine.

Most existing applications follow a three-layer architecture, of:

  1. User Interface
  2. Business Logic
  3. Database (often Relational; or other, viewed through an object mapping layer such as Hibernate).

To integrate two such applications with FHIR, you need to build a FHIR adapter in front of each application. When application A is acting as a FHIR client, while B is a FHIR server, both using relational databases, this looks like:

Initially we assume that Application B is acting as a read-only FHIR server – able to respond to FHIR searches by sending the selected resources, but making no changes to its own data. Application A, which acts as the FHIR client, may need to update its own data. For application A, decisions about when to update data may need to involve the business logic and the user interface; but for Application B, they do not. The FHIR adapter for B needs just read-only access tothe underlying database, if the adapter is designed in that way. We shall see that there are someadvantages in doing so.

Consider the sequence of events which occur when the client Application A requests some information from the server application B, using a FHIR RESTful interface. The simplest sequence is:

  1. (Client A) compose and send a FHIR search request, on a resource in server B.
  2. (Server B) translate the FHIR search request into its own internal search commands. This could, for instance, be an SQL query on its database. Run the query.
  3. (Server B) translate the results of the internal search into FHIR data format. Send the resources to A in an AtomFeed bundle.
  4. (Client A) translate the incoming FHIR resources to its own internal data representations, and use business logic to act on them.

This involves three distinctpieces of data translation, between the FHIR representation and some internal data representation of an application. These are:

Source Data Representation / Target Data Representation
FHIR search request / Server (B) internal search instructions
Server (B) internal data structures, returned by the search / FHIR resource (XML or JSON)
FHIR resource (XML or JSON) / Client (A) internal data structures

Although FHIR uses simple data structures as far as possible, the data structures of applications A and B may not be simple; and the intrinsic complexity of the healthcare domain mean that these three data transformations will not always be straightforward. All three are needed for even a simple interaction. Developing and testing these transforms may require significant effort and elapsed time, if they are developed in a procedural language without any use of automated tools.

So even with FHIR, integration projects may not be straightforward, without using appropriate tools. The good news is this:

By using the FHIR Transform Engine, all three types of transform can be automatically generated from declarative mappings, rather than being coded in a procedural language. This process is much less time-consuming and more reliable than developing procedural transform code. Using staff who are experienced in the mapping techniques and understand the database structures, any existing application can be enabled as a read-only FHIR server in a few day’s effort per resource type.

To understand how this can be done, it is necessary to understand a bit about how the FTE works.

FHIR Transform Development Toolset

The FHIR Transform Engine makes runtime transformation to and from FHIR by using sets of mappings, rather than procedural code. The toolsused to develop and run these mappings differ from most mapping and transform tools (such as AltovaMapForce, or BizTalk Mapper) in one key respect.

In most mapping and transform toolsets, the user defines direct mappings from one data structure (such as an XML tree, or a relational database) to another – typically using drag-and-drop across the two structures to make eachmapping. Then the tools attempt to generate a transform between the two data structures. However, they often do not succeed (e.g. when there are structureclashes between the data structures – as there often are). When those tools do not succeed in generating a transform, you need to do the job with procedural code, and you are in effect back where you started.

The FTE mapping tools do not map directly from one data structure to another. Instead, they map several data structures onto one common logical model of the information – which is the FHIR resource model. Whenever two or more data structures have been mapped to the same logical model, the tools can automatically generate and run the transforms to and from FHIR, or from any mapped data structure to any other. This has bigbenefits over the direct structure-to-structure mapping tools:

  • Mappings are easier to make, because you do not need to bridge between two different complex data structures in one mapping operation. You only bridge from one complex data structure to a logical model.
  • Mappings are declarative and easily testable, with testing tools in the mapping toolset. Therefore the generated transforms are much more reliable.
  • When the data structures differ markedly (e.g.a flat relational database, versus deep XML) the tools generate accurate transforms where other tools fail.

The FTE use of this mapping approach to FHIR is shown here:

The central logical model (which all the data structures are mapped to) is the logical model of a FHIR feed and resources – generated automatically from the FHIR specification and whatever set of profiles you are using. The mappings are created and tested using the mapping tools.

If any data structure is mapped onto a FHIR resource class model, the tools can automatically generate and run transforms in both directions between that data structure and FHIR, either in XML or JSON form. This can be done for commonly used healthcare data structures such as HL7 Version 2 or CDA – or for any relational database.

When an application’s data is stored in a relational database (or on any database using a Hibernate persistence layer), data in the database can be transformed to FHIR (in both directions) using transforms generated automatically from database mappings. This is a fast and effective way to create the transforms in rows (2) and (3) of the table above. The other requiredtransforms – from a RESTfulFHIR search string to an SQL or HQL query on the database – arealso generated automatically by the tools, from the database mappings.

Therefore all the transforms required for a FHIR adapter to an application with a database can be generated automatically from mappings, and need not be hand-coded. This is a major saving in the development effort to build a FHIR adapter.

The screenshot below illustrates what you do, using the FTE development tool, to develop mappings of a data structure (in this case, HL7 V2) onto a FHIR resource.

The centre pane shows the structure of the HL7 V2 message. The right-hand pane shows theclass model for a bundle containing a set of FHIR resources.

A mapping is a declarative statement which means: ‘node A in the data structure represents feature B in the class model’. To make this mapping, you select a node in the class model tree, select the appropriate node in the central structure tree, and make the mapping using a menu command. Simple mappings take a few seconds to make.

An important aspect of mapping any data structure onto a FHIR resource is the data format conversions, required to convert from the data structure's representation of some property, to the standard FHIR data formats. If these format conversions are simple translations of a small number of code values, a conversion table can be defined directly in the mappings. For more complex format transforms, you may need to write a small piece of Java code (to be invoked by the mappings) to make the conversions, if such code does not already exist in a library of common format transforms. This code can access any part of the input or output messages, and can be written to perform any specialised function you require. Usually , it is possible to make any such code highly localised, modular, and reusable.

A key part of developing any set of transforms is testing the transforms, to make sure they do what you intend. Here, the FTE toolset provides mature and capable support. Because one set of mappings can generate transforms in either direction (from some other format to FHIR, and back from FHIR to the other format) it is possible to test round trips and sequences of transforms. The FTE has built in tools to do this, and to evaluate the results automatically (as well as inspect them).

A screenshot of the testing tool is shown below, testing a transform between HL7 V2 and FHIR.

Using the testing tool, you can rapidly:

  • Run all the possible transforms and round trips between a two or more data formats, including FHIR
  • View summary scores of all test results (in the right-hand pane)
  • Make automated comparisons of input and output files
  • See diagnostic messages of issues which arose in transformation (in the lower pane)
  • Open and view the result of any transform or round-trip (in the left-hand pane).

This powerful test methodology soon exposes any errors in mappings. These tests can be done immediately and automatically, after making any change to the mappings. So you can developmappings using a highly effective iterative map/test/map/test cycle, seeing the results at any stage.

Once you understand the principles of mapping, it is straightforward to make and test the mappings. Typically, the mappings required for one typical FHIR resource to an application database can be made in a few days’ effort (at most).

Configuring a FHIR server

Once you have mapped a database to a FHIR resource, it is a straightforward process to build a working FHIR server which uses that database to serve the resource. No further coding is required. You need only provide four simple configuration files to define the server. These are:

  1. servers.csv defines the FHIR servers provided by the web service. For each server, it defines the required connection to the application database.
  2. resources.csv defines the resources supported by each server. For each resource, it defines the mapping file which maps from the database onto the resource class model
  3. searches.csv defines the FHIR searches supported for each resource
  4. A small XML narrative template file per resource defines the format and content of an automatically generated narrative for the resource.

Having built these configuration files, you assemble them together with the necessary java jar files and mapping files into therequired folder structure, package it as a .war file, and it will run the FHIR service (at least under Tomcat).

How the FHIR Server Works

When one of these FHIR servers responds to a FHIR search request, the following happens on the server:

  1. The FHIR search request is converted to an object query which uses only the language of the FHIR resource class model.
  2. Using the database mappings, the object query is converted to an SQL or HQL query against the database, to retrieve the logical ids of all resources which satisfy the query conditions.
  3. The query is run to retrieve the resource ids. Another filter enforces any conditions which are not expressed in SQL or HQL.
  4. A further query is generated and runonce for each resource id, to retrieve all mapped features of that resource
  5. Using the mappings, the retrieved recordsfor each resource id are converted into a FHIR resource instance, expressed in EMF Ecore.
  6. The resource narrative is constructed from the resource instance and its narrative template file
  7. The resources are packaged up into an AtomFeed bundle, and are sent to the client (using the bridge and the java FHIR reference implementation)

An architecture which does this is shown here:

All this happens with no resource-specific or database-specific code, except possibly for a few small data format conversions.

FHIR Servers and Clients with Update Capability

So far we have discussed read-only FHIR servers, which can respond to searches and provide data in FHIR resource form, but which do not update their databases in response to update requests.

If any application is to update its database, we would expect that the application’s business logic will be involved in deciding whether and when to update the database. So it is generally not possible to adapt an application as a read/write FHIR server without some coding in the business logic layer.

Even in this case, mapping the application database onto the FHIR resource provides a significant benefit. Given the mappings, a generated transform can be run in the reverse direction, to transform an incoming FHIR resource into an XML tree version of all the changes which will need to be made in database tables. So while the business logic of making the database update is not addressed by the mapping tools, the data transform aspect is addressed. This applies to applications acting as FHIR clients, as well as those acting as servers.

FHIR and Data Quality

When two or more applications have been enabled as FHIR servers for the same resource type – for instance, enabled to find and return Patient resources – it is then possible to search the same resource on all those servers, and compare the results. Comparisons are possible and easy because all resource data are returned in the same FHIR logical form, independent of which server returned them. It becomes easy either to inspect the data from two or more servers side by side, or to build software which assists in making the comparison. Issues of data quality and consistency across the different applications are easily exposed.

The mapping toolset which is used to configure the FHIR servers includes a comparative query tool, which makes it easy to run these comparisons, and inspect and store the results. The process for doing this is as follows:

  1. Compose a query in a simple object-oriented query language, which selects FHIR resources and displays some of their properties. Queries are expressed entirely in terms of the FHIR resource class models.
  2. Connect the query tool to a number of application databases which have been mapped to a FHIR resource model.
  3. Using the mappings, the query tool automatically converts the query to SQL retrievals against each database.
  4. The query tool runs the SQL queries and translates the results to express them in terms of the FHIR resource logical model
  5. The results are displayed in tabular form, which can be sorted on any columns, or saved as csv files/spreadsheets

Inspecting these query results give a clear view of the data quality and consistency between the different application databases. An example query (using artificial data) is shown below. First, the query tool is connected by jdbc to two application databases, both of which have been mapped to the FHIR patient resource: