HL7 processing tutorial

Prerequisites

It is assumed that you have a basic understanding of Camel and Groovy.
It is assumed that you have read the first details tutorial or at least the first steps tutorial.
Also make sure you've correctly set up your development environment.

This tutorial guides you through an HL7 version 2 message processing example. Message processing is done via IPF's HL7 DSL and via HL7-specific extensions to the Camel DSL. We will create an IPF application that reads an HL7 version 2 message from an HTTP endpoint, validates and transforms the message, and writes the transformation result to an output file.

HL7 features of Camel and IPF

This tutorial currently doesn't use Camel's HL7 features. Camel's HL7 features and those provided by IPF are complementary to each other and can perfectly be combined. For example to receive HL7 version 2 messages via MLLP you can do so using Camel's HL7 component.

Validation

Let's look at the message we will use in our example. The message is an ADT A01 message, version 2.2, and we want to enforce that

all primitive type values have the correct format and
the message itself contains a defined sequence of segments

The HL7 module of IPF comes with an HL7 validation DSL that makes the definition of validation rules very easy. Even better, IPF provides a complete set of rules that check whether the primitive type values meet the constraints defined in the HL7 specification.

Transformation

Now lets look the transformation we want to perform if we passed validation.

The room number and bed number from the PV1[3] field shall be dropped. This field is a composite field and we want to drop the second and the third component.
The birth date in PID[7] shall be reformatted by dropping the last 6 digits. This field is also a composite field with only a single component.
The gender code in PID[8] shall be mapped to a code from another code system. We will use a simple code mapping service for that.

Finally, we will derive the destination file name from the MSH[4] field (sending facility).

Route design

Here's the message processing route using enterprise integration pattern symbols.

An ADT A01 event message is received via a jetty endpoint (inbound HTTP endpoint).
The message is validated as described above.
The message is forwarded to a transformer that makes the changes to the HL7 message as described above.
The transformation result is placed into different files whose names are derived from the MSH[4] field.

Source code

The source code for this tutorial can be downloaded from here.

Project creation

We start by creating an example IPF project by entering

mvn org.apache.maven.plugins:maven-archetype-plugin:2.0-alpha-4:generate 
-DarchetypeGroupId=org.openehealth.ipf.archetypes 
-DarchetypeArtifactId=ipf-archetype-basic 
-DarchetypeVersion=2.0.0 
-DgroupId=org.openehealth.tutorial 
-DartifactId=hl7 
-Dversion=1.0-SNAPSHOT 
-DinteractiveMode=false

on the command line (make sure to have the command on a single line). This will create a folder named hl7 in the current directory. Change to the hl7 folder an enter

mvn install

This will compile the project and install the project artifacts into your local Maven cache. To import the project into Eclipse navigate to File->Import->Existing Projects into Workspace in the Eclipse menu and select the created hl7 folder as root directory. After having imported the project into Eclipse it should look like in the following figure.

You can find a detailed description of the created project structure in the project creation section of the first details tutorial.

Clean project after import

Eclipse might report compile errors for the imported project. In this case it is sufficient to clean the org.openehealth.tutorial.hl7 project. Select the project and then select Project->Clean ... from the main menu. In the dialog select Clean projects selected below (make sure that the org.openehealth.tutorial.hl7 project is actually selected) and press the OK button.

Extend project descriptor

In order to enable HL7 processing and communication over HTTP we have to include further dependencies into the Maven project descriptor pom.xml. These are

Dependency	Description
`platform-component-hl7`	IPF component that provides HL7-specific extensions to the Camel DSL.
`modules-hl7dsl`	IPF component that provides the HL7 DSL and is transitively included via `platform-camel-hl7`.
`modules-hl7`	IPF component that provides the HAPI extensions and is transitively included via `platform-camel-hl7`.
`camel-jetty`	Camel component that enables inbound communication over HTTP.

The pom.xml file is located directly under the project's root directory. Here's an excerpt of the extended project descriptor.

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project
    xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>org.openehealth.tutorial</groupId>
    <artifactId>hl7</artifactId>
    <name>hl7</name>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.openehealth.ipf.platform-camel</groupId>
            <artifactId>platform-camel-hl7</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-jetty</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>ca.uhn.hapi</groupId>
            <artifactId>hapi-structures-v22</artifactId>
            <version>0.6</version>
        </dependency>
        <dependency>
            <groupId>ca.uhn.hapi</groupId>
            <artifactId>hapi-structures-v25</artifactId>
            <version>0.6</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>2.5.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.4</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        ...
    </build>

    ...

</project>

Extend application context

The components we included in the previous section must also be configured in the Spring application context.xml file. This file is located under src/main/resources. In addition to the default configuration created by the archetype we need to

Register the HL7 DSL extensions provided by the platform-camel-hl7 component at the model extender
Register the HAPI extensions provided by the modules-hl7 component at the model extender
Configure a mapping service that we will use for code mappings
Add a couple of beans needed for HL7 validation

Here's the complete application context.xml file.

context.xml

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xmlns:camel="http://camel.apache.org/schema/spring"
       xsi:schemaLocation="
http://www.springframework.org/schema/beans 
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/lang 
http://www.springframework.org/schema/lang/spring-lang-2.5.xsd
http://camel.apache.org/schema/spring 
http://camel.apache.org/schema/spring/camel-spring.xsd">

    <camel:camelContext id="camelContext">
        <camel:routeBuilder ref="routeBuilder"/>        
    </camel:camelContext>

    <bean id="routeBuilder" depends-on="routeModelExtender"
        class="org.openehealth.tutorial.SampleRouteBuilder">
    </bean>

    <!-- Code mapping service using map.groovy as mapping table -->
    <bean id="mappingService" 
        class="org.openehealth.ipf.commons.map.BidiMappingService">
        <property name="mappingScript" value="classpath:map.groovy"/>
    </bean>

    <!-- DSL extensions provided by platform-camel-core -->
    <bean id="coreModelExtension" 
        class="org.openehealth.ipf.platform.camel.core.extend.CoreModelExtension">
    </bean>

    <!-- DSL extensions provided by platform-camel-hl7 -->
    <bean id="hl7ModelExtension"
        class="org.openehealth.ipf.platform.camel.hl7.extend.Hl7ModelExtension"/>

    <!-- HAPI extensions provided by modules-hl7 -->
    <bean id="hapiModelExtension"
        class="org.openehealth.ipf.modules.hl7.extend.HapiModelExtension">
        <property name="mappingService" ref="mappingService" />
    </bean>

    <bean id="mappingExtension" 
        class="org.openehealth.ipf.commons.map.extend.MappingExtension">
        <property name="mappingService" ref="mappingService" />
    </bean>

    <bean id="routeModelExtender" 
        class="org.openehealth.ipf.platform.camel.core.extend.DefaultModelExtender">
        <property name="routeModelExtensions">
            <list>
                <ref bean="coreModelExtension" />  <!-- Register core DSL extensions -->
                <ref bean="hl7ModelExtension" />   <!-- Register HL7 DSL extensions -->
                <ref bean="hapiModelExtension" />  <!-- Register HAPI extensions -->
                <ref bean="mappingExtension" />    <!-- Register mapping extensions -->
            </list>
        </property>
    </bean>

   <!-- All we need for validation: a validator, a validation context, -->
   <!-- and two set of rules                                           -->
   <bean id="myValidatorBean" class="org.openehealth.ipf.modules.hl7.validation.support.HL7Validator"/>

   <bean id="validationContext"
       class="org.openehealth.ipf.modules.hl7.validation.ValidationContextFactoryBean"/>

   <bean id="defaultTypeRules"
       class="org.openehealth.ipf.modules.hl7.validation.builder.DefaultTypeRulesBuilder"/>

   <bean id="myCustomRules"
       class="org.openehealth.tutorial.SampleRulesBuilder"/>
       
</beans>

The mapping table map.groovy referenced by the mappingService bean is described in the code mapping section.

Route definition

Lets start with an overview of the complete route and then discuss it step by step. Before doing so open the SampleRouteBuilder.groovy file in Eclipse and replace the existing routes in the configure() method.

'SampleRouteBuilder.groovy'

package org.openehealth.tutorial

import ca.uhn.hl7v2.validation.ValidationContext

import org.apache.camel.Exchange
import org.apache.camel.spring.SpringRouteBuilder

class SampleRouteBuilder extends SpringRouteBuilder {

    void configure() {
        from('jetty:http://localhost:8080/tutorial')                  // start HTTP server
            .to('direct:input')                                       // forward request

        from('direct:input')                                          // receive HL7 message
            .unmarshal().ghl7()                                       // create message adapter (HL7 DSL support)
            .validate().ghl7()
                .profile(lookup(ValidationContext.class))             // validate against custom validation context
            .transmogrify { msg ->
                msg.PV1[3][2] = ''                                    // clear room nr.
                msg.PV1[3][3] = ''                                    // clear bed nr.
                msg.PID[7][1] = msg.PID[7][1].value.substring(0, 8)   // format birth date
                msg.PID[8]    = msg.PID[8].mapGender()                // map 'gender' code
                msg                                                   // return result
            }
            .setHeader(Exchange.FILE_NAME) {exchange ->               // set filename header to
                exchange.in.body.MSH[4].value + '.hl7'                // sending facility (MSH[4])
            }
            .marshal().ghl7()                                         // convert to external representation
            .to('file:target/output')                                 // write external representation to file
    }    
    
}

First, we define a jetty endpoint for receiving HL7 messages over HTTP.

...
from('jetty:http://localhost:8080/tutorial')                  // start HTTP server
    .to('direct:input')                                       // forward request
...

This will start an HTTP server listening on port 8080. The context path is tutorial. The message is then forwarded to a direct endpoint that we also use inside our JUnit tests for sending test messages. HL7 messages arrive as InputStream at the direct:input endpoint . We won't work on this representation of the HL7 message. Instead we create a so-called message adapter that implements the HL7 DSL. Creation of a message adapter from an InputStream is done via unmarshal().ghl7() (this also works if the message is a String).

...
from('direct:input')                                          // receive HL7 message
     .unmarshal().ghl7()                                      // create message adapter (HL7 DSL support)
...

The next step in the HL7 message validation using the HL7 Validation framework.

...
.validate().ghl7()
    .profile(lookup(ValidationContext.class))             // validate against custom validation context
...

The validation is executed against the HAPI message object. The profile DSL extension initializes the validator with the HL7 ValidationContext that has been injected into the route builder.

Now, we add our custom SampleRulesBuilder.groovy script, that defines the allowed sequence and cardinalities of HL7 groups and segments for this message. The file is added to the src/main/groovy directory in the package org.openehealth.tutorial:

SampleRulesBuilder.groovy

package org.openehealth.tutorial

import ca.uhn.hl7v2.validation.ValidationContext

import org.openehealth.ipf.modules.hl7.validation.builder.RuleBuilder
import org.openehealth.ipf.modules.hl7.validation.builder.ValidationContextBuilder

class SampleRulesBuilder extends ValidationContextBuilder {

    // We define only a subset of the segments defined in the HL7 2.2 spec
    
    public RuleBuilder forContext(ValidationContext context) {
        new RuleBuilder(context)
          .forVersion('2.2')
            .message('ADT', 'A01').abstractSyntax(
                    'MSH',
                    'EVN',
                    'PID',
                    [  {  'NK1'  }  ],  
                    'PV1',
                    [  {  INSURANCE(
                              'IN1',  
                              [  'IN2'  ] , 
                              [  'IN3'  ]  
                    )}]                                                  
            )
    }
    
}

The sequence and cardinality of groups and segments is defined in a syntax that is very closely related to the HL7 Abstract Message Syntax. The message is required to contain a MSH, EVN, PID and PV1 segment; it may contain any number of NK1 segments and it may contain a repeatable INSURANCE group. For detail, please read the documentation on HL7 validation.
As our test message matches this definition, we expect the message to pass validation.

The next step in the route is the HL7 message transformation using the HL7 DSL.

...
.transmogrify { msg ->
    msg.PV1[3][2] = ''                                    // clear room nr.
    msg.PV1[3][3] = ''                                    // clear bed nr.
    msg.PID[7][1] = msg.PID[7][1].value.substring(0, 8)   // format birth date
    msg.PID[8]    = msg.PID[8].mapGender()                // map 'gender' code
    msg                                                   // return result
...
}

We do the transformation with a transmogrifier closure. Inside the closure we use the HL7 DSL. The HL7 DSL allows us to manipulate HL7 messages on a very high level without dealing with low level HL7 API details. You immediately see how the message is processed by looking at the code. For accessing a message segment you directly use the segment name like msg.PV1 or msg.PID. Fields and sub-fields (components of a composite field) are accessed by indices starting from 1. For example, PV1[3][2] denotes the second component of the third PV1 field. For a complete reference of the HL7 DSL refer to the HL7 DSL section of the reference manual. Code mapping is done with the mapGender() method. This method translates the English F code (female) contained in PID[8] to a German W code (weiblich). This will be explained in more detail in the code mapping section.

The transformation result can now be written to a file. We derive the name of the file from the content of the MSH[4] field. For example, from HZL we write a file named HZL.hl7, from PKL a file named PKL.hl7 ... and so on. We achieve this by setting the Exchange.FILE_NAME message header which is understood by the file component.

...
.setHeader(Exchange.FILE_NAME) {exchange ->               // set filename header to
    exchange.in.body.MSH[4].value + '.hl7'                // sending facility (MSH[4])
}
.marshal().ghl7()                                         // convert to external representation
.to('file:target/output')                                 // write external representation to file
...

For setting the message header we use the setHeader() method. Inside the setHeader closure we again use the HL7 DSL to access the MSH[4] field. Before the message can actually be written to a file we have to marshal the message adapter with marshal().ghl7(). This is the reverse operation to unmarshal().ghl7() at the beginning of the route. The file endpoint is configured in a way that it writes files to the target/output directory. Writing to an existing file will overwrite its content. Before we can start testing the route we have to setup the code mapping table.

Code mapping

In the extend application context section we've configured the code mapping service to use a mappingScript named map.groovy. In our example, we only need a single gender mapping table with a single entry that maps F to W.

map.groovy

mappings = {
  gender(
    F      : 'W',

    (ELSE) : { it }
  )
}

This format is valid Groovy syntax and is understood by the default bi-directional mapping service provided by IPF. For a complete reference refer to the mapping service section of the reference manual. You may also provide your own mapping service implementation by implementing the org.openehealth.ipf.modules.hl7.mappings.MappingService interface. To install the mapping table create a map.groovy file under src/main/resources and copy the above mappings block into that file.

It is interesting to see how the gender table is selected for translating F to W. There is a correspondence between the name of the mapping method and the name of the selected code table. Using mapGender() instructs IPF to use the gender mapping table. If you write mapXyz() IPF would try to find a table named xyz ... and so on. You may also use the map*() methods on java.lang.String objects like in the following example.

assert 'W' == 'F'.mapGender()

This is achieved via dynamic Groovy language features.

Route testing

Automated test

For automated testing we will use the following HL7 message.

msg-01.hl7

MSH|^~\&|SAP-ISH|HZL|||20040805152637||ADT^A01|123456|T|2.2|||ER
EVN|A01|20040805152637
PID|1||79471||Meier^Elfriede|Meier|19400101000000|F|||Hauptstrasse 23^^Essen^NW^11000^DE^H|||||S|||111-11-1111||||Essen
NK1|1|Meier^Elfriede|EMC|Hauptstrasse 23^^Essen^NW^11000^DE|333-4444~333-5555|
PV1|1|I|ISKA^13^4|R||||823745217||||||||N|||79237645|||||||||||||||||||||||||20040805000000

After processing we expect the following output.

msg-01.hl7.expected

MSH|^~\&|SAP-ISH|HZL|||20040805152637||ADT^A01|123456|T|2.2|||ER
EVN|A01|20040805152637
PID|1||79471||Meier^Elfriede|Meier|19400101|W|||Hauptstrasse 23^^Essen^NW^11000^DE^H|||||S|||111-11-1111||||Essen
NK1|1|Meier^Elfriede|EMC|Hauptstrasse 23^^Essen^NW^11000^DE|333-4444~333-5555|
PV1|1|I|ISKA|R||||823745217||||||||N|||79237645|||||||||||||||||||||||||20040805000000

Create two files, msg-01.hl7 and msg-01.hl7.expected, with the above content under the src/test/resources folder.

These files will be used inside our JUnit test. To implement the test open the SampleRouteTest.java file in Eclipse and replace its content with the following.

package org.openehealth.tutorial;

import static org.junit.Assert.assertEquals;

import org.apache.camel.ProducerTemplate;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openehealth.ipf.modules.hl7dsl.MessageAdapters;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.TestExecutionListeners;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.support.DependencyInjectionTestExecutionListener;

@RunWith(SpringJUnit4ClassRunner.class)
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class})
@ContextConfiguration(locations = { "/context.xml" })
public class SampleRouteTest {

    @Autowired
    private ProducerTemplate producerTemplate;

    @Test
    public void testRoute() throws Exception {
        Resource input = new ClassPathResource("/msg-01.hl7");
        producerTemplate.requestBody("direct:input", input.getInputStream());
        Resource result = new FileSystemResource("target/output/HZL.hl7");
        assertEquals(
                MessageAdapters.load("msg-01.hl7.expected").toString(),
                MessageAdapters.make(result.getInputStream()).toString());
    }

}

The testRoute method opens an InputStream on the input file and sends that stream in the in-message body to the direct:input endpoint. The processing result is loaded from the created HZL.hl7 file and compared with the expected processing result. For reading HL7 messages from a stream or a resource we use the MessageAdapters utility class. To execute the test right-click on SampleRouteTest.java and select Run As->JUnit Test from the context menu. The test should now successfully execute and you should also see an HZL.hl7 file in the target/output directory containing the processing result.

Just for fun, modify the validation part, so that the message does not validate against the rules. Modify the SampleRulesBuilder.groovy script and comment the EVN segment

          .forVersion('2.2')
            .message('ADT', 'A01').abstractSyntax(
                    'MSH',
                 // 'EVN',
                    'PID',
                    [  {  'NK1'  }  ],

If you run the test again, the console shows exceptions and the test fails:

...
SEVERE: Invalid message
ca.uhn.hl7v2.validation.ValidationException: The structure EVN appears in the message but not in the profile
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
...

Before continuing, undo the change in SampleRulesBuilder.groovy by removing the comment again.

Manual test

For manual testing we use the Eclipse HTTP Client to send HL7 messages to the jetty endpoint that we configured in SampleRouteBuilder.groovy. Before we can send messages over HTTP we have to start the route server. To do so, right-click on SampleServer.java and select Run As->Java Application. Now we have a server running that accepts HL7 messages on port 8080 under the tutorial context path. Then open the Eclipse HTTP client and

Enter http://localhost:8080/tutorial in the address field (top-left corner of the window)
Select POST in the HTTP method field (top-right corner of the window)
Copy the following HL7 message into the Body field

MSH|^~\&|SAP-ISH|PKL|||20040805152637||ADT^A01|123456|T|2.2|||ER
EVN|A01|20040805152637
PID|1||79471||Meier^Elfriede|Meier|19400101000000|F|||Hauptstrasse 23^^Essen^NW^11000^DE^H|||||S|||111-11-1111||||Essen
NK1|1|Meier^Elfriede|EMC|Hauptstrasse 23^^Essen^NW^11000^DE|333-4444~333-5555|
PV1|1|I|ISKA^13^4|R||||823745217||||||||N|||79237645|||||||||||||||||||||||||20040805000000

This is the same message we used for the JUnit test but with a different MSH[4] value (PKL instead of HZL). To submit the request press the green arrow at the top of the window. The result should look like:

Because we derive the result filename from the MSH[4] field we should now see a PKL.hl7 file in the target/output directory.

Remember to terminate the server that we started for the manual test. You can do this in Eclipse within the Console-Window by clicking the red terminate icon:

Assembly and installation

We finally want to create a distribution package from our project, install (unzip) that package somewhere and start a standalone integration server (i.e. an integration server that runs outside Eclipse). Before continuing make sure that you stopped the SampleServer that you started within Eclipse, otherwise, you won't be able to start another server. To create the package enter

mvn assembly:assembly

on the command line. The created package hl7-1.0-SNAPSHOT-bin.zip is written to the project's target folder. Copy the package to a new location and unzip it. This will create a folder named hl7-1.0-SNAPSHOT with the following content:

The lib folder contains the project jar file (hl7-1.0-SNAPSHOT.jar) as well as all required runtime dependencies. The conf folder contains a log4j.xml configuration file. Startup scripts are located directly under the root folder. The way how the project is packaged can of course be customized by changing the project's src/main/assembly/bin.xml assembly descriptor. The script startup.sh is currently empty i.e. for testing-purposes you have to run the server on Windows using startup.bat.

Start server

To start the server run startup.bat. The console output should like like

Finally, submit the HTTP request again that you've configured before with the Eclipse HTTP client. This should again create a file PKL.hl7 in the newly created target/output folder. The server can be stopped by pressing CTRL+C.

HL7 processing tutorial