lib7 - a fast hl7 parser

Synopsis

The goal of lib7 is to be a fast hl7 parser library that can deal with basically any hl7 files as long as the delimiters are used apropriately.

Features:

• reduce malloc() and getc() to a minimum, these seem to be the 2 bottle necks when parsing hl7 files
• make use of delimiter declaration in MSH-1 (default |) and MSH-2 (default ^~\&)
• parse any hl7 file, we do not care if the schema is valid (many systems that produce hl7 also don't)
• compile as static and dynamic library to be used in C/C++ applications and language bindings

It is used as a library for 7view (a Qt5 base hl7 viewer) and as library for Python 3 bindings which can be used for experimentation and rapid development.

For lib7 being quick at parsing hl7 files, the following compromises have been made:

• every element content is a byte array, lib7 has no understanding encodings, these high level functionality must be implemented by the user
• lib7 has no understanding of encodings, see point 1
• lib7 does not care about hl7 schemas, this is actually a feature since real world experience shows, that many system produce hl7 files that are structurally invalid

Using lib7

Minimal parser example

If you just want to use lib7 as a shared library the library is built as lib7.so with a corresponding lib7.h.

#include <stdio.h>
#include "lib7.h"

int main(int argc, char* argv[]) {
    // open a file handle to your hl7 file
    FILE *fd;
    if ((fd = fopen(argv[1], "rb")) == NULL) {
        print_error(errno, argv[1]);
        return 1;
    }

    // parse hl7 file
    message_t *message = create_message_t(NULL);
    ret = hl7_decode(fd, &message);

    // we have everything in memory, file can be closed
    fclose(fd);

    // check for parser errors
    if (ret != 0) {
        fprintf(stderr, "We failed to parse message at line %d, exiting.\n", message->num_children);
    }

    // to see how to access elements in the returned structure message, see:
    // dump_structure(message);

    // release resources
    if (message != NULL)
        free_message_t(message);
}

In the above example there is some magic happening which you may or may not like.

• message_t->meta contains delimiter information about a hl7 file. If this structure is not explicitly initialized and set, then the parser tries to read the delimiter information from the file. This is a feature but might fail on malformed hl7 files. For example, some hl7 files start with MSH|||||:
  • character 4 defines the field separator meta_t->sep_field which defaults to |
  • if there is an empty filed after the first | we assume default separator characters ^~\&, this might be wrong
  • The parser tries to guess the line separator (default \r but in the wild it may also be \n or \r\n). if the line separator is \r\n (Windows, 2 character delimiter) the meta_t->crlf is set to 1 else 0. meta_t->crlf is initially set to -1 which means the parser has to guess at the first encounter of a newline character.
  • the parser tries to detect if the file starts with a bom and ignores the bom characters

Better control over separators

If you know the separators in a file and do not trust the autodetection (or the file defines wrong separators in MSH-1 and MSH-2) then you have 2 options:

Manual control

To have better control over separators, you may initialize meta_t first:

// create new meta_t structure with default values
meta_t *meta = init_hl7_meta_t();

// set delimiters manually so the parser does not have to guess
meta->sep_message = 0x13; // for windows, first line separator
meta->crlf = 1;           // this will skip 0x10 that follows 0x13

// the parser relies on the separators to be ascii characters.
// you may set all or partial delmiiters, init_hl7_meta_t() will 
// set default values according to the HL7 specification
meta->sep_field   = '§';  // default '|';
meta->sep_comp    = '°';  // default'^';
meta->sep_rep     = '@';  // default'~';
meta->sep_escape  = '#';  // default'\\';
meta->sep_subcmp  = '^';  // default'&';

// check if this file has a bom; you'll have to do this manually.
// this will move df at the first byte after the BOM
meta->bom = detect_bom(fd); 

// parse hl7 file
message_t *message = create_message_t(meta);
ret = hl7_decode(fd, &message);

See 7parse.c for an example implementation.

Autotdetect and manipulate

You may also manually detect the separators, then check and adjust, for example:

int ret = -1;

// initialize separator data structure
hl7_meta_t* meta = init_hl7_meta_t();

// check if this file has a bom;
// this will move df at the first byte after the BOM
meta->bom = detect_bom(fd); 

// find delimiters, this will read the first bytes of the file to find MSH-1 
// and MSH-2 if possible. The file pointer will be reset to the end of BOM or 
// beginning of file if there is no BOM
ret = read_meta(meta, fd);

// possible errors, 2: file too short, 1: MSH-3 does not start with field separator
if (ret != 0) {
    free_hl7_meta(meta);
    meta = NULL;
    error(12, "Parser: failed to find delimiters");
}

// this will parse the file upon the firs `\r` or `\n` and
// rewind the file pointer where it was before the function was called
line_delimiter_t d = find_line_delimiter(fd);
meta->crlf = 0;
meta->sep_message = d;

// make sure ti set meta->crlf in case of double line delimiter
if (d == DELIM_CRLF) {
    meta->crlf = 1;
    meta->sep_message = '\r';
} else if (d == DELIM_CR)
    meta->sep_message = '\r';
else 
    meta->sep_message = '\n';

// now you may manually chane ny delimiter where you think autodetect 
// screwed up. you may also just stop parsing in case you were just interested 
// in a delimiter analysis.
fprintf(stdout, "%s", hl7_meta_string(meta));

See 7pdf.c for an example.

Data structures