ETL Tool
We have a number of pre-built tools and accelerators to make the process fo connecting to source systems of record, transforming to FHIR standard, and loading to our FHIR server

To orchestrate and track data ingested and transformed into FHIR we deploy and maintain an Apache NiFi cluster. NiFi helps source data, route transformations through processors, maintain provenance, and track and log errors.
NiFi is our ETL powerhouse. The flexibility, scalability, and usability offered by NiFi make it an integral part of our solution. In many ways NiFi is a hybrid information controller and event processor. An event is usually triggered when your file lands on an SFTP server or an application making an API request to 1up’s server. The event then triggers the ETL process interacting with NiFi’s three major components: FlowFiles, Flowfile processors, and Connections.
Flowfiles typically start with a default set of attributes that are then added to additional operations. Attributes can be referenced via the NiFi expression language. NiFi processors do all the actual work in NiFi. They’re self-contained segments of code that in most cases have inputs and outputs. One of the most common processors, GetSFTP, retrieves files from an SFTP server and creates a flowfile. The flowfile includes attributes about the directory it was retrieved from — such as, creation date, filename, and a payload containing the file’s contents.
NiFi details how flowfiles should travel between processors. Common connections are for success and failure, which are simple error handling for processors. Flowfiles that are processed without fault are sent to the success queue while those with problems are sent to a failure queue.
NiFi is a critical part of our ETL process and makes our data processing seamless.

  • Membership eligibility feed (active, current enrollees and dependents)
  • Patient EMPI
  • Patient picker/ JWT
  • Back-end Data Feeds for Claims Data (claims data warehouse)
  • Back-end Data Feeds for Clinical Data (HIE, labs, care management, etc.)
  • Formulary data
  • Provider Directory Data

We have a number of processors which can map and transform data from its original format in to the canonical FHIR format. We have built our ETL processes by extending Apache NiFi. We can convert the following types of data:
  • CSV
  • Proprietary Data models
  • HL7 (v2, v3 CCDA, FHIR)
  • EDI (835, 837, etc.)

  • Enhanced and optimized our parser on 100,000s of CCDAs
  • Parsed 300K CCDAs medical records into 7M FHIR resources
  • Normalized & aggregated data from 100K CCDAs with data from FHIR APIs into our FHIR Bulk Data store

  • Experience in converting flat file extracts from Claims Data warehouses
  • Worked with Facets, Cognizant, proprietary Oracle and SQL Server databases
  • Map to CARIN Blue Button IG (i.e. Common Payer Consumer Data Set (CPCDS)) FHIR resources - ExplanationOfBenefit, Coverage, Patient, etc.
Last modified 7mo ago
Copy link
On this page
Integration Options
Apache NiFi
Common Payer Data Integrations
Data Conversion
CCDA > FHIR Parser
Claims Data > FHIR