Clinical Data Ingestion Release 2.1

Back to Release Notes Summary

Data Ingestion - Clinical Release Notes 2.1

FHIR Converter

Identified 6 characters present in the raw data for Clinical upload that were getting HTML encoded as below: < = <

= > & = & ‘ = ' “ = " = –> =

Note: This issue is fixed in the CSV to FHIR JSON Conversion part of the main ADF pipeline. Only double quotes cannot be represented as such as the required output FHIR JSON uses double quotes to represent string values. Hence double quotes, if present, is being converted to single quotes

Audit and Log

Audit and Log incorporated as part of Main ADF from preprocessing, this would automatically generate the report Date parameters for the report. Staged Copy - To handle the environments which deny public database access to ADF (connects thru only Private Endpoints).

Archival

Added source file Data Lake archiving to the following location: ingestion > YYYY > mm > DD Added folder clearing functionality for Clinical preprocess folder: ingestion > preprocess > USCore > Payor Name > CSV Added folder clearing functionality for Clinical landing folder: ingestion > landing > Payor Name > Clinical

Back to Release Notes Summary