Project 16 · Data / ETL
DE 4
PharmaFlow
Informatica-Style ETL Framework
01 Abstract
Informatica-Style ETL Framework
A Python ETL framework that mirrors Informatica PowerCenter semantics — Mappings, Sessions, Workflows, and eleven transformation types — applied to pharmaceutical data.
02 Highlights
Methods & contributions
- PowerCenter model — Mappings / Sessions / Workflows with 11 transformations (Expression, Lookup, Router, Aggregator, and more).
- Pharmacovigilance ETL — PRR, ROR, and Yates chi-squared with MedDRA term standardization (British spelling) and FAERS age-unit handling.
- Slowly changing dimensions — SCD Type 2 PL/SQL procedures.
- Operations — Control-M jobs, UNIX automation, and Great Expectations data validation.
03 Architecture
Data flow
source → Mapping (Expression · Lookup · Router · Aggregator …)
→ Session → Workflow → SCD2 target (Control-M scheduled)
04 Stack