Sr Data Engineer
Health Catalyst, SG Remote
The role
- Data: integrate data in a flexible, open & scalable platform to power healthcare’s digital transformation
- Analytics: deliver analytic applications & services that generate insight on how to measurably improve
- Expertise: provide clinical, financial & operational experts who enable & accelerate improvement
- Engagement: attract, develop and retain world-class team members by being a best place to work
What you will do
What you'll own in this role
- Design dimensional data marts (star schemas, conformed dimensions, slowly changing dimensions) on Health Catalyst's healthcare data model, for areas like pharmacy, quality, finance and operational KPIs.
- Extend the core model by clinical domain, so each hospital or institution adds its own columns instead of maintaining a separate copy.
- Build, tune and maintain pipelines on Azure Databricks with PySpark, Spark SQL, Delta Lake and Lakeflow Jobs, through bronze, silver and gold layers, with an eye on compute cost.
- Migrate legacy reports and ETL to Databricks. Translate the SQL and PL/SQL, rebuild the logic, and reconcile outputs against the old system until the numbers match.
- Ingest new sources such as EHR extracts, ERP data and flat files. Design incremental loads that don't drop late-arriving or frequently updated records.
- Build Databricks AI/BI dashboards and help users move off pivot tables and legacy BI tools.
- Build and validate clinical registries and cohort logic using ICD-10, SNOMED CT, LOINC and local code sets.
- Implement access controls in Unity Catalog, including row-level security and attribute-based policies tied to identity groups.
- Ship through Git-based CI/CD with Declarative Automation Bundles (DABs). Write unit and regression tests, and review other engineers' code.
- Use AI coding agents to automate source mapping, code generation and testing, and check their output before it ships.
- Work directly with client engineers, analysts and clinicians. Run requirements sessions, explain data issues in plain language, and own fixes to closure.
- Mentor junior engineers and interns.
- Maintain compliance with security and privacy regulations, including HIPAA, Singapore's PDPA and client data residency requirements.
Information Security and Compliance Responsibilities
- Maintain compliance with training directives required by the organization pertaining to Information Security, Acceptable Use Policy and HIPAA Privacy and Security.
- Adhere to and comply with the organizations Acceptable Use Policy.