Databricks Developer

About the role

The Cloud Data Engineer (Azure Databricks) will build, test, and maintain data pipelines in the healthcare domain, turning raw data into actionable insights that improve patient outcomes.

Key responsibilities

• Design and develop ETL/ELT pipelines on Azure Databricks using PySpark, Spark SQL, and notebooks for batch and streaming workloads.
• Implement Delta Lake tables, manage schema evolution, and optimize performance with OPTIMIZE, Z‑ORDER, and VACUUM.
• Orchestrate workflows with Databricks Jobs and Azure Data Factory, supporting end‑to‑end data ingestion from databases, APIs, SFTP and file stores.
• Collaborate with clinical and analytical teams to deliver curated datasets following the Medallion architecture.

Required profile

• 2–4 years of experience building cloud data engineering solutions, preferably in healthcare.
• Proven ability to translate business or clinical requirements into robust data models and pipelines.
• Experience with Azure services such as ADLS Gen2, Azure SQL, and Synapse Analytics.

Required skills

• Azure Databricks, PySpark, Spark SQL, Databricks notebooks.
• Delta Lake concepts and optimization techniques.
• Azure Data Factory, Databricks Workflows/Jobs.
• SQL, Azure Data Lake Storage Gen2, Azure Blob Storage.
• Version control (Git) and basic CI/CD practices.