Remote Senior Data Platform Lead Databricks & Lakehouse

Astrana Health, Inc.; Monterey Park (CA); Teletrabalho; USD 170 000 - 240 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Remote in the U.S.

Competitive compensation

Resumo da oferta

Astrana Health, Inc. is seeking a Lead - Data Engineer to guide the design, development, and operation of our enterprise data platform, including the Constellation Lakehouse and Lakebase-backed applications.

You will build scalable pipelines, integrate healthcare data sources, and develop unified data models that support analytics, AI, and operational applications. You will lead hands-on implementation, establish engineering standards, and mentor other engineers while collaborating with

Qualificações

• Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field.
• 5+ years of data engineering experience building and supporting enterprise data platforms and pipelines.
• 3+ years of hands-on experience with Databricks in production environments.
• Strong proficiency in Python, PySpark, SQL, and Apache Spark.
• Experience with Delta Lake and modern lakehouse architectures.
• Hands-on experience with Azure data services and cloud-based data platforms.
• Strong knowledge of data modeling, relational databases, and ETL/ELT best practices.
• Experience with Git, CI/CD pipelines, and automated deployment practices.

Responsabilidades

• Lead the design, development, and delivery of core platform components, including the Constellation Lakehouse in Databricks, Connect API (GraphQL) layer, and Lakebase-backed applications.
• Build and maintain scalable batch and incremental data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake.
• Integrate data from transactional databases, APIs, files, and external partners into governed datasets that support analytics and operational applications.
• Design and maintain unified healthcare data models spanning membership, providers, claims, encounters, clinical data, and other business domains.
• Develop reliable ingestion and transformation processes, including change data capture, schema evolution, deduplication, historical tracking, and reconciliation.
• Build and optimize data access layers and APIs in partnership with software engineering teams, supporting consistent business definitions and reliable downstream consumption.
• Develop and support Lakebase-backed applications and PostgreSQL data structures, including integration with lakehouse datasets and application services.
• Maintain and optimize Databricks workspaces, compute resources, job orchestration, and environment configurations in collaboration with infrastructure and security teams.
• Implement data governance and access controls through Unity Catalog and established organizational standards, including permissions, lineage, and auditability.
• Establish automated data quality checks, source-to-target reconciliation, freshness monitoring, and alerting to ensure datasets are accurate, complete, and usable.
• Apply DevOps practices, including Git, automated testing, CI/CD, infrastructure as code, and repeatable deployment across development, testing, and production environments.
• Tune pipelines, Spark workloads, SQL queries, and compute configurations to improve performance, scalability, and cost efficiency.
• Own the investigation and resolution of platform incidents and pipeline failures; perform root-cause analysis and implement preventive improvements.
• Partner with infrastructure teams on backup, recovery, and business continuity procedures, and maintain operational documentation and runbooks.
• Conduct code and design reviews, establish engineering standards, and mentor engineers across internal and vendor teams.

Conhecimentos

Python

PySpark

SQL

Apache Spark

Formação académica

Bachelor’s degree in CS/Engineering/Data Science

Ferramentas

Databricks

Delta Lake

Azure

GraphQL

PostgreSQL

Terraform

Descrição da oferta de emprego

Astrana Health, Inc. is seeking a Lead - Data Engineer to guide the design, development, and operation of our enterprise data platform, including the Constellation Lakehouse and Lakebase-backed applications.

You will build scalable pipelines, integrate healthcare data sources, and develop unified data models that support analytics, AI, and operational applications. You will lead hands-on implementation, establish engineering standards, and mentor other engineers while collaborating with