Senior Data Engineer: Lead Lakehouse Pipelines & ETL

Dynata; United States; On-site; USD 130,000 - 150,000; Full time

Benefits offered by this job

Medical benefits

Discretionary incentive program

Job summary

Dynata is seeking a Senior Data Engineer to design, build, and maintain data pipelines, data models, and the enterprise lakehouse. You will deliver robust ELT/ETL solutions and mentor junior engineers, shaping scalable batch and streaming workflows.

Equipped with Python/SQL expertise and PySpark, you will work with Spark, Delta Lake, and Iceberg, integrating diverse data sources, enforcing quality checks, and collaborating with data architects to evolve data models.

Qualifications

• 6+ years of data engineering experience with enterprise-scale delivery.
• Expert proficiency in Python and SQL; PySpark experience required.
• Hands-on with Spark, Delta Lake, or Iceberg.
• Experience with Airflow, Prefect, or Dagster.
• Strong knowledge of cloud data services (AWS, Azure, GCP).
• Git, CI/CD, and containerization (Docker/Kubernetes).
• dbt for transformation layer management.
• Bachelor's degree in Computer Science, Engineering, or related field.

Responsibilities

• Design and build scalable batch and streaming data pipelines using Spark, Kafka, and Flink.
• Develop lakehouse architecture and ETL/ELT solutions with Delta Lake/Iceberg.
• Create and optimize SQL and PySpark transformations for large datasets.
• Integrate diverse data sources into the lakehouse and evolve data models.
• Implement data quality checks and monitoring.
• Write infrastructure as code (Terraform, Helm).
• Participate in code reviews and enforce standards.
• Troubleshoot pipelines and mentor junior engineers.

Skills

Python

SQL

PySpark

Apache Spark

Kafka

Flink

Delta Lake

Iceberg

Airflow

Prefect

Dagster

AWS Glue

Azure Data Factory

GCP Dataflow

Git

CI/CD

Docker

Kubernetes

dbt

Databricks

Snowflake

Apache Hudi

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Apache Spark

Delta Lake

Apache Iceberg

Airflow

Prefect

Dagster

Git

CI/CD pipelines

Docker

Kubernetes

dbt

Databricks

Snowflake

Apache Kafka

AWS Glue

Azure Data Factory

GCP Dataflow

Job description

Dynata is seeking a Senior Data Engineer to design, build, and maintain data pipelines, data models, and the enterprise lakehouse. You will deliver robust ELT/ETL solutions and mentor junior engineers, shaping scalable batch and streaming workflows.

Equipped with Python/SQL expertise and PySpark, you will work with Spark, Delta Lake, and Iceberg, integrating diverse data sources, enforcing quality checks, and collaborating with data architects to evolve data models.