Senior Data Engineer: Scale Pipelines & Lakehouse Leadership

Survey Sampling International Hyderabad Private Ltd. (India); Westport (CT); On-site; USD 130,000 - 150,000; Full time

Benefits offered by this job

Medical benefits

Discretionary incentive program

Job summary

Dynata is seeking a Senior Data Engineer to design, build, and maintain data pipelines powering the enterprise lakehouse. You will anchor the data engineering team, delivering robust ELT/ETL solutions and mentoring junior engineers.

You will design scalable batch and streaming pipelines using Spark, Kafka, and Flink, implement lakehouse architecture with Delta Lake or Iceberg, and optimize complex PySpark transformations on large datasets.

Qualifications

• 6+ years of data engineering experience with enterprise-scale delivery.
• Expert proficiency in Python and SQL; PySpark experience required.
• Hands-on experience with Spark, Delta Lake, or Iceberg.
• Experience with orchestration tools: Airflow, Prefect, or Dagster.
• Strong knowledge of cloud data services: AWS Glue, Azure Data Factory, GCP Dataflow.
• Proficiency with Git, CI/CD pipelines, and containerization (Docker/Kubernetes).
• Experience with dbt for transformation layer management.
• Bachelor's degree in Computer Science, Engineering, or related field.

Responsibilities

• Design, build, and maintain scalable data pipelines and data models.
• Implement Bronze/Silver/Gold medallion lakehouse architecture.
• Develop and optimize SQL and PySpark transformations for large datasets.
• Integrate structured, semi-structured, and unstructured data sources.
• Mentor junior engineers and participate in code reviews.
• Write Infrastructure-as-Code for pipelines (Terraform, Helm).
• Ensure data quality through tests and monitoring.

Skills

Python

SQL

Mentoring

Data engineering

Education

Bachelor's degree in CS/Engineering or related field

Tools

Apache Spark

Delta Lake

Apache Iceberg

Airflow

Prefect

Dagster

Git

CI/CD

Docker

Kubernetes

dbt

Job description

Dynata is seeking a Senior Data Engineer to design, build, and maintain data pipelines powering the enterprise lakehouse. You will anchor the data engineering team, delivering robust ELT/ETL solutions and mentoring junior engineers.

You will design scalable batch and streaming pipelines using Spark, Kafka, and Flink, implement lakehouse architecture with Delta Lake or Iceberg, and optimize complex PySpark transformations on large datasets.