Senior AWS Data Engineer - Databricks & DMS Pipelines

CloudBigD

Wilmington (DE)

Vor Ort

USD 120.000 - 180.000

Vollzeit

vor 1 Stunde
Sei unter den ersten Bewerbenden

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

CloudBigD is seeking a hands-on Senior Data Engineer to design and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. You will own pipelines from source databases through transformation to analytics-ready tables, collaborating with customers and meeting tight delivery timelines on AWS.

The role requires deep experience with DMS replication, Delta Lake design, and Spark performance tuning.

Qualifikationen

  • Hands-on experience with AWS DMS in production.
  • Databricks on AWS with Delta Lake and MERGE/upsert patterns.
  • Proficient in PySpark and Python for production data pipelines.
  • Experience building end-to-end data pipelines from source to curated data.

Aufgaben

  • Design, build, and maintain end-to-end data pipelines, from source ingestion through transformation to curated output tables.
  • Configure and run AWS DMS replication tasks for full loads and CDC.
  • Land DMS output in S3 and process with Databricks using Delta Lake.
  • Write complex SQL for transformations, data validation, reconciliation, and troubleshooting.
  • Handle CDC logic: upserts/merges, deletes, late-arriving data, schema changes, and deduplication.

Kenntnisse

PySpark
Python
SQL
AWS DMS
Databricks
Delta Lake
CDC
End-to-end pipelines
Communication

Tools

Databricks on AWS
AWS DMS
S3
IAM
Lambda
Delta Live Tables

Jobbeschreibung

CloudBigD is seeking a hands-on Senior Data Engineer to design and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. You will own pipelines from source databases through transformation to analytics-ready tables, collaborating with customers and meeting tight delivery timelines on AWS.

The role requires deep experience with DMS replication, Delta Lake design, and Spark performance tuning.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior AWS Data Engineer – DMS & Databricks Expert
Senior AWS Data Engineer – DMS & Databricks Expert

Ethereum Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 180.000
Senior Data Engineer: AWS DMS + Databricks
Senior Data Engineer: AWS DMS + Databricks

Robotics Technologies • Wilmington (DE)

Vor Ort
USD 120.000 - 180.000
Senior AWS Data Engineer - DMS + Databricks Pipelines
Senior AWS Data Engineer - DMS + Databricks Pipelines

Valiant Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 200.000
Senior Data Engineer - AWS DMS & Databricks Lead
Senior Data Engineer - AWS DMS & Databricks Lead

Quantum Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 210.000
Senior Data Engineer - AWS DMS & Databricks
Senior Data Engineer - AWS DMS & Databricks

Robotic Process Automation LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 200.000
Data Engineer
Data Engineer

Quantum Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 210.000
Data Engineer
Data Engineer

Robotics Technologies • Wilmington (DE)

Vor Ort
USD 120.000 - 180.000
Data Engineer
Data Engineer

Valiant Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 200.000
Data Engineer
Data Engineer

Robotic Process Automation LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 200.000
Data Engineer
Data Engineer

Ethereum Technologies LLC • Wilmington (DE)

Vor Ort
USD 140.000 - 180.000