Senior Data Engineer - AWS DMS & Databricks Lead

Quantum Technologies. LLC

Wilmington (DE)

On-site

USD 140,000 - 210,000

Full time

12 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Quantum Technologies. LLC is seeking a hands-on Senior Data Engineer to build and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. The role owns pipelines from source-database replication through transformation to analytics-ready data, working directly with customers.

Immediate AWS delivery is required with no ramp-up time from Azure or other clouds. The position requires deep AWS DMS and Databricks experience, strong SQL, and a proven ability to manage

Qualifications

  • Hands-on AWS DMS setup, runs and troubleshooting in production.
  • Databricks on AWS with notebooks, jobs, Delta Lake, MERGE patterns.
  • Write production-grade PySpark and Python code.
  • Build end-to-end pipelines from source to curated data.
  • Strong SQL with joins, window functions, data reconciliation.
  • Clear communication; ability to lead customer conversations.
  • 6+ years in data engineering with at least 3 on AWS data services and Databricks.

Responsibilities

  • Design, build, and maintain end-to-end data pipelines, from source ingestion through transformation to curated output tables.
  • Configure and run AWS DMS replication tasks for full loads and CDC from Oracle, SQL Server, MariaDB.
  • Land DMS output in S3 and process in Databricks with PySpark, Delta Lake, and medallion design.
  • Write complex SQL for transformations, validation, reconciliation, and troubleshooting.
  • Handle CDC logic: upserts, merges, deletes, late-arriving data, schema changes, and deduplication.
  • Tune Spark jobs and DMS tasks for performance, cost, and reliability; diagnose lag and failures.
  • Build data quality checks, logging, alerting, and restart/recovery for hands-off pipelines.
  • Orchestrate workloads with Databricks Workflows and keep code in source control with peer review.
  • Meet with customer stakeholders to gather requirements, explain choices, and provide status updates.
  • Write clear technical documentation for pipelines, data flows, and runbooks.

Skills

AWS DMS
Databricks on AWS
PySpark
Python
SQL
End-to-end pipelines
Communication
Experience: 6+ years

Job description

Quantum Technologies. LLC is seeking a hands-on Senior Data Engineer to build and run end-to-end data pipelines on AWS DMS and Databricks using PySpark, Python, and SQL. The role owns pipelines from source-database replication through transformation to analytics-ready data, working directly with customers.

Immediate AWS delivery is required with no ramp-up time from Azure or other clouds. The position requires deep AWS DMS and Databricks experience, strong SQL, and a proven ability to manage

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer: AWS DMS + Databricks
Senior Data Engineer: AWS DMS + Databricks

Robotics Technologies LLC • Wilmington (DE)

On-site
USD 120,000 - 180,000
Senior AWS Data Engineer – DMS & Databricks Expert
Senior AWS Data Engineer – DMS & Databricks Expert

Ethereum Technologies • Wilmington (DE)

On-site
USD 140,000 - 180,000
Senior Databricks & AWS Data Engineer
Senior Databricks & AWS Data Engineer

Quantum Technologies LLC • Sunnyvale (CA)

On-site
USD 76,000 - 108,000
Senior Databricks Engineer – Data Platform Mastery
Senior Databricks Engineer – Data Platform Mastery

Quantum Technologies LLC • Pennsylvania

On-site
USD 96,000 - 131,000
Data Engineer — Databricks & ADF Pipelines
Data Engineer — Databricks & ADF Pipelines

Quantum Technologies. LLC • Atlanta (GA)

On-site
USD 90,000 - 150,000
Senior Databricks Data Engineer: Scale Big Data Pipelines
Senior Databricks Data Engineer: Scale Big Data Pipelines

Quantum Technologies. LLC • Dallas (TX)

On-site
USD 150,000 - 190,000
Senior Data Engineer - AWS DMS & Databricks
Senior Data Engineer - AWS DMS & Databricks

Robotic Technologies Automation • Wilmington (DE)

On-site
USD 140,000 - 200,000
Senior AWS Data Engineer - DMS + Databricks Pipelines
Senior AWS Data Engineer - DMS + Databricks Pipelines

Valiant Technologies LLC • Wilmington (DE)

On-site
USD 140,000 - 200,000
Senior Data Engineer — Databricks & AWS Architect
Senior Data Engineer — Databricks & AWS Architect

Quantum Technologies. LLC • Tampa (FL)

On-site
USD 89,000 - 126,000
Data Engineer
Data Engineer

Robotic Technologies Automation • Wilmington (DE)

On-site
USD 140,000 - 200,000