Senior Data Engineer – Databricks & ADF ETL

Quantum Technologies. LLC

Sunnyvale (CA)

On-site

USD 96,000 - 124,000

Full time

5 hours ago

Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Quantum Technologies. LLC in Sunnyvale, CA is seeking a Lead Data Engineer with expertise in Databricks and Azure Data Factory (ADF) to design, build, and optimize data pipelines and ETL processes.

You will implement scalable data workflows, write efficient SQL and Python/PySpark code, and integrate diverse data sources into data warehouses or data lakes while ensuring data quality and reliability. This 10-month contract role offers an hourly rate of $78 for C2C, with W-2 or 1099 options, and

Qualifications

Experience delivering ETL/ELT pipelines with modern data platforms.
Strong SQL skills and data transformation capabilities.
Proficiency in Python or PySpark for large-scale processing.
Familiarity with Azure cloud services and data lake concepts.
Experience with Databricks and Azure Data Factory (ADF).

Responsibilities

Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
Work with Databricks for data processing and transformation.
Write and optimize SQL queries for data extraction and transformation.
Use PySpark or Python for large-scale data processing tasks.
Integrate data from multiple sources into data warehouses or data lakes.
Monitor, troubleshoot, and optimize data pipelines.
Collaborate with teams to ensure data quality and reliability.

Skills

ETL/ELT concepts

SQL

Python/PySpark

Azure cloud services

Data lake concepts

Tools

Databricks

Azure Data Factory (ADF)

Job description

Quantum Technologies. LLC in Sunnyvale, CA is seeking a Lead Data Engineer with expertise in Databricks and Azure Data Factory (ADF) to design, build, and optimize data pipelines and ETL processes.

Get your free, confidential resume review.

or drag and drop your file here.