Data Engineer: Databricks & ADF ETL in Dallas

Quantum Technologies. LLC; Dallas (TX); On-site; USD 89,000 - 126,000; Full time

Job summary

Quantum Technologies. LLC in Dallas, TX seeks a Data Engineer to design, build and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will develop ETL/ELT processes, transform data, and ensure data quality across sources and targets.

Proficiency with SQL and Python/PySpark for large-scale processing is required, along with familiarity with Azure cloud services and data lake concepts. This role is an on-site data engineering position in Dallas.

Qualifications

• Experience with Databricks and Azure Data Factory (ADF).
• Strong ETL/ELT concepts.
• SQL and data transformations.
• Python or PySpark for large-scale data processing.
• Azure cloud services and data lake concepts.

Responsibilities

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with Python or PySpark for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Skills

ETL/ELT concepts

SQL

Python/PySpark

Tools

Databricks

Azure Data Factory (ADF)

Job description

Quantum Technologies. LLC in Dallas, TX seeks a Data Engineer to design, build and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will develop ETL/ELT processes, transform data, and ensure data quality across sources and targets.

Proficiency with SQL and Python/PySpark for large-scale processing is required, along with familiarity with Azure cloud services and data lake concepts. This role is an on-site data engineering position in Dallas.