Data Engineer - Spark, Databricks & Azure ETL Expert

Quantum Technologies. LLC; Plano (TX); On-site; USD 98,000 - 139,000; Full time

Job summary

Quantum Technologies. LLC is seeking a Data Engineer in Plano, TX to design and optimize data pipelines using Databricks and Azure Data Factory. The role involves building ETL/ELT processes, writing SQL queries, and leveraging PySpark for large-scale data processing.

You will integrate data from multiple sources into data warehouses or data lakes, monitor pipelines, and collaborate to ensure data quality and reliability. Equal opportunity employer.

Qualifications

• Experience with Databricks and Azure Data Factory (ADF).
• Strong understanding of ETL/ELT concepts.
• Good knowledge of SQL and data transformations.
• Basic experience with Python or PySpark.
• Understanding of Azure data lake concepts and data warehouses.

Responsibilities

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with PySpark or Python for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Skills

SQL

Python

ETL/ELT concepts

Tools

Databricks

Azure Data Factory (ADF)

Azure cloud services

Job description

Quantum Technologies. LLC is seeking a Data Engineer in Plano, TX to design and optimize data pipelines using Databricks and Azure Data Factory. The role involves building ETL/ELT processes, writing SQL queries, and leveraging PySpark for large-scale data processing.

You will integrate data from multiple sources into data warehouses or data lakes, monitor pipelines, and collaborate to ensure data quality and reliability. Equal opportunity employer.