Data Engineer – Databricks & Azure Data Factory

CloudBigD

Plano (TX)

On-site

USD 98,000 - 139,000

Full time

4 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

CloudBigD is seeking a Data Engineer in Plano, TX to design and optimize data pipelines using Databricks and Azure Data Factory. You will develop ETL/ELT processes, write efficient SQL, and leverage PySpark for large-scale data tasks.

The role involves integrating data from various sources into data lakes or warehouses, monitoring pipelines, and ensuring data quality across teams in a fast-paced environment.

Qualifications

  • Experience with Databricks and Azure Data Factory (ADF).
  • Solid ETL/ELT concepts and data transformations.
  • Proficient in SQL and data transformation techniques.
  • Basic Python or PySpark knowledge for large-scale processing.
  • Understanding of Azure cloud services and data lake concepts.

Responsibilities

  • Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
  • Work with Databricks for data processing and transformation.
  • Write and optimize SQL queries for data extraction and transformation.
  • Use PySpark or Python for large-scale data processing.
  • Integrate data from multiple sources into data warehouses or data lakes.
  • Monitor, troubleshoot, and optimize data pipelines.
  • Collaborate with teams to ensure data quality and reliability.

Skills

Databricks
Azure Data Factory (ADF)
ETL/ELT concepts
SQL
Python or PySpark
Azure cloud services & data lakes

Job description

CloudBigD is seeking a Data Engineer in Plano, TX to design and optimize data pipelines using Databricks and Azure Data Factory. You will develop ETL/ELT processes, write efficient SQL, and leverage PySpark for large-scale data tasks.

The role involves integrating data from various sources into data lakes or warehouses, monitoring pipelines, and ensuring data quality across teams in a fast-paced environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer - Databricks + ADF Data Pipelines
Data Engineer - Databricks + ADF Data Pipelines

Ethereum Technologies LLC • Plano (TX)

On-site
USD 118,000 - 136,000
Data Engineer: Databricks & Azure Data Factory Expert
Data Engineer: Databricks & Azure Data Factory Expert

Ethereum Technologies LLC • Plano (TX)

On-site
USD 98,000 - 139,000
Data Engineer — Databricks & Azure ETL Specialist
Data Engineer — Databricks & Azure ETL Specialist

Robotics Technologies LLC • Plano (TX)

On-site
USD 96,000 - 152,000
Data Engineer: Azure Databricks & ADF Pipelines
Data Engineer: Azure Databricks & ADF Pipelines

Ethereum Technologies LLC • Plano (TX)

On-site
USD 101,000 - 136,000
Data Engineer - Spark, Databricks & Azure ETL Expert
Data Engineer - Spark, Databricks & Azure ETL Expert

Quantum Technologies LLC • Plano (TX)

On-site
USD 98,000 - 139,000
Data Engineer - Databricks & Azure Data Factory
Data Engineer - Databricks & Azure Data Factory

Quantum Technologies LLC • Plano (TX)

On-site
USD 98,000 - 139,000
Data Engineer: Azure Databricks & ADF Pipelines (Plano)
Data Engineer: Azure Databricks & ADF Pipelines (Plano)

Robotic Process Automation LLC • Plano (TX)

On-site
USD 101,000 - 136,000
Databricks Data Engineer - Azure, CDC Pipelines, PySpark
Databricks Data Engineer - Azure, CDC Pipelines, PySpark

CloudBigD • Phoenix (AZ)

On-site
USD 81,000 - 115,000
Data Engineer: Databricks & ADF ETL Pipelines (Azure)
Data Engineer: Databricks & ADF ETL Pipelines (Azure)

Ethereum Technologies LLC • Dallas (TX)

On-site
USD 89,000 - 126,000
Data Engineer: Databricks & Azure Data Factory ETL Pro
Data Engineer: Databricks & Azure Data Factory ETL Pro

Quantum Technologies LLC • Richardson (TX)

On-site
USD 72,000 - 118,000