Azure Data Engineer: Databricks & Spark Expert (Hybrid)

Signature IT World Inc

New Jersey

Hybrid

USD 110,000 - 150,000

Full time

8 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Signature IT World Inc. seeks an experienced Data Engineer to design and maintain scalable data pipelines on the Databricks platform.

The role focuses on processing large volumes of structured and unstructured data, implementing robust ETL/ELT workflows, and ensuring data quality and governance across cloud environments. The ideal candidate will have hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake, and will collaborate with analysts and business teams to deliver reliable

Qualifications

  • Experience designing and maintaining scalable data pipelines using Databricks.
  • Strong SQL, PySpark and Spark SQL proficiency.
  • Ability to build batch and streaming data processing solutions.

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
  • Develop complex data transformations using PySpark and Spark SQL.
  • Build batch and streaming data processing solutions.
  • Implement ETL/ELT workflows to ingest data from multiple sources.
  • Work with Delta Lake, Delta Tables, and Lakehouse architecture.
  • Optimize Spark jobs for performance, scalability, and cost efficiency.
  • Implement partitioning, caching, broadcasting, and other Spark optimization techniques.
  • Develop and maintain data workflows using Databricks Workflows and Jobs.
  • Implement incremental data loading, CDC, and SCD Type 1/Type 2.
  • Ensure data quality, data validation, and pipeline reliability.
  • Work with cloud storage and data services on Azure/AWS.
  • Implement security, access control, and governance using Unity Catalog.
  • Collaborate with data analysts, architects, and business teams.

Skills

Databricks
Apache Spark
PySpark
SQL

Tools

Delta Lake
Delta Tables
Unity Catalog
Databricks Workflows

Job description

Signature IT World Inc. seeks an experienced Data Engineer to design and maintain scalable data pipelines on the Databricks platform.

The role focuses on processing large volumes of structured and unstructured data, implementing robust ETL/ELT workflows, and ensuring data quality and governance across cloud environments. The ideal candidate will have hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake, and will collaborate with analysts and business teams to deliver reliable

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Engineer
Azure Data Engineer

Signature IT World Inc • New Jersey

Hybrid
USD 110,000 - 150,000
Azure Data Engineer
Azure Data Engineer

Signature IT World Inc • Pennsylvania

Hybrid
USD 120,000 - 150,000
Azure Databricks Data Engineer – Spark & Lakehouse
Azure Databricks Data Engineer – Spark & Lakehouse

Signature IT World Inc • Pennsylvania

Hybrid
USD 120,000 - 150,000
Senior Data Engineer
Senior Data Engineer

Veriipro • Atlanta (GA)

On-site
USD 140,000 - 180,000
Azure Data Engineer: PySpark & Databricks Expert
Azure Data Engineer: PySpark & Databricks Expert

TechDigital Group • Minneapolis (MN)

On-site
USD 90,000 - 150,000
Senior Data Engineer – Databricks
Senior Data Engineer – Databricks

Quantum Technologies LLC • Madison (WI)

On-site
USD 110,000 - 140,000
Senior Data Engineer – Databricks
Senior Data Engineer – Databricks

Valiant Technologies LLC • Madison (WI)

On-site
USD 120,000 - 150,000
Azure Databricks Engineer
Azure Databricks Engineer

IQuest Solutions Corporation • Newtown Square

On-site
USD 120,000 - 170,000
Hybrid Data Solutions Engineer - Azure & Databricks
Hybrid Data Solutions Engineer - Azure & Databricks

PowerSecure, Inc. • Raleigh (NC)

Hybrid
USD 90,000 - 130,000
Medical, dental, vision and life보험
Competitive pay
401(k) match
+4
Azure Data Engineer - Databricks, PySpark & Healthcare Data
Azure Data Engineer - Databricks, PySpark & Healthcare Data

Valiant Technologies LLC • Cupertino (CA)

On-site
USD 120,000 - 170,000