Azure Data Engineer

Signature IT World Inc

New Jersey

Hybrid

USD 110,000 - 150,000

Full time

7 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Signature IT World Inc. seeks an experienced Data Engineer to design and maintain scalable data pipelines on the Databricks platform.

The role focuses on processing large volumes of structured and unstructured data, implementing robust ETL/ELT workflows, and ensuring data quality and governance across cloud environments. The ideal candidate will have hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake, and will collaborate with analysts and business teams to deliver reliable

Qualifications

  • Experience designing and maintaining scalable data pipelines using Databricks.
  • Strong SQL, PySpark and Spark SQL proficiency.
  • Ability to build batch and streaming data processing solutions.

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
  • Develop complex data transformations using PySpark and Spark SQL.
  • Build batch and streaming data processing solutions.
  • Implement ETL/ELT workflows to ingest data from multiple sources.
  • Work with Delta Lake, Delta Tables, and Lakehouse architecture.
  • Optimize Spark jobs for performance, scalability, and cost efficiency.
  • Implement partitioning, caching, broadcasting, and other Spark optimization techniques.
  • Develop and maintain data workflows using Databricks Workflows and Jobs.
  • Implement incremental data loading, CDC, and SCD Type 1/Type 2.
  • Ensure data quality, data validation, and pipeline reliability.
  • Work with cloud storage and data services on Azure/AWS.
  • Implement security, access control, and governance using Unity Catalog.
  • Collaborate with data analysts, architects, and business teams.

Skills

Databricks
Apache Spark
PySpark
SQL

Tools

Delta Lake
Delta Tables
Unity Catalog
Databricks Workflows

Job description

Location: Princeton NJ-Hybrid

Contract Duration: 12months

We are looking for an experienced Data Engineer with strong expertise in Databricks, Apache Spark, PySpark, and SQL to design, develop, and maintain scalable data engineering solutions. The candidate will be responsible for building robust ETL/ELT pipelines, processing large volumes of structured and unstructured data, and implementing efficient data transformation workflows using the Databricks platform.

Key Responsibilities
  • Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
  • Develop complex data transformations using PySpark and Spark SQL.
  • Build batch and streaming data processing solutions.
  • Implement ETL/ELT workflows to ingest data from multiple sources.
  • Work with Delta Lake, Delta Tables, and Lakehouse architecture.
  • Optimize Spark jobs for performance, scalability, and cost efficiency.
  • Implement partitioning, caching, broadcasting, and other Spark optimization techniques.
  • Develop and maintain data workflows using Databricks Workflows and Jobs.
  • Implement incremental data loading, CDC, and SCD Type 1/Type 2.
  • Ensure data quality, data validation, and pipeline reliability.
  • Work with cloud storage and data services on Azure/AWS.
  • Implement security, access control, and governance using Unity Catalog.
  • Collaborate with data analysts, architects, and business teams.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Engineer
Azure Data Engineer

Signature IT World Inc • Pennsylvania

Hybrid
USD 120,000 - 150,000
Azure Databricks Engineer
Azure Databricks Engineer

IQuest Solutions Corporation • Newtown Square

On-site
USD 120,000 - 170,000
Azure Data Engineer
Azure Data Engineer

TechDigital Group • Pleasanton (CA)

On-site
USD 120,000 - 170,000
Azure Data Engineer: Databricks & Spark Expert (Hybrid)
Azure Data Engineer: Databricks & Spark Expert (Hybrid)

Signature IT World Inc • New Jersey

Hybrid
USD 110,000 - 150,000
Azure Data Engineer: Spark, Delta Lake & Pipelines
Azure Data Engineer: Spark, Delta Lake & Pipelines

TechDigital Group • Pleasanton (CA)

On-site
USD 120,000 - 170,000
Data Engineer
Data Engineer

TechDigital Group • Fort Worth (TX)

On-site
USD 100,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Veriipro • Atlanta (GA)

On-site
USD 140,000 - 180,000
Senior Databricks Data Engineer
Senior Databricks Data Engineer

Virtue America • New York (NY)

On-site
USD 140,000 - 190,000
Azure Databricks Engineer
Azure Databricks Engineer

ZEUS SOLUTIONS INC • Houston (TX)

On-site
USD 140,000 - 180,000
Sr Data Engineer
Sr Data Engineer

Golden Technology • Cincinnati (OH)

On-site
USD 100,000 - 130,000