Azure Data & PySpark Engineer: Delta Lake Mastery

ITC Worldwide

United States

Remote

USD 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ITC Worldwide is seeking an experienced Data Engineer to design and manage data pipelines across Azure Synapse, Delta Lake, and PySpark environments. The role focuses on building robust ETL processes, maintaining data accuracy, and collaborating with Customer teams to deliver scalable data solutions.

You will work with the Data Architect and DevOps on CI/CD for IaC, while tackling complex data challenges and ensuring secure access in customer environments. Strong SQL skills are essential.

Qualifications

  • 5+ years in Dynamics 365 ecosystem with a strong PySpark development background.
  • Extensive SQL experience in Azure SQL, Synapse Analytics, and Delta Lake environments.
  • Experience implementing and supporting ETL, Data Lakes, and data engineering solutions.
  • Proficiency with Azure Synapse Analytics: workspace management, pipelines and data flows.
  • Hands-on PySpark for data processing and automation.
  • Ability to operate in secure customer environments using VPNs, MFA, RDP and jump hosts.
  • Familiarity with Data Engineering in Microsoft Fabric and Delta Lake concepts.

Responsibilities

  • Develop and manage data pipelines using Azure Synapse Pipelines and PySpark for transformation and orchestration.
  • Perform data integration and schema updates in Delta Lake environments to ensure accurate reporting.
  • Collaborate with Azure DevOps on CI/CD for IaC and workspace artifacts deployment.
  • Create custom data solutions as directed by the Data Architect and iteratively improve data patterns.
  • Document ticket resolutions, testing protocols, and data validation processes.
  • Triage Jira tickets, troubleshoot issues with SQL, Synapse and Delta Lake, and coordinate with FinOps/Databricks as needed.
  • Address ETL failures and data inconsistencies to maintain data integrity and performance.

Skills

Dynamics 365
PySpark
SQL
ETL development
Delta Lake
Azure Synapse
Azure Data Lake Gen2
Databricks

Tools

Azure Synapse Analytics
Delta Lake
Azure Data Lake Gen2
Databricks
Azure DevOps
Jira

Job description

ITC Worldwide is seeking an experienced Data Engineer to design and manage data pipelines across Azure Synapse, Delta Lake, and PySpark environments. The role focuses on building robust ETL processes, maintaining data accuracy, and collaborating with Customer teams to deliver scalable data solutions.

You will work with the Data Architect and DevOps on CI/CD for IaC, while tackling complex data challenges and ensuring secure access in customer environments. Strong SQL skills are essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Platform Engineer: Delta Lake & Spark Expert
Azure Data Platform Engineer: Delta Lake & Spark Expert

Techwish • New York (NY)

On-site
USD 140,000 - 190,000
Azure Databricks Data Engineer — Lakehouse & Spark Expert
Azure Databricks Data Engineer — Lakehouse & Spark Expert

Quantum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 180,000
Hybrid Data Engineer — Delta Lake, PySpark & Azure DevOps
Hybrid Data Engineer — Delta Lake, PySpark & Azure DevOps

BinaryBees Business Solutions LLC • Itasca (IL)

Hybrid
USD 100,000 - 130,000
Azure Data Engineer
Azure Data Engineer

Signature IT World Inc • Pennsylvania

Hybrid
USD 120,000 - 150,000
Azure Data Engineer
Azure Data Engineer

Signature IT World Inc • New Jersey

Hybrid
USD 110,000 - 150,000
Cloud Data Engineer II: Spark, Delta Lake & AWS
Cloud Data Engineer II: Spark, Delta Lake & AWS

International Game Technology PLC • Las Vegas (NV), Reno (NV), Chicago (IL), Austin (TX)

On-site
USD 110,000 - 140,000
Azure Data Engineer: Databricks & Spark Expert (Hybrid)
Azure Data Engineer: Databricks & Spark Expert (Hybrid)

Signature IT World Inc • New Jersey

Hybrid
USD 110,000 - 150,000
Azure Data Engineer: Spark, Delta Lake & Pipelines
Azure Data Engineer: Spark, Delta Lake & Pipelines

TechDigital Group • Pleasanton (CA)

On-site
USD 120,000 - 170,000
Cloud Data Engineer: Spark, Delta Lake & Azure Pipelines
Cloud Data Engineer: Spark, Delta Lake & Azure Pipelines

Govx • Colorado

Hybrid
USD 120,000 - 140,000
Paid Time Off
Competitive Medical Insurances
401(k) plan with discretionary match
+1
Azure Data Engineer
Azure Data Engineer

TechDigital Group • Pleasanton (CA)

On-site
USD 120,000 - 170,000