Azure Databricks Data Engineer - ELT Modernization

Valiant Technologies LLC; Wilmington (DE); On-site; USD 140,000 - 190,000; Full time

Job summary

Valiant Technologies LLC seeks an experienced data engineering leader to drive the migration of complex data pipelines from Informatica PowerCenter to Azure Databricks. You will define the target architecture, craft phased execution plans, and establish success metrics to ensure data quality and performance.

You will guide solution design, engineering standards, and testing strategies, while mentoring teams in Databricks patterns and modern ELT approaches to achieve scalable and observable data

Qualifications

• Experience leading end-to-end data migration projects from legacy ETL to modern platforms.
• Strong knowledge of Azure Databricks best practices, Delta Lake, and data quality.
• Experience assessing, designing, and executing migration strategies with phased delivery.

Responsibilities

• Lead end-to-end migration of data pipelines, workflows, and integration patterns from Informatica to Azure Databricks.
• Define target state architecture and migration plans with phased execution and milestones.
• Provide technical leadership across design, standards, testing, deployment, and production readiness.
• Collaborate with business, architecture, security, and operations to align outcomes with enterprise priorities.
• Establish migration success criteria for data quality, performance, resilience, and observability.
• Mentor engineers on Databricks design patterns and modern data engineering practices.

Skills

Informatica

Azure Databricks

PySpark

Data migration

ETL/ELT

Data pipelines

Architecture

Tools

Informatica PowerCenter

Azure Databricks

Delta Live Tables

Unity Catalog

ADLS Gen2

Oracle

Job description

Valiant Technologies LLC seeks an experienced data engineering leader to drive the migration of complex data pipelines from Informatica PowerCenter to Azure Databricks. You will define the target architecture, craft phased execution plans, and establish success metrics to ensure data quality and performance.

You will guide solution design, engineering standards, and testing strategies, while mentoring teams in Databricks patterns and modern ELT approaches to achieve scalable and observable data