Senior Data Engineer: Spark, Delta Lake & AI Data Platform

ITAC Solutions; Birmingham (AL); On-site; USD 104,000 - 127,000; Full time

Benefits offered by this job

Next-gen data platform

Single source of truth

Azure Databricks Spark stack

Collaborative culture

Flexible work environment

Standards & governance influence

Job summary

ITAC Solutions is seeking a Sr. Data Engineer for a client in the Birmingham, AL area to build and enhance data ingestion pipelines and data models across CRM and portfolio platforms. You will enable analytics and AI initiatives by delivering reliable, governed data workflows.

You will work with Spark, Delta Lake, and Databricks on Azure, developing production-grade pipelines and datasets for reporting, analytics, and internal applications while adhering to governance and testing practices.

Qualifications

• Experience with Spark, Delta Lake, and Databricks data engineering environments.
• Advanced SQL skills and Python development experience with testing practices.
• Experience building data frameworks and production-grade pipelines.
• Familiarity with Azure data services and infrastructure-as-code deployment approaches.
• Experience with financial services or regulated data environments is a plus.

Responsibilities

• Build and enhance data ingestion frameworks for API and file-based sources.
• Integrate data from systems including CRM, portfolio management, and other platforms.
• Develop curated datasets supporting reporting, analytics, internal applications, and AI initiatives.
• Design data models for accounts, entities, households, and related structures.
• Establish testing, monitoring, quality, and governance practices.
• Create reliable production workflows and documentation for long-term scalability.

Skills

Spark

Delta Lake

Databricks

SQL

Python

Azure

IaC

Financial data or regulated data

Tools

Databricks

Azure Data Services

Job description

ITAC Solutions is seeking a Sr. Data Engineer for a client in the Birmingham, AL area to build and enhance data ingestion pipelines and data models across CRM and portfolio platforms. You will enable analytics and AI initiatives by delivering reliable, governed data workflows.

You will work with Spark, Delta Lake, and Databricks on Azure, developing production-grade pipelines and datasets for reporting, analytics, and internal applications while adhering to governance and testing practices.