Senior Data Engineer: Delta Lake & Data Pipelines

Socket.dev; Chicago (IL); On-site; USD 140,000 - 190,000; Full time

Job summary

Fortellar is seeking a senior data engineer to own and operate production pipelines that migrate and integrate data across regulated environments. This role focuses on building scalable data pipelines, conformed models, and reliable data delivery.

You will design canonical data models, implement change data capture, and ensure auditability and reconciliation for every migrated record. Collaboration with clients and stakeholders is essential.

Qualifications

• 6+ years in data engineering with 3+ years building production pipelines.
• Deep Delta Lake expertise, including MERGE, CDC, and schema evolution.
• Proficiency in SQL and PySpark, with tested, source-controlled code.
• Hands-on experience with Azure data stack (ADF, ADLS, Spark lakehouse).
• Strong client-facing communication and documentation skills.

Responsibilities

• Build and operate CDC and snapshot ingestion from relational systems.
• Design conformed data models to a single trusted record per entity.
• Implement incremental change detection, deduplication, and idempotent replay.
• Create outbound integrations to enterprise platforms.
• Document business rules with owner tracking.
• Build reconciliation metrics and data quality controls.
• Promote code via source control and CI/CD; run unattended jobs.
• Develop reporting tables, semantic models, and analytics datasets.
• Advise clients and document findings for stakeholders.

Skills

Data engineering

Delta Lake

Change Data Capture

Azure data stack

SQL

PySpark

Data quality controls

Client communication

Tools

Azure Data Factory

Azure Data Lake Storage

Spark

Job description

Fortellar is seeking a senior data engineer to own and operate production pipelines that migrate and integrate data across regulated environments. This role focuses on building scalable data pipelines, conformed models, and reliable data delivery.

You will design canonical data models, implement change data capture, and ensure auditability and reconciliation for every migrated record. Collaboration with clients and stakeholders is essential.