Senior Data Platform Engineer - Hybrid, Fabric, Delta Lake

Investigative Case Management powered by Tapestrii; Dallas (TX); Hybrid; USD 150,000 - 190,000; Full time

Benefits offered by this job

Health insurance

Health savings account

Dental insurance

Vision insurance

401(k) matching

Life insurance

Paid time off

Parental leave

Disability insurance

Childcare assistance

Education reimbursement

Fitness membership

Volunteer time off

Job summary

Aperia Solutions, a Dallas-based payments SaaS company, is modernizing its risk analysis platform using Microsoft Fabric. The role owns end-to-end medallion data architecture, crafting Bronze/Silver/Gold tables and Direct Lake models, and advancing a high-utilization engineering effort with a small, senior team.

The candidate will design scalable data pipelines, optimize storage layouts, and collaborate with DevOps on IaC, while ensuring robust performance for analytics surfaces across Fabric

Qualifications

• 5+ years data engineering experience.
• 2+ years hands-on Spark development (PySpark preferred).
• Deep Delta Lake internals knowledge (logs, retention, VACUUM).
• Parquet format expertise (footers, statistics, encoding).
• Microsoft Fabric hands-on experience (workspaces, lakehouses, semantic models).
• SQL performance tuning for Fabric and PostgreSQL workloads.
• Strong Python data engineering skills (pandas, PyArrow, delta-rs).
• Ability to communicate architecture clearly in writing and work autonomously on well-scoped features.

Responsibilities

• Design Bronze, Silver, and Gold Delta table schemas with optimized layout and retention settings.
• Build Fabric Spark notebooks for feature pipelines across multiple windows.
• Author per-flavor Silver adapters and test against golden fixtures.
• Create the canonical MERGE DAG (Bronze → canonical Silver) ensuring idempotence.
• Develop Gold materialization pipelines for fact tables with proper score-source stamping.
• Design and implement Direct Lake semantic models; performance-test against data volumes.
• Configure OneLake namespace layouts and cross-workspace patterns.
• Establish idempotent Delta write patterns using delta-rs from Python containers.
• Support Fabric IaC with Terraform capacity provisioning and workspace bootstrap.
• Consider ClickHouse contingency for latency-sensitive client UIs.

Skills

Data engineering

Spark development

Python

Tools

PySpark

Delta Lake

Microsoft Fabric

SQL (PostgreSQL)

Parquet

Delta-rs

Job description

Aperia Solutions, a Dallas-based payments SaaS company, is modernizing its risk analysis platform using Microsoft Fabric. The role owns end-to-end medallion data architecture, crafting Bronze/Silver/Gold tables and Direct Lake models, and advancing a high-utilization engineering effort with a small, senior team.

The candidate will design scalable data pipelines, optimize storage layouts, and collaborate with DevOps on IaC, while ensuring robust performance for analytics surfaces across Fabric