Lead Data Engineer for Federal Data Pipelines & Lakehouse

SHR CONSULTING GROUP, LLC; Paris (VA); On-site; USD 150,000 - 165,000; Full time

Benefits offered by this job

Competitive compensation

Certification reimbursement

401(k) with company contribution

Paid time off

Job summary

SHR Consulting Group, LLC is seeking a senior Lead Data Engineer to head data pipeline engineering, operations, and migration for a FEMA enterprise effort. You’ll design, implement, and maintain scalable batch and streaming data pipelines and mentor other engineers.

The role requires hands-on expertise with cloud-based data platforms, metadata-driven ingestion, and robust data-quality practices, in a high-security federal environment. DOE-level clearance preferred.

Qualifications

• Extensive hands-on experience designing and developing enterprise data pipelines.
• Experience with Databricks or comparable cloud-based Lakehouse platforms.
• Strong knowledge of metadata-driven data ingestion frameworks.
• Experience with CI/CD, automated testing, and deployment automation.

Responsibilities

• Lead technical delivery for FEMA pipeline engineering, operations, performance, and migration.
• Design, construct, test, deploy, and maintain scalable batch and streaming data pipelines.
• Build and maintain metadata-driven ingestion pipelines.
• Implement robust schema evolution and data-processing patterns.
• Diagnose and resolve production pipeline failures and complex data-storage issues.
• Design pipelines consistent with FEMA's multi-zone/Medallion architecture.
• Develop modular, parameterized, and reusable pipeline patterns.
• Implement scalable ingestion and transformation solutions for increasing data volume.

Skills

Databricks

SQL

Python

Ingestion frameworks

CI/CD

Data quality

Streaming pipelines

Tools

CI/CD pipelines

Job description

SHR Consulting Group, LLC is seeking a senior Lead Data Engineer to head data pipeline engineering, operations, and migration for a FEMA enterprise effort. You’ll design, implement, and maintain scalable batch and streaming data pipelines and mentor other engineers.

The role requires hands-on expertise with cloud-based data platforms, metadata-driven ingestion, and robust data-quality practices, in a high-security federal environment. DOE-level clearance preferred.