Principal Data Engineer

Jobtailor

Vienna (VA)

On-site

USD 150,000 - 190,000

Full time

34 hours ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Jobtailor is seeking a senior data engineering leader to drive enterprise modernization across data pipelines, analytics, and reporting.

You will architect and implement batch, near real-time, and streaming pipelines using Spark, Python, SQL, and Azure services, while guiding teams and enforcing governance and security standards.

Qualifications

Bachelor's degree in information systems, CS, or engineering or equivalent experience.
Advanced hands-on expertise in Spark, Python, SQL, Databricks, Azure Data Factory, Microsoft Fabric, and cloud data integration.
Experience designing data pipelines, lakehouse architectures, and analytical data stores.
Experience with data governance, metadata, lineage, and monitoring.
Familiarity with Unity Catalog or Alation and BI platforms (Power BI/Tableau).

Responsibilities

Provide technical leadership and hands-on engineering for data modernization across data engineering, analytics, and reporting.
Design and optimize scalable data pipelines including batch, near real-time, and real-time.
Modernize legacy analytics workloads by migrating to cloud-native platforms.
Lead API-driven data onboarding using Gravitee, MuleSoft, Kafka and related tech.
Establish reusable ETL frameworks, automation, quality validation, and SLA management.
Mentor engineers and drive adoption of enterprise standards.
Ensure compliance with security, governance, and regulatory expectations.
Support Agile, DevSecOps, CI/CD, and production readiness practices.

Skills

Data Engineering

Leadership

Problem-Solving

Communication

Architecture

Education

Bachelor's degree in IT/CS/Engineering

Tools

Spark

Python

SQL

Databricks

Azure Data Factory

Microsoft Fabric

Unity Catalog

Alation

Gravitee

MuleSoft

Kafka

Power BI

Tableau

Azure DevOps

Job description

Provide technical leadership and hands‑on engineering expertise for Enterprise Data and Analytics Modernization initiatives across data engineering, analytics, reporting, and data consumption
Design, develop, and optimize scalable batch, near real‑time, and real‑time data pipelines using Spark, Python, SQL, Databricks, Microsoft Fabric, Azure Data Factory, and related Azure cloud services
Modernize legacy Analytical Data Store and BI workloads by migrating fragmented reporting and data assets into secure, standardized, governed, and cloud‑native analytics platforms
Lead API‑driven and event‑based data onboarding patterns using Gravitee, MuleSoft, Kafka, and related technologies
Establish reusable engineering ETL frameworks, design patterns, and automation for ingestion, transformation, validation, reconciliation, monitoring, and production support
Implement automated data quality validation, reconciliation controls, exception handling, alerting, monitoring, SLA management, and production readiness practices
Enable metadata management, lineage, cataloging, and access governance through Unity Catalog, Alation, and related governance capabilities
Partner with data architects, analysts, product owners, business stakeholders, governance teams, and platform teams to translate business needs into scalable technical solutions
Guide engineering teams through architecture reviews, implementation decisions, coding practices, design standards, performance tuning, and operational resilience improvements
Ensure compliance with engineering, information security, data governance, and regulatory expectations
Support Agile delivery, DevSecOps, CI/CD deployment, release readiness, defect resolution, and production support
Mentor senior and mid‑level engineers, promote engineering excellence, and drive adoption of enterprise standards

Requirements

Bachelor’s degree in information systems, Computer Science, Engineering, Data Engineering, or a related field, or the equivalent combination of education, training, and experience
Advanced hands‑on expertise in Spark, Python, SQL, Databricks, Azure Data Factory, Microsoft Fabric, and cloud‑native data integration, transformation, and analytics solutions
Strong experience designing, building, and supporting scalable data pipelines, lakehouse architecture, data warehouses, data marts, and analytical data stores
Expertise in automated data quality validation, data reconciliation, metadata management, lineage, monitoring, alerting, error handling, and SLA management
Experience with governance and catalog platforms such as Unity Catalog, Alation, or similar tools
Experience with BI and analytics platforms such as Power BI, Tableau, Microsoft Fabric, and enterprise reporting modernization patterns
Working knowledge of Azure DevOps, CI/CD pipelines, Agile delivery, production deployment, and operational support practices
Ability to communicate complex technical concepts clearly to business stakeholders, technology leaders, engineers, and cross‑functional delivery teams
Strong problem‑solving skills, architectural judgment, ownership mindset, and ability to lead delivery in complex, highly regulated enterprise environments
Applicants must be authorized to work in the United States without the need for current or future sponsorship
Ability to work Monday–Friday, 8:00AM–4:30PM

Core Competencies

Demonstrates advanced expertise in designing and optimizing scalable data pipelines and cloud‑native analytics solutions using Spark, Python, SQL, and Azure services. Proven ability to lead technical teams, implement data governance practices, and ensure compliance in complex enterprise environments.

Highest‑signal resume keywords