Senior Data Engineer - Azure Databricks | Hybrid/Remote

Nationwide IT Services, Inc.; Andrews (MD); Hybrid; USD 125,000 - 178,000; Full time

Job summary

Nationwide IT Services is seeking a Senior Data Engineer / Databricks Engineer to design, build, test, deploy, optimize, and sustain enterprise data pipelines supporting the ANG CDAO data ecosystem. The role will focus on Azure Databricks, SQL, Python, Git-based CI/CD, and integrated DoD/DAF/ANG source systems to provide reliable ingestion and data quality.

The position is Hybrid located at Joint Base Andrews, MD with up to 3 telework days per week; occasional travel may be required.

Qualifications

• Production-grade ETL/ELT pipelines with Azure Databricks or Spark-based platforms.
• Advanced SQL and Python for data transformation, validation, automation, and performance.
• Experience with batch/streaming ingestion, APIs, schema management, orchestration, monitoring, data quality, and failure recovery.
• Git-based version control and CI/CD for data-engineering solutions.
• Knowledge of data warehousing/lakehouse patterns, indexing/query optimization, access control, encryption, and secure handling of sensitive/CUI data.

Responsibilities

• Design, build, test, deploy, and maintain ETL/ELT pipelines for multiple source systems.
• Implement ingestion patterns for structured/unstructured data; support near-real-time currency requirements.
• Develop SQL and Python transformation logic to cleanse, standardize, deduplicate, validate, and enrich data per standards.
• Build and operate Azure Databricks jobs, orchestration sequences, triggers, and schedules; optimize performance and cost.
• Implement automated data-quality checks, reconciliation, invalid-record quarantine, monitoring, alerts, and recoverable error handling.
• Triage pipeline failures, perform root-cause analysis, and restore service within required windows.
• Maintain pipeline code, configuration, infrastructure/deployment scripts, and test assets in Git; run automated CI/CD builds and security tests.
• Create and maintain data lineage, source-to-target mappings, architecture docs, runbooks, and quarterly updates.
• Support data warehouse/lake administration, query optimization, Power BI dataflows, and secure cross-platform data movement.

Skills

Azure Databricks

SQL

Python

Git-based CI/CD

Data warehousing

Data quality

Security/compliance (DoD/DAF)

Job description

Nationwide IT Services is seeking a Senior Data Engineer / Databricks Engineer to design, build, test, deploy, optimize, and sustain enterprise data pipelines supporting the ANG CDAO data ecosystem. The role will focus on Azure Databricks, SQL, Python, Git-based CI/CD, and integrated DoD/DAF/ANG source systems to provide reliable ingestion and data quality.

The position is Hybrid located at Joint Base Andrews, MD with up to 3 telework days per week; occasional travel may be required.