Sr. Databricks Data Engineer

Tetra Tech; McLean (VA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Medical insurance

Dental & vision insurance

401(k) with company match

Tuition assistance

Paid time off

Job summary

Halvik Corp delivers data modernization and digital services across U.S. Government agencies. The Senior Databricks Data Engineer will design, migrate, and operate secure data pipelines in a Databricks Lakehouse to support DOT modernization.

You will lead cross‑functional teams, ensure data quality and governance, and optimize pipelines for performance and cost in an AWS environment.

Qualifications

• Bachelor's degree in computer science or related field or equivalent experience.
• 7+ years of professional data engineering experience with Databricks/Spark.
• Advanced Python, PySpark, and SQL skills; experience with Databricks and Delta Lake.
• Experience with AWS S3, batch/file ingestion and REST APIs; Git/CI/CD.
• Must meet DOT security/suitability requirements; secure handling of PII data.

Responsibilities

• Lead design and implementation of reusable ingestion/transformation patterns for legacy and external data sources.
• Build and operate Bronze, Silver, and Gold pipelines with orchestration and monitoring.
• Collaborate with PMs, Data Architects, BI, Cloud/DevSecOps, security, and governance teams.
• Develop and optimize Python, PySpark, SQL workloads in Databricks; focus on reliability and cost.
• Implement data quality, lineage, and governance for production pipelines.

Skills

Databricks

PySpark

Python

SQL

AWS S3

REST APIs

Git CI/CD

Data orchestration

Data governance

PII security

Education

Bachelor's degree

Tools

Delta Lake

Unity Catalog

Auto Loader

Lakeflow

Job description

Halvik Corp delivers a wide range of services to 13 executive agencies and 15 independent agencies. Halvik is a highly successful WOB business with more than 50 prime contracts and 500+ professionals delivering Digital Services, Advanced Analytics, Artificial Intelligence/Machine Learning, Cyber Security and Cutting-Edge Technology across the US Government. Be a part of something special!

Position Overview

The Senior Databricks Data Engineer / Technical Lead will support the U.S. Department of Transportation (DOT) data modernization effort by designing, building, migrating, and operating secure, scalable data pipelines in a Databricks Lakehouse. This is a highly hands‑on role focused on legacy and external‑source migration, metadata‑driven ingestion, Medallion Architecture, data quality and reconciliation, production operations, and transition to the AWS/OneDOT environment.

Core Responsibilities

• Strategic Execution: Lead the design and implementation of reusable ingestion and transformation patterns for legacy and external data sources into Databricks, including support for migration to the AWS/OneDOT environment.
• Operational Oversight: Build and operate Bronze, Silver, and Gold pipelines; manage batch and incremental loads, orchestration, dependencies, retries/recovery, monitoring, alerting, and production support.
• Collaboration: Partner with the Technical Project Manager, Data Architects, business/data SMEs, BI developers, Cloud/DevSecOps, security, and governance teams to translate requirements into implementable data solutions.
• Technical Performance: Develop and optimize Python, PySpark, and SQL workloads using Databricks, Delta Lake, Unity Catalog, Auto Loader and/or Lakeflow capabilities, and Databricks Jobs/Workflows with a focus on reliability, performance, and cost.
• Data Quality & Governance: Implement source‑to‑target reconciliation, data validation, schema evolution, lineage, access controls, documentation, and reusable engineering standards for production data pipelines.

Minimum Requirements

• Education: Bachelor's degree in computer science, Data Engineering, Information Technology, Engineering, or a related field; equivalent relevant experience may be considered.
• Experience: Minimum of 7 years of professional data engineering experience, including significant hands‑on experience with Databricks/Spark and production data pipelines.
• Technical Proficiency: Advanced Python, PySpark, and SQL skills; strong experience with Databricks, Delta Lake, AWS/S3, batch/file ingestion (CSV/JSON), REST APIs, Git/CI/CD, and data pipeline orchestration.
• Compliance: Must meet applicable DOT contract security and suitability requirements and demonstrate secure handling of PII and other sensitive data, including access controls and secrets management.
• Certifications: Relevant Databricks and/or AWS data engineering certification preferred; equivalent demonstrated hands‑on experience will be considered.

Preferred Expertise

• Demonstrated success modernizing legacy data platforms (Sybase/SAP IQ or similar) into Databricks, including source‑to‑target reconciliation, cutover, and production stabilization.
• Experience with Unity Catalog, metadata/configuration‑driven ingestion, Auto Loader/Lakeflow, Databricks Jobs/Workflows, lineage/cataloging tools, and scalable multi‑source ingestion frameworks.
• Federal/DOT delivery experience; familiarity with Agile/Kanban, production O&M, cloud cost optimization, and data governance. Streaming and AI/ML data pipeline exposure is a plus.

Halvik offers a competitive full benefits package including:

• Company-supported medical, dental, vision, life, STD, and LTD insurance
• Benefits include 11 federal holidays and PTO
• Eligible employees may receive performance‑based incentives in recognition of individual and/or team achievements.
• 401(k) with company matching
• Flexible Spending Accounts for commuter, medical, and dependent care expenses
• Tuition Assistance
• Charitable Contribution matching

Halvik Corp is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status.

Halvik'spay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.