Hybrid Lead Data Engineer — Big Data Pipelines & Observability

Nyla Technology Solutions; Arlington (VA); Hybrid; USD 160,000 - 230,000; Full time

Benefits offered by this job

4 weeks annual leave

11 holidays

Annual leave accrual options

Nyla FLEX program

100% covered health care (medical, den

Dental and vision

10% 401k matching

$5,000 professional development

Student loan repayment

8 hours volunteering annually

Job summary

Nyla Technology Solutions seeks a visionary Lead Data Engineer to guide the design, building, and optimization of high-throughput big data solutions. You will lead automated data ingestion pipelines in PySpark/Databricks, shape the Data Observability Layer, and drive enterprise analytics initiatives for the DoW CDAO.

The role requires 10+ years in data engineering, a Top Secret/Sci eligibility clearance, and a Bachelor’s in a related field.

Qualifications

• 10+ years of progressive professional experience in data engineering or related technical discipline (Levels III/IV).
• Active Top Secret, SCI Eligible clearance.
• Bachelor’s Degree in Computer Science, Data Science, Information Technology, or a related field.
• Work Location: Arlington County, VA (Hybrid).

Responsibilities

• Lead automated data ingestion pipelines using PySpark and Databricks.
• Guide data structures with Delta Tables for versioning, backups, and rollback safeguards.
• Connect Delta Tables with Vector Databases while enforcing RBAC.
• Mentor engineers and oversee technical delivery with high engineering standards.

Skills

Python

PySpark

Databricks

Delta Tables

Unity Catalog

RBAC

SQL

ETL/ELT

Team leadership

Clearance

Education

Bachelor’s Degree in Computer Science, Data Science, IT or related field

Tools

Delta Lake

Vector Databases

RBAC enforcement tooling

Job description

Nyla Technology Solutions seeks a visionary Lead Data Engineer to guide the design, building, and optimization of high-throughput big data solutions. You will lead automated data ingestion pipelines in PySpark/Databricks, shape the Data Observability Layer, and drive enterprise analytics initiatives for the DoW CDAO.

The role requires 10+ years in data engineering, a Top Secret/Sci eligibility clearance, and a Bachelor’s in a related field.