Defense Data Engineer: Databricks, Spark & Palantir

careers-fedwriters; Bailey's Crossroads (VA); On-site; USD 105,000 - 115,000; Full time

Benefits offered by this job

Health Insurance

Dental Insurance

Vision Insurance

Long-term Disability Insurance

Short-term Disability Insurance

Life Insurance

401(k) Plan

Holiday Pay

Paid Time Off

Job summary

FWI in Arlington, VA seeks a Data Engineer to design and build end‑to‑end data pipelines powering national defense decision‑making. You will work with Databricks, Spark, Python, SQL, and Palantir Foundry to transform raw data into actionable insights.

This full‑time role requires 3–10 years of related experience and an active Secret clearance. You’ll collaborate with cross‑functional teams, enforce data governance, and optimize large datasets for performance and reliability, making a direct

Qualifications

• 3–10 years of directly related experience and a BA/BS or MA/MS in a related field.
• 3+ years of experience as a Data Engineer.
• Proficiency in ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks.
• Experience with large datasets and big data technologies to process and analyze data efficiently.
• Understanding of data modeling/visualization, database design principles, and data governance practices.
• Must possess an active Secret clearance at minimum.

Responsibilities

• Design, develop, and implement end-to-end data pipelines using ETL and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL.
• Work closely with cross-functional teams to understand data requirements and design optimal data models and architectures.
• Create and optimize data pipelines from scratch, ensuring scalability, reliability, and high-performance processing.
• Perform data cleansing, data integration, and data quality assurance for large datasets.
• Leverage big data technologies to efficiently process and analyze large datasets.
• Collaborate with data scientists, analysts, and stakeholders to provide timely data insights and support decision-making.
• Troubleshoot data-related problems and provide innovative solutions to address complex data challenges.
• Implement and enforce data governance policies and procedures, ensuring regulatory compliance and industry best practices.

Skills

Databricks

Python

SQL

Spark

ETL

Jupyter Notebooks

Big data

Data governance

Security clearance

Education

BA/BS or MA/MS in related field

Tools

Databricks

Python

Spark

Scala

JavaScript/JSON

SQL

Jupyter Notebooks

Palantir Foundry

Workshop

Ontology Manager

Gaia

Advana

Job description

FWI in Arlington, VA seeks a Data Engineer to design and build end‑to‑end data pipelines powering national defense decision‑making. You will work with Databricks, Spark, Python, SQL, and Palantir Foundry to transform raw data into actionable insights.

This full‑time role requires 3–10 years of related experience and an active Secret clearance. You’ll collaborate with cross‑functional teams, enforce data governance, and optimize large datasets for performance and reliability, making a direct