Platform Engineer: ETL Pipelines & Data Observability

VTG Defense

Herndon (VA)

On-site

USD 120,000 - 180,000

Full time

6 days ago

Be an early applicant

Application generator

An application made for this job — a tailored resume and cover letter that speak straight to the posting.

Get past ATS filters

Job summary

VTG Defense is seeking a Platform Engineer to design, integrate, and optimize ETL data pipelines within Palantir Foundry. You will connect custom ETL workflows, implement logging and observability, and collaborate with cross-functional teams to deliver reliable data products.

The role requires PySpark and Python expertise, experience with Palantir Foundry, and a TS/SCI with Polygraph clearance. You will work on data conditioning, governance, and AI-augmented processing to enable advanced

Qualifications

Active TS/SCI with Polygraph clearance.
Bachelor's degree or equivalent practical experience.
Experience with Palantir Foundry.
Proficiency with PySpark and Python.
Experience implementing ETL pipelines.

Responsibilities

Design, develop, and maintain scalable ETL data pipelines within the Palantir ecosystem.
Connect custom ETL conditioners and workflows to Palantir tooling.
Develop data transformation and conditioning solutions using PySpark and Python.
Work with Palantir Foundry and ontology models to organize, integrate, and manage complex datasets.
Implement logging, monitoring, and observability capabilities across data pipelines.
Provide stakeholders with end-to-end visibility into data transformation and conditioning processes.
Leverage AI capabilities to augment data pipelines and improve processing speed and efficiency.
Apply data governance best practices throughout the data lifecycle.
Perform validation and quality assurance of data products to ensure accuracy, reliability, and readiness for downstream use.
Integrate validated data products into advanced analytics and data exploitation tools.
Support deployment and operational activities using DevOps methodologies and practices.
Collaborate with cross-functional engineering, data, and stakeholder teams to deliver reliable and actionable data products.
Identify opportunities to improve data conditioning, pipeline performance, scalability, and observability.