Platform Engineer: ETL Pipelines & Data Observability

VTG Defense; Herndon (VA); On-site; USD 120,000 - 180,000; Full time

Job summary

VTG Defense is seeking a Platform Engineer to design, integrate, and optimize ETL data pipelines within Palantir Foundry. You will connect custom ETL workflows, implement logging and observability, and collaborate with cross-functional teams to deliver reliable data products.

The role requires PySpark and Python expertise, experience with Palantir Foundry, and a TS/SCI with Polygraph clearance. You will work on data conditioning, governance, and AI-augmented processing to enable advanced

Qualifications

• Active TS/SCI with Polygraph clearance.
• Bachelor's degree or equivalent practical experience.
• Experience with Palantir Foundry.
• Proficiency with PySpark and Python.
• Experience implementing ETL pipelines.

Responsibilities

• Design, develop, and maintain scalable ETL data pipelines within the Palantir ecosystem.
• Connect custom ETL conditioners and workflows to Palantir tooling.
• Develop data transformation and conditioning solutions using PySpark and Python.
• Work with Palantir Foundry and ontology models to organize, integrate, and manage complex datasets.
• Implement logging, monitoring, and observability capabilities across data pipelines.
• Provide stakeholders with end-to-end visibility into data transformation and conditioning processes.
• Leverage AI capabilities to augment data pipelines and improve processing speed and efficiency.
• Apply data governance best practices throughout the data lifecycle.
• Perform validation and quality assurance of data products to ensure accuracy, reliability, and readiness for downstream use.
• Integrate validated data products into advanced analytics and data exploitation tools.
• Support deployment and operational activities using DevOps methodologies and practices.
• Collaborate with cross-functional engineering, data, and stakeholder teams to deliver reliable and actionable data products.
• Identify opportunities to improve data conditioning, pipeline performance, scalability, and observability.

Skills

PySpark

Python

Palantir Foundry

ETL pipelines

Ontology

Data governance

Logging & Monitoring

DevOps

AI for data pipelines

Data validation

Education

Bachelor's degree in Computer Science, Engineering, Finance, or related technical field

Job description

VTG Defense is seeking a Platform Engineer to design, integrate, and optimize ETL data pipelines within Palantir Foundry. You will connect custom ETL workflows, implement logging and observability, and collaborate with cross-functional teams to deliver reliable data products.

The role requires PySpark and Python expertise, experience with Palantir Foundry, and a TS/SCI with Polygraph clearance. You will work on data conditioning, governance, and AI-augmented processing to enable advanced