Platform Engineer: Data Pipelines, AI & Observability

VTG; Herndon (VA); On-site; USD 130,000 - 170,000; Full time

Job summary

VTG seeks a Platform Engineer to design, integrate, and optimize Palantir-based data pipelines, focusing on end-to-end observability, data conditioning, and scalable ETL workflows.

You will work with Palantir Foundry, PySpark, and Python to implement logging, governance, and DevOps practices, delivering reliable data products to stakeholders while leveraging AI to accelerate processing and ensure data quality. This role requires active TS/SCI with polygraph to support sensitive data projects.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Finance, or a related technical field, or equivalent practical experience.
• Experience with Palantir Foundry.
• Strong understanding of Ontology and Data Ontology.
• Demonstrated experience designing, building, and integrating ETL pipelines.
• Proficiency with PySpark and Python.

Responsibilities

• Design, develop, and maintain scalable ETL data pipelines within the Palantir ecosystem.
• Connect custom ETL conditioners and workflows to Palantir tooling.
• Develop data transformation and conditioning solutions using PySpark and Python.
• Work with Palantir Foundry and ontology models to organize, integrate, and manage complex datasets.
• Implement logging, monitoring, and observability capabilities across data pipelines.
• Provide stakeholders with end-to-end visibility into data transformation and conditioning processes.
• Leverage AI capabilities to augment data pipelines and improve processing speed and efficiency.
• Apply data governance best practices throughout the data lifecycle.
• Perform validation and quality assurance of data products to ensure accuracy, reliability, and readiness for downstream use.
• Integrate validated data products into advanced analytics and data exploitation tools.
• Support deployment and operational activities using DevOps methodologies and practices.
• Collaborate with cross-functional engineering, data, and stakeholder teams to deliver reliable and actionable data products.
• Identify opportunities to improve data conditioning, pipeline performance, scalability, and observability.

Skills

ETL pipelines design

PySpark

Python

Data observability

Data governance

DevOps practices

AI augmentation

Data validation

Cross-functional collaboration

Ontology knowledge

Education

Bachelor's degree in Computer Science or related field

Tools

Palantir Foundry

Ontology models

Job description

VTG seeks a Platform Engineer to design, integrate, and optimize Palantir-based data pipelines, focusing on end-to-end observability, data conditioning, and scalable ETL workflows.

You will work with Palantir Foundry, PySpark, and Python to implement logging, governance, and DevOps practices, delivering reliable data products to stakeholders while leveraging AI to accelerate processing and ensure data quality. This role requires active TS/SCI with polygraph to support sensitive data projects.