Platform Engineer – ETL & Data Observability

VT Group (VTG); Herndon (VA); On-site; USD 120,000 - 170,000; Full time

Job summary

VT Group (VTG) seeks a Platform Engineer to design, integrate, and optimize ETL data pipelines within the Palantir ecosystem. You will connect custom ETL workflows to Palantir tooling, develop data transformation solutions with PySpark and Python, and ensure end-to-end data observability across pipelines.

You will collaborate with cross-functional teams to implement logging, governance, and data conditioning, leveraging AI to speed processing and enhance data quality.

Qualifications

• Bachelor's degree in a technical field or equivalent practical experience.
• Experience with Palantir Foundry.
• Proficiency with PySpark and Python.
• Experience implementing logging and monitoring solutions.
• Knowledge of data governance best practices.

Responsibilities

• Design, develop, and maintain scalable ETL data pipelines within Palantir.
• Connect custom ETL workflows to Palantir tooling.
• Develop data transformation and conditioning solutions using PySpark and Python.
• Work with Palantir Foundry and ontology models to organize, integrate, and manage complex datasets.
• Implement logging, monitoring, and observability capabilities across data pipelines.
• Provide stakeholders with end-to-end visibility into data transformation and conditioning processes.
• Leverage AI capabilities to augment data pipelines and improve processing speed and efficiency.
• Apply data governance best practices throughout the data lifecycle.
• Perform validation and quality assurance of data products to ensure accuracy, reliability, and readiness for downstream use.
• Collaborate with cross-functional teams to deliver reliable data products.

Skills

ETL pipelines

PySpark

Python

Ontology concepts

Data governance

DevOps familiarity

AI integration for pipelines

Data observability

Data conditioning

Data quality assurance

Education

Bachelor's degree in Computer Science, Engineering, Finance, or related field

Tools

Palantir Foundry

Job description

VT Group (VTG) seeks a Platform Engineer to design, integrate, and optimize ETL data pipelines within the Palantir ecosystem. You will connect custom ETL workflows to Palantir tooling, develop data transformation solutions with PySpark and Python, and ensure end-to-end data observability across pipelines.

You will collaborate with cross-functional teams to implement logging, governance, and data conditioning, leveraging AI to speed processing and enhance data quality.