Data Engineer: Foundry/Databricks with Secret Clearance

LMI Consulting, LLC; Pittsburgh (Allegheny County); On-site; USD 101,144 - 174,591; Full time

Job summary

LMI Consulting, LLC is seeking a Data Engineer in Pittsburgh with a Secret clearance. You will build and operate data pipelines in Palantir Foundry (Vantage) and Databricks, lead data engineering efforts, and collaborate with data scientists and software teams. On-site Pittsburgh preference, remote option available.

The role emphasizes leadership, technical mentorship, and advancing AI-enabled data solutions for Army AI2C mission applications.

Qualifications

• Bachelor's degree in Computer Science, Data Science, Data Engineering, Software Engineering, Information Systems, or related field, or equivalent work experience.
• 3-5 years of data engineering experience with Palantir Foundry (Vantage)
• Experience with Foundry pipeline development (PySpark/Pipeline Builder) and Foundry Workshop
• Databricks experience as a secondary platform (notebook, Delta Lake, Spark)
• Proficiency in Python and SQL for data manipulation
• Familiarity with data visualization tools for pipeline outputs
• Experience leading data pipelines and applying CI/CD practices
• Active Secret Clearance or ability to obtain one

Responsibilities

• Lead data engineering efforts within Palantir Foundry (Vantage) including ontology design and pipeline development.
• Build and optimize Foundry pipelines using PySpark and Foundry's framework.
• Develop data workflows in Databricks with notebooks and Delta Lake tables.
• Write robust Python/SQL code for Foundry Workshop or Slate apps.
• Integrate AI/ML capabilities into Foundry and Databricks apps.
• Mentor teammates and drive AI-forward development practices.
• Collaborate with data scientists, UX designers, and developers using Git and Agile methods.
• Engage Army stakeholders to expand development footprint.

Skills

Python

PySpark

SQL

Palantir Foundry (Vantage)

Databricks

Git

Agile

Communication

Education

Bachelor's degree in CS/related field

Master's degree (preferred)

Tools

Databricks

Git

Job description

LMI Consulting, LLC is seeking a Data Engineer in Pittsburgh with a Secret clearance. You will build and operate data pipelines in Palantir Foundry (Vantage) and Databricks, lead data engineering efforts, and collaborate with data scientists and software teams. On-site Pittsburgh preference, remote option available.

The role emphasizes leadership, technical mentorship, and advancing AI-enabled data solutions for Army AI2C mission applications.