Software Engineer - Hosted Model Infrastructure

Palantir Technologies; Washington (District of Columbia); On-site; USD 145,000 - 200,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

Paid time off

401(k) plan enrollment

Relocation assistance

Job summary

Palantir Technologies is looking for a software engineer specializing in machine learning infrastructure in Washington, D.C. You will design and maintain high-performance systems integrating AI capabilities with strict security measures.

The position requires strong coding skills, experience with containers, and a background in relevant engineering fields. Attractive salary range between $145,000 and $200,000 plus comprehensive benefits.

Join us to solve critical problems at the intersection of infrastructure and AI.

Qualifications

• 4+ years of experience building and operating production systems.
• Strong coding skills in Java, C++, Python, Rust, etc.
• Experience with containers and Kubernetes in production.

Responsibilities

• Build high-performance model-serving infrastructure.
• Design intelligent request handling for multi-tenant access.
• Develop observability for production AI systems.
• Debug complex issues across the stack.

Skills

Strong coding skills in programming languages (Java, C++, Python, Rust)

Engineering background (Computer Science, Mathematics, etc.)

Experience with containers and Kubernetes

Strong debugging skills

Ability to work across multiple languages and layers

Education

4+ years of professional software engineering experience

Bachelor's degree in a relevant field

Tools

Docker

Kubernetes

Job description

About Palantir

Palantir builds the world’s leading software for data‑driven decisions and operations. By bringing the right data to the people who need it, our platforms empower partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.

The Role

We are a software engineering team that specializes in enabling machine‑learning models in production. We deploy AI models to run in a variety of environments: air‑gapped government networks, forward‑deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access.

Our approach treats models like any other software: continuously tested, continually delivered, packaged for reproducible deployment, and built for long‑term maintainability. You will own services end‑to‑end and work across the full stack, from inference engines and GPU scheduling to deployment pipelines, observability, and integration with the Palantir platform. The goal is to deliver new models and capabilities quickly and continuously.

Join us if you want to solve problems at the intersection of infrastructure and machine learning that directly enable critical customers.

Technologies We Use

• Back‑end languages: Java, Rust, Python, Go
• Model serving engines for GPU‑accelerated inference
• Docker and Kubernetes for containerization and orchestration
• Industry‑standard build tooling (Gradle, GitHub)

Core Responsibilities

• Build high‑performance model‑serving infrastructure that integrates with security models, hardware constraints, and different inference engines.
• Design intelligent request handling, including authentication, rate limiting, concurrency control, and audit logging for multi‑tenant model access.
• Build and maintain packaging and deployment pipelines that enable fast, secure, and reliable model rollouts across on‑premises and air‑gapped environments.
• Develop observability for production AI systems to enable easy service monitoring and fast incident triage and resolution.
• Debug complex issues and performance problems throughout the stack, including open‑source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly.
• Design and run testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production.
• Work with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need.
• Integrate hosted model infrastructure with Palantir’s deployment, configuration, and identity systems.

What We Value

• Ownership mindset and bias toward quality.
• High empathy for customer needs and drive to deliver reliable, easy‑to‑use models.
• Ability to work effectively across multiple languages and layers of the stack.
• Strong debugging skills and motivation to trace problems from application code through containers, orchestration, and hardware.
• Curiosity about emerging AI capabilities and the ability to evaluate and integrate new models and technologies quickly.
• Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary.

What We Require

• 4+ years of professional software engineering experience building and operating production systems.
• Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or a similar field.
• Strong coding skills in programming languages such as Java, C++, Python, Rust, or similar. Familiarity with the Python ML ecosystem is valuable.
• Experience with containers, Kubernetes, and deploying backend services in production environments.
• Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality.

Salary

The salary range for this position is estimated to be $145,000 – $200,000 per year. Total compensation may also include restricted stock units, a sign‑on bonus, and other incentives. Compensation will be determined by individual qualifications, work experience, skills, and other factors. The estimate excludes the value of any potential sign‑on bonus and benefits.

Benefits

• Medical, dental, vision, and voluntary life insurance for employees and eligible dependents.
• Basic life, AD&D, and disability insurance provided automatically.
• Commuter benefits.
• Relocation assistance.
• Paid time off (no accrual). 2 weeks built into the end of each year (subject to team and business needs).
• 10 paid holidays each year.
• Supportive leave of absence program including time off for military service and medical events.
• Paid leave for new parents and subsidized backup care for all parents.
• Fertility and family‑building benefits, including adoption, surrogacy, and preservation.
• Stipend to help with expenses that come with a new child.
• 401(k) plan enrollment.

Equal Opportunity Employer Statement

We are proud to be an Equal Opportunity Employer for all, including but not limited to Veterans and those with disabilities. Palantir is committed to making the application and hiring process accessible to everyone and will provide a reasonable accommodation for those living with a disability.