Foundry Engineering Lead: Architect Scalable Data Platform

Nscale; San Francisco (CA); On-site; USD 180,000 - 300,000; Full time

Benefits offered by this job

Cloud platforms experience

Data governance & security

Mentorship & leadership

Job summary

Nscale is seeking a Foundry Engineering Lead to act as the technical authority for our Foundry implementation, guiding design, development, and governance of our data platform that underpins internal operations, workflows, and customer-facing capabilities.

You will lead architectural direction, build core data infrastructure, and collaborate with Operations, Infrastructure, Platform Engineering, Product, and Commercial teams to deliver reliable data products at scale.

Qualifications

• Deep, hands-on Palantir Foundry expertise including ontology modelling, pipeline development, data architecture, and large-scale platform design.
• Proven experience providing technical leadership—setting architectural direction, mentoring engineers, and defining standards in a data engineering context.
• Strong proficiency in Python, with experience applying data engineering frameworks (e.g. Spark, PySpark, Dask, pandas) to large, complex datasets.
• Familiarity with API-driven data integration, including REST, GraphQL, and Foundry Action APIs.
• Experience with Git-based development workflows, including code reviews, version control, and CI/CD pipelines.
• Excellent communication skills—able to explain complex technical concepts clearly to both technical and non-technical stakeholders.
• Comfortable operating in ambiguous, fast-moving environments where requirements evolve quickly.
• A bias toward ownership, pragmatism, and delivering useful solutions.

Responsibilities

• Lead the design and architecture of Nscale's Foundry implementation, ensuring scalability, security, and long-term data integrity.
• Define the technical strategy for Foundry development, aligning with business objectives and the programme-wide data roadmap.
• Act as the technical authority on Foundry across the organisation—setting standards, driving architectural decisions, and owning the evolution of the platform.
• Conduct code reviews and ensure adherence to best practices in data modelling, pipeline development, and integration.
• Mentor engineers and analysts working within the Foundry environment, raising technical capability and quality across the team.
• Design and build scalable, reliable data pipelines that ingest data from infrastructure, platform services, and business systems.
• Define data models, schemas, and ontology structures that support operational workflows and use cases across the business.
• Clean, transform, and structure data to create a digital twin of Nscale.
• Implement permissioning and manage access and security of the Foundry implementation.
• Create trusted datasets and metrics that power workflows, internal tools, and customer-facing insights.
• Enable self-service analytics by establishing clear data contracts, documentation, and semantic layers.
• Build use cases including, but not limited to: Capacity planning, Cost optimisation, Reliability analysis, Customer reporting.
• Collaborate with Product and Commercial teams to translate real-world business questions into robust data solutions.
• Implement data quality checks, monitoring, and alerting to ensure data correctness and availability.
• Codify data lineage, freshness, and consistency across systems.
• Establish and enforce governance standards around data versioning, access control, and compliance appropriate for a fast-scaling company.
• Oversee the integration of Foundry with existing systems and data sources, ensuring seamless data flows.

Skills

Foundry Expertise

Platform architecture

Data engineering

Python

Spark/PySpark

Dask

pandas

API integration

Git and CI/CD

Communication

Ownership

Tools

Palantir Foundry

Python

Spark

PySpark

Dask

pandas

REST API

GraphQL

Foundry Action APIs

Git

Job description

Nscale is seeking a Foundry Engineering Lead to act as the technical authority for our Foundry implementation, guiding design, development, and governance of our data platform that underpins internal operations, workflows, and customer-facing capabilities.

You will lead architectural direction, build core data infrastructure, and collaborate with Operations, Infrastructure, Platform Engineering, Product, and Commercial teams to deliver reliable data products at scale.