Principal Accountabilities/ Responsibilities:
- Design, build, and maintain scalable data pipelines in Foundry (e.g., Pipeline Builder, Code Repositories using PySpark/SQL, transforms) to ingest, clean, and integrate data from ERP, MES, historian, and other enterprise sources.
- Develop and optimize data models and datasets, with attention to performance, reliability, and cost.
- Implement data quality checks, health monitoring, lineage, and documentation so downstream users can trust the data.
- Build and maintain data pipelines and transformations on other platforms (e.g., Databricks, Snowflake, Fabric) as needed, including integration with Foundry.
- Implement incremental data processing, data validation, and performance optimization techniques.
- Ensure alignment with enterprise data architecture standards and integration strategies.
- Design, build, and govern the Foundry ontology: object types, properties, link types, actions, and functions that reflect real business and manufacturing concepts.
- Partner with business and operations stakeholders to translate processes and requirements into clear, reusable semantic models.
- Manage ontology changes, versioning, permissions, and data security, including row- and column-level controls where required.
- Establish naming conventions, standards, and best practices for ontology and pipeline development.
Applications and AI Solutions
- Support the development of Foundry applications (e.g., Workshop, Slate, OSDK-based apps) that use the ontology to drive operational decisions.
- Design and build AI/ML and generative AI solutions (e.g., AIP Logic, agents, model integration, LLM-enabled workflows) grounded in ontology data when business needs require it.
- Work with data scientists, analysts, and developers to operationalize models and move prototypes into production.
Collaboration and Governance
- Work with IT, Cybersecurity, and business teams on data governance, access, and compliance.
- Troubleshoot and resolve pipeline, ontology, and platform issues, and provide production support.
- Document solutions and share knowledge, and coach team members on Foundry and modern data engineering practices.
- Evaluate new platform capabilities and tools, and recommend the right tool for each use case, whether Foundry, Databricks, or another platform.
- Communicate proactively with stakeholders across locations and time zones through virtual meetings, chat, and written documentation.
Qualifications/ Education and Experience Requirements:
- Bachelor's degree in Computer Science, Information Systems, Data Engineering, Engineering, or a related field, or equivalent experience.
- 5+ years of data engineering experience.
- Minimum 2 years of hands-on Palantir Foundry experience preferred, including:
- - Building production pipelines with Pipeline Builder and/or Code Repositories (PySpark/SQL).
- - Designing and maintaining ontology objects, links, actions, and functions.
- - Configuring permissions and data security in Foundry.
- Exposure to Foundry application development (Workshop, Slate, or OSDK) and AIP is preferred.
- Experience with Databricks or a comparable platform (e.g., Spark, Snowflake, Azure Synapse/Fabric) is required, or a demonstrated ability to learn one quickly.
- Palantir Foundry certifications (e.g., Data Engineer, Application Developer) are preferred.
- Databricks, Azure, or other cloud data certifications are a plus.
- Experience in manufacturing, supply chain, or industrial environments is preferred.
- Prior experience working effectively in a remote or distributed team is preferred.
- Remote role, performed from a home office.
- The employee must have a dedicated workspace and a reliable high-speed internet connection.
- The employee is expected to be available during core business hours aligned to Eastern Time, and to attend virtual meetings.