Project Role: Application Lead
Project Role Description: Lead the effort to design, build and configure applications, acting as the primary point of contact.
Requirements and Qualifications:
- Minimum 5 years of experience
- 15 years of full-time education
- Must have skills: Palantir Foundry
- Good to have skills: NA
Job Details
- Seniority level: Mid-Senior level
- Employment type: Full-time
- Job function: Information Technology and Engineering
- Industry: Software Development
Lead Data Engineer
Role Description: Design, build and enhance applications to meet business process and requirements in Palantir Foundry.
Responsibilities and Key Qualifications:
- Responsible for designing, developing, testing, and supporting data pipelines and applications on Palantir Foundry.
- Configure and customize Workshop to design and implement workflows and ontologies.
- Collaborate with data engineers and stakeholders to ensure successful deployment and operation of Palantir Foundry applications.
- Work with stakeholders including the product owner, data, and design teams to assist with data-related technical issues and design the data pipeline.
- Work independently, troubleshoot issues and optimize performance.
- Communicate design processes, ideas, and solutions clearly and effectively to team and client.
- Assist junior team members in improving efficiency and productivity.
- Proficiency in PySpark, Python, and SQL with demonstrable ability to write & optimize SQL and spark jobs.
- Hands on experience on Palantir Foundry services like Data Connection, Code repository, Contour, Data lineage & Health checks.
- Hands‑on experience in data engineering and building data pipelines (Code/No Code) for ELT/ETL data migration, data refinement & data quality checks on Palantir Foundry.
- Experience in ingesting data from different external source systems using data connections and sync.
- Good knowledge on Spark Architecture and experience in performance tuning & code optimization.
- Proficient in managing both structured and unstructured data, with expertise in handling various file formats such as CSV, JSON, Parquet, and ORC.
- Experience in developing and managing scalable architecture & managing large data sets.
- Good understanding of data loading mechanism and adeptly implement strategies for capturing CDC.
- Nice to have test driven development and CI/CD workflows.
- Experience in version control software such as Git and working with major hosting services (Azure DevOps, GitHub, Bitbucket, Gitlab).