Foundry Data Engineer - ETL Pipelines & Big Data

Quantum Technologies. LLC; Saint Paul (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Quantum Technologies. LLC is seeking a senior data engineer to design and optimize data pipelines on Palantir Foundry. You will leverage Python, PySpark and SQL to ingest and transform large-scale data while ensuring quality and governance.

Collaborate with data scientists and analytics teams to build ontologies and scalable data models, and drive performance improvements across complex pipelines in a cloud environment.

Qualifications

• Bachelor’s or Master’s degree in a related field.
• 5 to 7 years of Palantir Foundry experience.
• Advanced Python, PySpark, and SQL skills.
• Experience with AWS, Azure, and GCP and big data tech.
• Ability to design scalable ETL/ELT pipelines.
• Collaborate with data scientists to build data models and governance.
• Strong analytical thinking and problem-solving abilities.
• Knowledge of data quality and pipeline performance.

Responsibilities

• Build and optimize data pipelines within the Palantir Foundry platform.
• Design and implement scalable ETL/ELT pipelines using Foundry tools.
• Use Python, PySpark, and SQL to ingest, transform, and manage large data sets.
• Collaborate with data scientists and analytics teams on data models and governance.
• Perform data analysis, validation, and optimization to ensure data integrity.

Skills

Python

PySpark

SQL

Big data

Cloud platforms

Agile methodologies

Data modeling

Data analysis

Education

Bachelor’s or Master’s in CS/IS/Finance/Data Eng

Tools

Palantir Foundry

Job description

Quantum Technologies. LLC is seeking a senior data engineer to design and optimize data pipelines on Palantir Foundry. You will leverage Python, PySpark and SQL to ingest and transform large-scale data while ensuring quality and governance.

Collaborate with data scientists and analytics teams to build ontologies and scalable data models, and drive performance improvements across complex pipelines in a cloud environment.