Senior Data Engineer - Foundry & Big Data Pipelines

EPAM Systems; Portugal; Hybrid; EUR 70,000 - 100,000; Full time

Benefits offered by this job

Competitive compensation

Career growth opportunities

Flexible work hours

Job summary

EPAM Systems in Portugal is seeking a Senior Data Engineer with a strong software engineering background and expertise in Palantir Foundry. You will ingest data, develop large-scale PySpark pipelines, perform entity resolution, and apply data science techniques to enable stakeholders to select optimal candidates across features.

The role emphasizes English communication, collaboration with cross-functional teams, and working on cutting-edge data platforms with a focus on engineering excellence

Qualifications

• 4+ years of front-end software engineering experience.
• Hands-on experience with Palantir Foundry including code workspace, pipeline builder and ontology for back-end pipeline development using Python, PySpark and Spark.
• Experience with GenAI technologies.
• Proficiency in JavaScript, Workshop and Slate.

Responsibilities

• Ingest data from multiple sources and domains into the Palantir Foundry platform.
• Develop and maintain a large-scale Big Data pipeline (PySpark) consisting of more than 350 datasets including orchestration and monitoring.
• Perform entity resolution (fuzzy matching) of sites (healthcare centers) and investigators from diverse sources.
• Apply data science techniques for segmentation and targeting of sites and investigators enabling users to select the best potential candidates across various features.
• Create front-end applications to facilitate data consumption by end-users utilizing Workshop and coding in TypeScript.

Skills

Front-end development

English communication

Data engineering

Tools

Palantir Foundry

Python

PySpark

Spark

Workshop

Slate

Job description

EPAM Systems in Portugal is seeking a Senior Data Engineer with a strong software engineering background and expertise in Palantir Foundry. You will ingest data, develop large-scale PySpark pipelines, perform entity resolution, and apply data science techniques to enable stakeholders to select optimal candidates across features.

The role emphasizes English communication, collaboration with cross-functional teams, and working on cutting-edge data platforms with a focus on engineering excellence