Data Platform Engineer

Quantitative Engineering Design; Warszawa; On-site; PLN 240,000 - 360,000; Full time

Benefits offered by this job

Unusual, socially conscious projects

Little bureaucracy

Travel to exotic places

Job summary

Quantitative Engineering Design is seeking a mid/senior Data Platform Engineer (backend) to join our Warsaw team. You will design and maintain data pipelines, structure data across raw, cleaned, and curated layers, and choose optimal batch or streaming approaches based on latency and reliability.

You should have a strong software engineering background, experience with Python, DBT, Dagster, and containerization, and be comfortable collaborating in a multicultural, mission-driven environment.

Qualifications

• Experience designing and maintaining data pipelines using ETL/ELT approaches.
• Reasoning about trade-offs in data pipelines and architectures.
• Understanding data pipeline reliability concepts (idempotency, backfills, late data).
• Ability to structure data systems into raw/cleaned/curated layers and reason about purposes.
• Experience choosing batch vs streaming based on latency and operational complexity.
• Strong software engineering background with tests and readable code.

Responsibilities

• Design and maintain data pipelines for analytics and decision support.
• Build robust data architectures and clean data flow across layers.
• Collaborate in design sessions and code reviews with the team.

Skills

ETL/ELT design

Data pipelines

Data reliability

Data architecture layers

Batch/streaming decisions

Software engineering basics

English proficiency

Tools

Python

Django

DBT

Dagster

Luigi

ClickHouse

Docker

Kubernetes

Terraform

Nix

Job description

Use your talent to make a positive change in the world.

QED offers unique opportunities to talented people that are determined to positively affect the world.

For most software engineers and mathematicians, work is easy to come by, but meaningful work is nearly impossible to find. We select our partners and design our projects very carefully, so they can direct their talents on worthwhile problems whose solutions will clearly benefit the welfare of others.

Location

Warsaw

Employment Type

Full time

Department

QED (https://qed.ai ) is a tech company focused on public health and food security in Sub-Saharan Africa. We build the digital infrastructure and AI used at the intersection of aid and scientific inquiry, including surveillance of HIV/malaria/TB, and nutrient analysis of crops and soils, working at national-scale in several African countries. Our funding comes from multiple philanthropic and governmental organizations, including the Global Fund, Gates Foundation, and the CDC.

We are looking for a mid/senior Data Platform Engineer (backend) to join our team in Warsaw. Desired skills include:

experience designing and maintaining data pipelines using ETL and/or ELT approaches, with the ability to reason about trade-offs rather than defaulting to a single pattern

understanding of data pipeline reliability concepts, including idempotency, backfills, and handling late or corrected data

ability to structure data systems into clear layers (e.g. raw, cleaned, curated) and reason about their different purposes and guarantees

experience deciding between batch, micro-batch, and streaming approaches based on latency, correctness, and operational complexity

strong background in software engineering, version control, writing readable code and tests, robust design, and basic data structures and algorithms

ability to conceive logical software architectures and express yourself clearly, both verbally and in writing

willingness to tackle a wide array of problems and technologies

willingness to participate in regular design sessions, code reviews, and working in teams

development primarily on UNIX-based or OSX-Darwin platforms

working proficiency (≥C1) in speaking, reading, and typing English (≥45 words per minute)

willingness and interest in working with people from other cultures

emotional resilience and social intelligence

and you have to care about the work that you do

and you have to be optimistic at least sometimes, although pessimism tends to be useful

Below are additional skills that are a bonus, but are not required:

understanding of analytical data modeling concepts, including how to transform raw data into analytics-ready datasets and define clear data semantics for downstream consumers

experience with Python, Django, DBT, Dagster, Luigi or Clickhouse

experience with containerization (Docker), Terraform, Kubernetes or Nix

knowledge and experience with data warehousing, online analytical processing, metadata management, dimensional modeling, and relational database theory

experience with programming and/or math competitions

product-oriented mindset

willingness to go on an adventure

domain knowledge and/or interest in the sustainable development goals, particularly in public health, agriculture, and assisting developing countries

What can you expect:

unusual, socially conscious projects

a mix of product, internal platform, and client integration work

ability and encouragement to explore technologies other than your main expertise

optional travel to learn more about problems we're trying to solve

Requires a full-time commitment physically based in Warsaw, Poland, with a hybrid of remote and in-person work throughout the week. We expect at least 2-3 days per week in the office. The working hours are flexible, for example you do not need approval to take out a small chunk in the middle of the day to handle some personal business.

If you are

Passionate about your craft

Interested in international development

It looks like a perfect match! We would like to learn more about you.

Culture and Expectations

We are the opposite of a big corporation. We impose extremely light bureaucracy. There is no matrix organization structure, careerism, or politicking ... we focus only on the mission and delivering excellent work.

We are willing to take unconventional paths and learn quickly through rapid scientific experimentation, unconstrained by excessive regulations. We are highly motivated to work cleverly and succeed, as no one is there to bail us out and inefficiencies cannot be sustained.

Regardless of one’s age or position in life, we ask that job candidates make serious time commitments if they choose to work with us, which will be reciprocated by equal devotion to the cause from others.

We seek people who are eager for a challenge, willing to take on responsibility, passionate about your craft, and genuinely interested in doing good works for the developing world. If this sounds like you, we would love to learn more about you.

Perks of working with QED

Unusual, socially conscious projects

Little bureaucracy

Travel to exotic places around the world!

Work together with our partners changing the world for the better