Lead Data Engineer

Mindbox Sp. z o.o.; Kraków; Hybrid; PLN 200,000 - 320,000; Full time

Benefits offered by this job

Interpolska Health Care

Multisport card

Warta Insurance

Job summary

Mindbox Sp. z o.o. poszukuje doświadczonego Lead Data Engineer, by dołączyć do globalnego zespołu i zaprojektować nowe platformy danych.

Będziesz współpracować z zespołami w Polsce, Wielkiej Brytanii, Indiach i Chinach, aby zwiększyć możliwości platformy, poprawić wydajność i zapewnić operacyjną doskonałość. Zakres obowiązków obejmuje projektowanie i implementację potoków danych w PySpark i Iceberg, optymalizację wydajności oraz nadzorowanie utrzymania Iceberg.

Qualifications

• Duże doświadczenie w PySpark (DataFrame APIs, tuning, debugging).
• Praktyczna wiedza z Iceberg, w tym snapshots, manifests, schema evolution.
• Doświadczenie w CI/CD, obserwowalności i stabilności w produkcji.
• Umiejętność pracy w globalnych zespołach i skuteczna komunikacja po angielsku.
• Podejście ownership i odpowiedzialność za delivery end-to-end.

Responsibilities

• Projektowanie i implementacja potoków danych z PySpark i Iceberg.
• Zwiększanie wydajności Iceberg poprzez optymalizację partycjonowania i wielkości plików.
• Zarządzanie utrzymaniem Iceberg (compaction, rewrite, metadata).
• Zapewnienie wysokich standardów inżynieryjnych przez review, testy, CI/CD i on-call.
• Mentorowanie inżynierów i prowadzenie dyskusji technicznych.

Skills

PySpark

Apache Iceberg

CI/CD

Observability

Global teams

Ownership mindset

English fluency

Software engineering fundamentals

Job description

Nasze wymagania:

• Strong hands-on experience with PySpark (DataFrame APIs, tuning, debugging).
• Practical knowledge of Apache Iceberg, including:
• Optimisation techniques (partitioning strategy, file sizing, layout).
• Internal architecture understanding (snapshots, manifests, schema evolution).
• Maintenance operations (rewrite/compaction, orphan file cleanup, metadata handling).
• Solid software engineering fundamentals: clean, testable, maintainable code, problem-solving.
• Experience with CI/CD pipelines, observability, stability improvements in production systems.
• Ability to work in global teams, excellent communication skills.
• Ownership mindset and responsibility for end-to-end delivery.
• Fluency in English (spoken & written).

O projekcie:

We are seeking an experienced Lead Data Engineer to join our client's global development team and drive the design and implementation of next-generation data platforms. You will work closely with teams in Poland, the UK, India, and China to enhance platform capabilities, improve performance, and ensure operational excellence.

Zakres obowiązków:

• Analyse and capture functional & non-functional requirements for near-term deliveries and long-term evolution.
• Design and implement data pipelines and table management workflows using PySpark and Apache Iceberg.
• Drive Apache Iceberg performance and cost optimisation (e.g., file sizing, partitioning, clustering, query efficiency).
• Manage and improve Apache Iceberg maintenance (compaction, rewrite, snapshot/metadata management, orphan file handling).
• Apply strong understanding of Iceberg internals (metadata layout, schema evolution, partition specs).
• Ensure high engineering standards via code reviews, test automation, CI/CD, observability, and production readiness.
• Provide support for owned services and datasets, including on-call duties.
• Mentor and coach junior engineers; lead technical design discussions.
• Note: Detailed project information will be shared during the recruitment process.

Oferujemy:

• Flexible cooperation model
• Hybrid work setup – 6 days a month from the office in Kraków
• Collaborative team culture – work alongside experienced professionals eager to share knowledge.
• Continuous development – access to training platforms and growth opportunities.
• Comprehensive benefits – including Interpolska Health Care, Multisport card, Warta Insurance, and more.
• High quality equipment – laptop and essential software provided.