Senior Data Platform Engineer

European Tech Recruit

Berlin

Híbrido

EUR 90 000 - 130 000

Tempo integral

há 13 horas
Torna-te num dos primeiros candidatos

Recebe uma resposta deste empregador

O teu currículo e carta de apresentação, adaptados exatamente ao que estão a contratar.

Ultrapassa os filtros ATS

Resumo da oferta

European Tech Recruit is seeking a Senior Data Platform Engineer in Berlin, hybrid role, to design and operate petabyte-scale data-processing systems supporting AI research. You will enable reproducible datasets, scalable pipelines, and robust fault-tolerant infrastructure for ML workloads.

You will work with Ray, Spark, Flink and Kubernetes to optimize performance, cost and reliability across diverse GPU-accelerated stages, aligning with engineering standards and observability practices.

Qualificações

  • PhD in Computer Science, Computer Engineering, Software Engineering or a related discipline.
  • At least 3 years of relevant full-time experience building or operating distributed data-processing systems or ML data infrastructure.
  • Strong experience with distributed processing frameworks such as Ray, Apache Spark or Flink, alongside workflow orchestration and large-scale pipeline execution.
  • Solid understanding of distributed computing, performance profiling, throughput optimisation and fault tolerance.
  • Experience with high-performance data formats such as Apache Arrow or Parquet, and dataset versioning, lineage or reproducibility.
  • Familiarity with workload orchestration and scheduling tools such as Kubernetes or Slurm, and containerisation technologies such as Docker or Enroot.
  • Strong software engineering skills and experience owning production systems where reliability, performance and cost matter.

Responsabilidades

  • Design, build and operate distributed data-processing infrastructure for petabyte-scale workloads.
  • Develop and optimise data pipelines, execution frameworks and workflow orchestration across diverse computational workloads, including GPU-accelerated stages.
  • Improve system performance, throughput and resource utilisation while managing infrastructure costs.
  • Build reliable, observable pipelines with robust failure recovery, restartability and monitoring.
  • Implement dataset versioning, lineage and reproducibility to ensure research datasets can be reliably traced and recreated.
  • Integrate data infrastructure with large-scale ML training workflows and establish engineering standards and best practices.

Conhecimentos

Distributed systems
Workflow orchestration
Performance profiling
Fault tolerance

Formação académica

PhD in Computer Science/Engineering or related

Ferramentas

Ray
Apache Spark
Flink
Kubernetes
Slurm
Docker
Enroot
Apache Arrow
Parquet

Descrição da oferta de emprego

Berlin, Germany | Hybrid - 3 days onsite per week

Join a highly technical AI research and engineering organisation building the data infrastructure behind next-generation AI models. As a Senior Data Platform Engineer, you’ll design and operate distributed data-processing systems capable of handling petabyte-scale workloads, ensuring research teams can access reliable, reproducible datasets and run complex pipelines efficiently.

This role combines distributed systems engineering, workflow orchestration and performance optimisation, with a focus on scalability, fault tolerance and cost efficiency across large-scale data and machine learning infrastructure.

Key Responsibilities
  • Design, build and operate distributed data-processing infrastructure for petabyte-scale workloads.
  • Develop and optimise data pipelines, execution frameworks and workflow orchestration across diverse computational workloads, including GPU-accelerated stages.
  • Improve system performance, throughput and resource utilisation while managing infrastructure costs.
  • Build reliable, observable pipelines with robust failure recovery, restartability and monitoring.
  • Implement dataset versioning, lineage and reproducibility to ensure research datasets can be reliably traced and recreated.
  • Integrate data infrastructure with large-scale ML training workflows and establish engineering standards and best practices.
Required Experience & Skills
  • PhD in Computer Science, Computer Engineering, Software Engineering or a related discipline.
  • At least 3 years of relevant full-time experience building or operating distributed data-processing systems or ML data infrastructure.
  • Strong experience with distributed processing frameworks such as Ray, Apache Spark or Flink, alongside workflow orchestration and large-scale pipeline execution.
  • Solid understanding of distributed computing, performance profiling, throughput optimisation and fault tolerance.
  • Experience with high-performance data formats such as Apache Arrow or Parquet, and dataset versioning, lineage or reproducibility.
  • Familiarity with workload orchestration and scheduling tools such as Kubernetes or Slurm, and containerisation technologies such as Docker or Enroot.
  • Strong software engineering skills and experience owning production systems where reliability, performance and cost matter.
Why Apply?

Help build the infrastructure that enables cutting-edge AI research at scale. You’ll tackle challenging distributed-systems problems, influence core architectural decisions and work closely with researchers and engineers to make large-scale data and ML workflows more efficient, reliable and reproducible.

By applying to this role you understand that we may collect your personal data and store and process it on our systems. For more information please see our Privacy Notice (https://eu-recruit.com/about-us/privacy-notice/)

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Senior Data Engineer
Senior Data Engineer

Ai Futures • München

Híbrido
EUR 80 000 - 110 000
Modern, innovative workplace
Collaborative team culture
Flexible working arrangements
+1
Senior Data Engineer
Senior Data Engineer

Jobgether • Alemanha

Teletrabalho
EUR 91 000 - 127 000
Fully remote in Europe
Data Engineer
Data Engineer

Tides Digital  • Berlin

Presencial
EUR 60 000 - 80 000
Senior Site Reliability Engineer (Data Platform)
Senior Site Reliability Engineer (Data Platform)

Jobgether • Alemanha

Presencial
EUR 90 000 - 140 000
Competitive compensation package
Career development and continuous学习
Flexibility and autonomy
+4
Senior Data Engineer (m/f/d)
Senior Data Engineer (m/f/d)

eigenblue • München

Híbrido
EUR 90 000 - 130 000
Hybrid work setup
Lunch subsidy
Deutschlandticket
+4
Staff Data Engineer (Databricks specialist)
Staff Data Engineer (Databricks specialist)

Jobgether • Alemanha

Teletrabalho
EUR 150 000 - 190 000
Fully remote
Flexible hours
Global collaboration
Lead Data Platform Engineer
Lead Data Platform Engineer

Clera • Berlin

Presencial
EUR 90 000 - 130 000
Senior AI R&D Engineer (Model Training)
Senior AI R&D Engineer (Model Training)

Aleph Alpha GmbH • Alemanha

Presencial
EUR 90 000 - 150 000
30 Days vacation
Flexible working hours
Work with international industry and科学
+1
Data Engineer - Berlin
Data Engineer - Berlin

Yeah! Global • Berlin

Presencial
EUR 55 000 - 80 000
Senior Data Engineer – Hybrid in Düsseldorf | Analytics
Senior Data Engineer – Hybrid in Düsseldorf | Analytics

beON consult • Alemanha

Híbrido
USD 104 000 - 150 000