Senior Algorithm Engineer (Python/Spark–Distributed Processing)

Xcede; Berlin; Remote; EUR 70.000 - 90.000; Vollzeit

Zusammenfassung

A leading data solutions firm seeks a Senior Algorithm Engineer to design and deploy algorithms/data models, primarily using Python and Spark for large-scale data processing. The ideal candidate will have proven experience in delivering production-grade algorithms, alongside strong proficiency in modern data stacks and distributed systems. This role offers the flexibility of remote work while providing opportunities for technical leadership and mentorship in a collaborative setting.

Qualifikationen

• Proven experience delivering production algorithms/data models supporting pricing, forecasting, and optimisation.
• Strong Python proficiency and exposure to modern data stacks.
• Experience with distributed systems and scalable data processing.

Aufgaben

• Design and deploy algorithms/data models.
• Develop and optimise Spark/PySpark batch pipelines for large-scale data processing.
• Provide technical leadership and mentorship.

Kenntnisse

Production algorithms/data models

Python proficiency

Spark/PySpark pipelines

Data communication

Tools

Databricks

SQL

Pandas

NumPy

Jobbeschreibung

Senior Algorithm Engineer (Python / Spark – Distributed Data Processing)

Location: UK (O/IR35) / Belgium / Netherlands / Germany (B2B)

Working model: Remote

Start: ASAP

This is not an ML / Data Science / GenAI role

What you’ll be doing

• Design, build and deploy algorithms/data models supporting pricing, forecasting and optimisation use cases in production
• Develop and optimise distributed Spark / PySpark batch pipelines for large-scale data processing
• Write production-grade Python workflows implementing complex, explainable business logic
• Work with Databricks for job execution, orchestration and optimisation
• Improve pipeline performance, reliability and cost efficiency across high-volume workloads
• Collaborate with engineers and domain specialists to translate requirements into scalable solutions
• Provide senior-level ownership through technical leadership, mentoring and best-practice guidance

Key experience required

• Proven experience delivering production algorithms/data models (forecasting, pricing, optimisation or similar)
• Strong Python proficiency and modern data stack exposure (SQL, Pandas/NumPy + PySpark; Dask/Polars/DuckDB a bonus)
• Build, schedule and optimise Spark/PySpark pipelines in Databricks (Jobs/workflows, performance tuning, production delivery)
• Hands-on experience with distributed systems and scalable data processing (Spark essential)
• Experience working with large-scale/high-frequency datasets (IoT/telemetry, smart meter, weather, time-series)
• Clear communicator able to influence design decisions, align stakeholders and operate autonomously

Nice to have

• Energy/utilities domain exposure
• Cloud ownership experience (AWS preferred, Azure also relevant)
• Experience defining microservices / modular components supporting data products