Senior Data Engineer

Teralytics; Berlin; Vor Ort; EUR 50.000 - 90.000; Vollzeit

Zusammenfassung

Join a forward-thinking company as a Senior Data Engineer, where you will optimize and scale data processing pipelines using cutting-edge big data technologies. Collaborate with talented data scientists and engineers to extract valuable insights from vast mobility datasets. This role offers the chance to work on impactful projects that enhance public transport systems, making cities more sustainable and livable. Enjoy a collaborative work environment with opportunities for learning and development, competitive compensation, and flexible work arrangements. If you are passionate about leveraging data to drive change, this is the perfect opportunity for you.

Leistungen

Collaborative work environment

Opportunities for impactful projects

Competitive compensation

Flexible work arrangements

Learning and development opportunities

Qualifikationen

• Expertise in big data technologies and cloud platforms.
• Strong knowledge of data integration and orchestration.

Aufgaben

• Develop and optimize scalable data pipelines using Scala and Apache Spark.
• Deploy and manage workflows in Kubernetes and orchestrate ETL pipelines.

Kenntnisse

Scala

Apache Spark

Kubernetes

Hadoop

Apache Airflow

Machine Learning

Artificial Intelligence

Problem-solving

Tools

AWS

GCP

Azure

Jobbeschreibung

Teralytics provides an AI-powered co-pilot for public transport agencies, helping them design better transportation networks that make our cities more sustainable, livable, and equitable. We serve Germany's most important public transport and rail agencies, including Deutsche Bahn, RMV, and BVG, by delivering innovative solutions that optimize transport planning and management.

The Role

We seek a Senior Data Engineer to optimize and scale our data processing pipelines. You’ll work with big data technologies to extract insights from vast mobility datasets, collaborating with data scientists and engineers to build efficient and reliable solutions.

Responsibilities

• Develop and optimize scalable data pipelines using Scala and Apache Spark .
• Deploy and manage workflows in Kubernetes .
• Work with Hadoop and orchestrate ETL pipelines with Apache Airflow .
• Enhance data integration with data science methodologies.
• Utilize machine learning (ML) and artificial intelligence (AI) where applicable.
• Monitor, troubleshoot, and optimize data pipelines.

Requirements

• Expertise in Scala and Apache Spark .
• Experience with Kubernetes and Hadoop .
• Familiarity with Apache Airflow and workflow orchestration.
• Strong knowledge of big data technologies and cloud platforms (AWS, GCP, or Azure preferred).
• Understanding of data science principles and machine learning (ML) and AI is a plus.
• Problem-solving skills and ability to work in a fast-paced environment.

What We Offer

• A collaborative, innovative work environment.
• Opportunities to work on impactful, large-scale data projects.
• Competitive compensation and flexible work arrangements.
• Learning and development opportunities.

How to Apply Submit your resume and a short cover letter detailing your experience and interest to [emailprotected] . We look forward to hearing from you!

Senior Data Engineer • Berlin, Berlin, Germany