Senior Data Engineer

ZipRecruiter; Berlin; Vor Ort; EUR 60.000 - 90.000; Vollzeit

Zusammenfassung

A leading company specializing in AI-powered solutions for public transport seeks a Senior Data Engineer in Berlin. The role involves optimizing scalable data pipelines, utilizing technologies such as Scala, Apache Spark, and Kubernetes. Candidates with a strong background in big data technologies, machine learning, and cloud platforms will thrive in this collaborative and innovative environment.

Leistungen

Collaborative work environment

Opportunities for impactful projects

Competitive compensation

Flexible work arrangements

Learning and development opportunities

Qualifikationen

• Expertise in Scala and Apache Spark required.
• Experience with Kubernetes and Hadoop.
• Familiarity with Apache Airflow and big data technologies.

Aufgaben

• Develop and optimize scalable data pipelines.
• Deploy workflows in Kubernetes and orchestrate ETL processes.
• Utilize machine learning and AI methodologies.

Kenntnisse

Scala

Apache Spark

Problem-solving

Data Science Principles

Machine Learning

Artificial Intelligence

Tools

Kubernetes

Hadoop

Apache Airflow

AWS

GCP

Azure

Jobbeschreibung

Job Description

About Us

Teralytics provides an AI-powered co-pilot for public transport agencies, helping them design better transportation networks that make our cities more sustainable, livable, and equitable. We serve Germany's most important public transport and rail agencies, including Deutsche Bahn, RMV, and BVG, by delivering innovative solutions that optimize transport planning and management.

The Role

We seek a Senior Data Engineer to optimize and scale our data processing pipelines. You’ll work with big data technologies to extract insights from vast mobility datasets, collaborating with data scientists and engineers to build efficient and reliable solutions.

Responsibilities

• Develop and optimize scalable data pipelines using Scala and Apache Spark.
• Deploy and manage workflows in Kubernetes.
• Work with Hadoop and orchestrate ETL pipelines with Apache Airflow.
• Enhance data integration with data science methodologies.
• Utilize machine learning (ML) and artificial intelligence (AI) where applicable.
• Monitor, troubleshoot, and optimize data pipelines.

Requirements

• Expertise in Scala and Apache Spark.
• Experience with Kubernetes and Hadoop.
• Familiarity with Apache Airflow and workflow orchestration.
• Strong knowledge of big data technologies and cloud platforms (AWS, GCP, or Azure ).
• Understanding of data science principles and machine learning (ML) and AI is a plus.
• Problem-solving skills and ability to work in a fast-paced environment.

What We Offer

• A collaborative, innovative work environment.
• Opportunities to work on impactful, large-scale data projects.
• Competitive compensation and flexible work arrangements.
• Learning and development opportunities.

How to Apply Submit your resume and a short cover letter detailing your experience and interest to [e-mail]. We look forward to hearing from you!