Software Data Engineer (AWS)

Lever, Inc.; Deutschland; Hybrid; EUR 27.000 - 32.000; Vollzeit

Benefits dieser Stelle

Restaurant meal/ticket benefits

Hybrid remote working model

Professional development opportunities

International environment

Zusammenfassung

Gomma Plastica seeks a Software Data Engineer in Germany to design and build modern data pipelines for global manufacturing operations.

You will work on streaming and batch pipelines, lakehouse architectures, and AWS infrastructure, collaborating with international teams to modernize legacy data solutions and enable data-driven decisions.

The role combines cloud, big data, data integration, and software engineering with a focus on production-ready solutions and strong observability.

Qualifikationen

• Master's degree in CS/EE or related field; PhD a plus.
• Strong Python and SQL programming and data engineering skills.
• Hands-on experience building streaming data pipelines with Kafka/MSK and Flink or Spark Streaming.
• Practical AWS experience (EKS, Fargate, Lambda, S3, Aurora/RDS PostgreSQL, DynamoDB, VPC).
• Knowledge of Spark, Hive, HDFS and lakehouse formats like Iceberg and Parquet.
• Experience with Git, CI/CD, Docker, Terraform or similar IaC, REST APIs, and Agile/Scrum.
• Understanding of scalable data architectures, production pipelines, data quality, monitoring, and ops.
• Knowledge of CDC, schema evolution, and Delta Lake is a plus.
• Strong communication and cross-functional collaboration across international teams.
• Curiosity for new tech and strong time-management.

Aufgaben

• Design and develop scalable streaming data pipelines using Kafka/MSK, Flink or Spark Streaming.
• Build batch transformations within a lakehouse architecture on Amazon S3 (Iceberg/Parquet).
• Design and manage storage with Aurora/RDS PostgreSQL, DynamoDB, and lakehouse storage.
• Design, deploy, and maintain AWS infrastructure for data and ML workloads (EKS, Fargate, Lambda, S3).
• Refactor, optimize, and deploy data solutions with CI/CD, containers, and IaC.
• Collaborate with international teams on requirements and data loading processes.
• Document developments, perform integration tests, and manage AWS deployments.
• Transition completed solutions to support and governance teams.
• Monitor pipelines for observability, data quality, performance, and reliability.
• Investigate issues and improve data ingestion in production environments.

Kenntnisse

Python

SQL

Kafka/MSK

Flink

Spark Streaming

AWS

Git

CI/CD

Docker

Terraform

REST APIs

Agile/Scrum

Data pipelines

Communication

Team collaboration

Learning mindset

Ausbildung

Master's degree in CS/CE

PhD a plus

Tools

Apache Iceberg

Parquet

AWS (EKS, Fargate, Lambda, S3, Aurora/RDS PostgreSQL, DynamoDB)

S3

DynamoDB

Kubernetes / EKS

Terraform

Docker

Jobbeschreibung

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Data Engineer (AWS) based in Germany.

As a Software Data Engineer, you will contribute to the development of a modern data platform supporting manufacturing operations across a global industrial environment.
You will design and build scalable streaming and batch data pipelines that connect plant systems and enable reliable, data-driven decision-making.
The role combines cloud engineering, big data, data integration, and software development, with a strong focus on AWS technologies.
You will work with Kafka, Flink or Spark Streaming, lakehouse architectures, and Infrastructure as Code to transform complex data into production-ready solutions.
You will take ownership of solutions from initial design and prototyping through deployment, optimization, monitoring, and ongoing maintenance.
Working closely with international teams, you will help modernize legacy data solutions and contribute to broader digital transformation initiatives.
This is an opportunity to work with advanced data technologies while having a direct impact on the efficiency, reliability, and scalability of critical manufacturing data systems.

Accountabilities

• Design and develop scalable streaming data pipelines using Kafka/MSK, Flink or Spark Streaming, and reliable routing into curated data storage.
• Build batch transformation pipelines within a medallion lakehouse architecture, supporting bronze, silver, and gold data layers on Amazon S3 and using formats such as Iceberg and Parquet.
• Design and manage data storage solutions across relational databases such as Aurora/RDS PostgreSQL, NoSQL technologies including DynamoDB, and object-based lakehouse storage.
• Design, deploy, and maintain AWS infrastructure supporting data and machine learning workloads, using services such as EKS, Fargate, Lambda, S3, VPC, Aurora/RDS, and DynamoDB.
• Take data solutions from prototype through to production by refactoring, optimizing, and applying software engineering, CI/CD, containerization, and deployment best practices.
• Collaborate with international teams to gather requirements, contribute to solution design, and define effective data loading and ingestion processes.
• Document technical developments, perform integration testing, and manage deployments within AWS environments.
• Support the transition of completed solutions to the relevant support and governance teams, ensuring reliable operation and clear ownership.
• Monitor production pipelines and ensure strong observability, data quality, performance, reliability, and maintainability.
• Participate in an application maintenance and support environment, investigating user-reported issues and implementing improvements to data ingestion processes.

Requirements:

• Master's degree in Computer Science, Computer Engineering, or a related technical discipline; a PhD is considered a plus.
• Strong programming and data engineering skills, particularly with Python and SQL.
• Hands-on experience building streaming data pipelines using Kafka/MSK and Flink or Spark Streaming.
• Practical experience with AWS services including EKS, Fargate, Lambda, S3, Aurora/RDS PostgreSQL, DynamoDB, and VPC.
• Solid understanding of big data technologies such as Spark, Hive, and HDFS, together with modern lakehouse formats such as Iceberg and Parquet.
• Experience with Git, CI/CD, Docker, Terraform or other Infrastructure as Code tools, REST APIs, and Agile/Scrum methodologies.
• Understanding of scalable data architectures, production data pipelines, data quality, monitoring, and operational best practices.
• Experience with Iceberg or Delta Lake, change data capture (CDC), schema evolution at scale, or Scala is considered an advantage.
• Strong communication skills and the ability to collaborate effectively with cross-functional and international teams.
• Curiosity and enthusiasm for learning new technologies, products, and technical capabilities.
• Strong attention to detail, organization, prioritization, and time-management skills.
• Ability to work independently while maintaining effective collaboration with technical and business stakeholders.

Benefits:

• Competitive compensation package aligned with experience, skills, and qualifications.
• Gross annual base salary starting from €29,500, with the possibility of a higher offer based on the candidate's profile.
• Employment contract governed by the applicable Gomma Plastica National Collective Bargaining Agreement.
• Welfare and employee benefits package, with further details provided during the recruitment process and subject to applicable policies.
• Restaurant meal/ticket benefits.
• Flexible working hours.
• Hybrid remote working model.
• Opportunities for professional development and career progression.
• Opportunity to work with advanced cloud, big data, AI, and digital transformation technologies in an international environment.
• Inclusive workplace committed to equal opportunity and non-discrimination.

We appreciate your interest and wish you the best!