Senior Software Engineer, Data Platform

Jobtailor; Deutschland; Hybrid; EUR 85.000 - 110.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a data engineer to lead the development of scalable Python data pipelines across streaming and batch workloads in a cloud environment. You will design data models and transformations to support advanced analytics and evolving BI needs, while building high-performance ETL/ELT workflows on AWS.

The role requires expert Python/SQL, strong AWS experience (DMS, S3, Kinesis, Glue), and a track record of mentoring engineers and maintaining data integrity.

Qualifikationen

• Experience designing and maintaining large-scale ETL/ELT pipelines.
• Hands-on with enterprise cloud data platforms (Redshift, BigQuery, Snowflake).
• Expert Python and SQL skills with optimization techniques for MySQL, Postgres, and SQL Server.
• Strong AWS experience: DMS, S3, Airflow, Kinesis, DocumentDB, CloudWatch, Kubernetes, Glue.
• GitHub version control and CI/CD for data infrastructure.
• Ability to mentor engineers and enforce data integrity.

Aufgaben

• Lead development and maintenance of scalable Python data pipelines.
• Design data models and transformation logic for analytics.
• Build ETL/ELT workflows on AWS with DMS, Kinesis, and related tools.
• Investigate pipeline issues and implement preventative measures.
• Mentor junior engineers and document processes.
• Own architecture of enterprise cloud data platforms.
• Drive CI/CD and IaC best practices.
• Leverage AI coding tools to boost reliability and performance.
• Collaborate to scale streaming and batch pipelines.

Kenntnisse

Python

SQL

AWS

ETL/ELT

Data Modeling

CI/CD

GitHub

Airflow

Kinesis

Tools

Redshift

Snowflake

BigQuery

DocumentDB

Kubernetes

Glue

Jobbeschreibung

• Lead development, testing, and maintenance of scalable Python code for data ingestion, transformation, and processing across streaming and batch architectures
• Design and optimize data models and transformation logic supporting advanced analytics and evolving BI needs
• Build and tune high-performance ETL/ELT workflows using AWS DMS, Kinesis, and related data integration technologies
• Diagnose and resolve pipeline issues and implement preventative measures to ensure data integrity and reliability
• Mentor junior engineers and establish documentation standards for code, models, and processes
• Own the architecture of enterprise cloud data platforms
• Drive CI/CD and infrastructure-as-code best practices
• Leverage AI-assisted tools to ensure system reliability, performance, and data integrity
• Collaborate in a quality-driven environment to architect and scale mission-critical streaming and batch pipelines

Requirements

• Experience in data engineering, demonstrating expertise in designing, conducting rigorous code reviews, and managing complex, large-scale ETL/ELT pipelines
• Deep understanding and hands-on experience with enterprise-grade cloud data platforms such as Redshift, BigQuery, or Snowflake, including architectural nuances and performance optimization strategies
• Expert-level skills in Python and SQL, including advanced optimization techniques in MySQL, Postgres, and SQL Server
• Extensive experience with complex transformations
• Strong command of AWS services: DMS, S3, Airflow, Kinesis, DocumentDB, CloudWatch, Kubernetes, and Glue
• Proven ability to establish and maintain robust version control workflows using GitHub
• Ability to architect and implement sophisticated CI/CD pipelines for data infrastructure
• Strong advocacy for infrastructure-as-code principles and tools
• Practical experience using AI coding tools such as Claude Code, Copilot, and Cursor
• Must reside in one of the listed states to be considered
• Must be authorized to work in the U.S.

Core Competencies

Demonstrates expertise in designing and optimizing scalable Python and SQL solutions for data ingestion and transformation, with a strong focus on cloud data platforms and CI/CD best practices. Proven ability to mentor junior engineers and ensure data integrity through robust ETL/ELT workflows and infrastructure-as-code principles.

Highest-signal resume keywords

• Python Programming
• SQL Optimization
• AWS Services (DMS, S3, Kinesis)
• ETL/ELT Pipeline Management
• Infrastructure-as-Code

ATS Optimization Keywords

Hard Skills

• Data Engineering
• ETL/ELT Workflows
• Data Modeling
• Complex Transformations
• Version Control (GitHub)
• AI Coding Tools
• Cloud Data Platforms (Redshift, BigQuery, Snowflake)
• CI/CD Pipeline Implementation
• Performance Optimization
• Data Integrity

Soft Skills

• Mentoring
• Collaboration
• Problem-Solving

Industry Keywords

• Data Ingestion
• Data Transformation
• Streaming Architectures
• Batch Architectures
• Business Intelligence

Tools & Technologies

• AWS DMS
• Kinesis
• Airflow
• DocumentDB
• CloudWatch
• Kubernetes
• Glue