Senior DevOps Engineer: Kafka, Spark & Data Pipelines (Remote)

DAC.digital Group; Województwo pomorskie; Hybrid; PLN 257,000 - 346,000; Full time

Benefits offered by this job

100% remote or on-site in Gdańsk

Job summary

DAC.digital Group seeks a data engineering expert to design and maintain Kafka-based streaming pipelines and optimize Apache Spark jobs for large-scale processing. The role includes deploying applications with Helm in Kubernetes, building robust ETL workflows, and advancing CI/CD practices.

You will operate within a hybrid infrastructure including bare metal, cloud VMs, and VPN-connected environments, contributing to scalable, reliable data solutions in a fast-paced setting.

Qualifications

• Minimum of 5 years of professional experience.
• Strong experience with Kafka.
• Solid expertise in Apache Spark.
• Hands-on experience with Helm.
• Proven ability to design and implement ETL processes for data integration and transformation.
• Strong experience in building and maintaining CI/CD pipelines.
• Knowledge of English (min. B2).
• Experience with Agile methodologies (Scrum, Kanban).
• High communication skills.
• Eager to learn and share knowledge.

Responsibilities

• Designing and maintaining Kafka-based data streaming pipelines for real-time data processing and integration.
• Developing and optimizing Apache Spark jobs for large-scale distributed data processing.
• Managing application deployments using Helm in Kubernetes-based environments.
• Building and maintaining robust ETL pipelines to ensure efficient and reliable data transformation and movement.
• Automating and improving CI/CD processes to streamline deployment cycles and increase delivery efficiency.
• Supporting the operation of a hybrid infrastructure including bare metal, cloud VMs, and VPN-connected environments.
• Contributing to the automation of deployment workflows across multiple parallel telco deployment projects.
• Ensuring high availability, scalability, and performance of data processing systems in production environments.

Skills

Kafka

Apache Spark

CI/CD pipelines

Agile (Scrum/Kanban)

English (B2)

Communication skills

Tools

Helm

Job description

DAC.digital Group seeks a data engineering expert to design and maintain Kafka-based streaming pipelines and optimize Apache Spark jobs for large-scale processing. The role includes deploying applications with Helm in Kubernetes, building robust ETL workflows, and advancing CI/CD practices.

You will operate within a hybrid infrastructure including bare metal, cloud VMs, and VPN-connected environments, contributing to scalable, reliable data solutions in a fast-paced setting.