Senior Data Engineer Apache Hadoop, Apache Spark, SQL, Kafka Gdańsk

Diverse CG Sp. z o.o. Sp.k.; Województwo pomorskie; On-site; PLN 50,000 - 70,000; Full time

Benefits offered by this job

Private medical care

Co-financing for the sports card

Constant support of dedicated consultant

Job summary

Diverse CG Sp. z o.o. Sp.k. is seeking an experienced Data Engineer in Poland to develop and maintain data pipelines supporting regulatory reporting. The candidate will design distributed data processing solutions using technologies like Hadoop and Spark.

Applicants should have strong SQL knowledge, experience with workflow orchestration tools, and good communication skills. The position includes perks such as private medical care and a sports card co-financing.

Qualifications

• Strong experience in data engineering and distributed data processing.
• Strong knowledge of SQL-based data transformation.
• Hands-on experience with Apache Hadoop, Spark, Oozie, Airflow, Kafka, and CI/CD tools.
• Ability to translate business requirements into SQL logic.
• Experience providing L3 production support.
• Strong analytical and problem-solving skills.
• Very good command of English.

Responsibilities

• Develop and maintain data pipelines for regulatory reporting.
• Design and implement distributed data processing solutions.
• Translate complex business requirements into SQL transformations.
• Manage and monitor workflow orchestration using Oozie and Airflow.
• Provide L3 support for production systems.
• Communicate technical issues clearly to stakeholders.

Skills

Data engineering

SQL-based data transformation

Apache Hadoop

Apache Spark

Oozie and Airflow

Kafka

CI/CD tools

Analytical skills

Communication skills

Tools

Apache Flink

Kubernetes

Job description

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:

Responsibilities

• Develop and maintain data pipelines supporting regulatory reporting for the initiative
• Design and implement distributed data processing solutions using Hadoop and Spark
• Translate complex business requirements into efficient and executable SQL transformations
• Build and maintain SQL‑based data transformations for regulatory data processing
• Manage and monitor workflow orchestration using Oozie and Airflow
• Work with event‑driven architectures, including Kafka
• Provide L3 support for production systems, including incident analysis and resolution
• Collaborate with Production Assurance teams to ensure system stability
• Communicate technical issues and SQL logic clearly to business stakeholders
• Contribute to CI/CD processes, including Git workflows, pull requests, and Jenkins pipelines
• Ensure high data quality, reliability, and compliance with regulatory requirements

Requirements

• Strong experience in data engineering and distributed data processing
• Strong knowledge of SQL‑based data transformation
• Hands‑on experience with Apache Hadoop, Apache Spark, Oozie, Apache Airflow, Kafka / event‑driven architectures and CI/CD tools (Git, PR workflows, Jenkins)
• Ability to translate business requirements into SQL logic
• Experience providing L3 production support
• Ability to collaborate with Production Assurance teams
• Strong analytical and problem‑solving skills
• Strong communication skills, including ability to explain SQL‑related issues to business stakeholders
• Very good command of English, both spoken and written

Nice to Have

• Experience with containerisation and orchestration (Kubernetes)
• Experience with Apache Flink

Offer

• Private medical care
• Co‑financing for the sports card
• Constant support of dedicated consultant