Senior Data Engineer - Hadoop, Spark, SQL & Kafka

Diverse CG Sp. z o.o. Sp.k.; Województwo pomorskie; On-site; PLN 50,000 - 70,000; Full time

Benefits offered by this job

Private medical care

Co-financing for the sports card

Constant support of dedicated consultant

Job summary

Diverse CG Sp. z o.o. Sp.k. is seeking an experienced Data Engineer in Poland to develop and maintain data pipelines supporting regulatory reporting. The candidate will design distributed data processing solutions using technologies like Hadoop and Spark.

Applicants should have strong SQL knowledge, experience with workflow orchestration tools, and good communication skills. The position includes perks such as private medical care and a sports card co-financing.

Qualifications

• Strong experience in data engineering and distributed data processing.
• Strong knowledge of SQL-based data transformation.
• Hands-on experience with Apache Hadoop, Spark, Oozie, Airflow, Kafka, and CI/CD tools.
• Ability to translate business requirements into SQL logic.
• Experience providing L3 production support.
• Strong analytical and problem-solving skills.
• Very good command of English.

Responsibilities

• Develop and maintain data pipelines for regulatory reporting.
• Design and implement distributed data processing solutions.
• Translate complex business requirements into SQL transformations.
• Manage and monitor workflow orchestration using Oozie and Airflow.
• Provide L3 support for production systems.
• Communicate technical issues clearly to stakeholders.

Skills

Data engineering

SQL-based data transformation

Apache Hadoop

Apache Spark

Oozie and Airflow

Kafka

CI/CD tools

Analytical skills

Communication skills

Tools

Apache Flink

Kubernetes

Job description

Diverse CG Sp. z o.o. Sp.k. is seeking an experienced Data Engineer in Poland to develop and maintain data pipelines supporting regulatory reporting. The candidate will design distributed data processing solutions using technologies like Hadoop and Spark.

Applicants should have strong SQL knowledge, experience with workflow orchestration tools, and good communication skills. The position includes perks such as private medical care and a sports card co-financing.