Senior Big Data Engineer: Spark, Scala & Data Pipelines

Grid Dynamics; Wrocław; On-site; PLN 80,000 - 120,000; Full time

Benefits offered by this job

Opportunity to work on bleeding-edge projects

Flexible schedule

Medical insurance

Corporate social events

Professional development opportunities

Job summary

Grid Dynamics is seeking a Senior Big Data Engineer in Wrocław, Poland, to design and optimize big data processing pipelines. You will collaborate with an international team, leveraging technologies like Scala, Apache Spark, and Kubernetes to build cutting-edge infrastructure.

The ideal candidate has strong software development experience in Scala, practical experience with Spark Streaming, and a good understanding of workflow orchestration with Apache Airflow. Competitive salary and benefits are offered.

Qualifications

• Strong software development experience with Scala and deep proficiency in Apache Spark.
• Practical experience with Spark Streaming for real-time data processing.
• Good understanding of Kubernetes and how Spark operates on it.
• Hands-on expertise with data lakehouse formats like Hive and Iceberg.
• Proven experience in workflow orchestration using Apache Airflow.

Responsibilities

• Design, develop, and optimize scalable big data processing pipelines.
• Implement and verify complex data jobs for reliable metrics.
• Work with cloud environments, leveraging Kubernetes for efficiency.
• Orchestrate complex workflows using Apache Airflow.
• Build robust data streaming architectures using Spark Streaming.

Skills

Scala

Apache Spark

Apache Airflow

Kubernetes

SQL

Hadoop

Spark Streaming

Tools

Apache Kafka

Hive

Iceberg

Oracle

Postgres

Teradata

Cassandra

Job description

Grid Dynamics is seeking a Senior Big Data Engineer in Wrocław, Poland, to design and optimize big data processing pipelines. You will collaborate with an international team, leveraging technologies like Scala, Apache Spark, and Kubernetes to build cutting-edge infrastructure.

The ideal candidate has strong software development experience in Scala, practical experience with Spark Streaming, and a good understanding of workflow orchestration with Apache Airflow. Competitive salary and benefits are offered.