Senior/Lead Big Data Engineer (Scala/Spark)

Grid Dynamics; Horsham; On-site; GBP 60,000 - 90,000; Full time

Job summary

A leading technology consulting firm is seeking a Senior/Lead Big Data Engineer to join their team in Horsham. This role involves developing Spark jobs, managing data pipelines, and optimizing databases. The position offers opportunities for professional development and the chance to work on innovative projects in a collaborative environment.

Benefits

Opportunity to work on cutting-edge projects

Collaboration with a motivated and dedicated team

Flexible schedule

Opportunities for professional development

Qualifications

• Strong expertise in Spark and Scala.
• Hands-on experience with Hadoop.
• Proficiency with data processing frameworks like Kafka and Spark.

Responsibilities

• Development of Spark jobs for data synchronization.
• Designing, developing, and maintaining RESTful services.
• Managing real-time data streams using Kafka.

Skills

Spark

Scala

Hadoop

Kafka

PostgreSQL

Job description

Join to apply for the Senior/Lead Big Data Engineer (Scala/Spark) role at Grid Dynamics.

Responsibilities

1. Development of Spark jobs for data synchronization between different storage systems.
2. Development of big data pipelines and infrastructure.
3. Planning and deploying data schemas for data warehousing.
4. Designing, developing, and maintaining RESTful services and microservices.
5. Managing real-time data streams using Kafka.
6. Writing efficient backend code in Java/Scala.
7. Optimizing Postgres databases and handling complex queries.
8. Creating automated build processes using Gradle.

Requirements

1. Strong expertise in Spark and Scala.
2. Hands-on experience with Hadoop.
3. Proficiency with data processing frameworks like Kafka and Spark.
4. Experience with database engines such as Oracle, PostgreSQL, Teradata, Cassandra.
5. Understanding of distributed computing technologies, approaches, and patterns.

Nice to have

• Experience with Data Lakes, Data Warehousing, or analytics systems.

We offer

• Opportunity to work on cutting-edge projects.
• Collaboration with a motivated and dedicated team.
• Flexible schedule.
• Opportunities for professional development.

About Us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Founded in 2006 and headquartered in Silicon Valley, we have offices across the Americas, Europe, and India. We focus on enterprise AI, data, analytics, cloud & DevOps, application modernization, and customer experience, enabling positive business outcomes for our clients.