Senior Big Data Engineer — Spark Architect, Flexible Hours

Grid Dynamics; Lisboa; Presencial; EUR 70 000 - 100 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Competitive salary

Flexible schedule

Professional development opportunities

Resumo da oferta

Grid Dynamics seeks a Senior Big Data Engineer to lead architecture and implementation of high-throughput data pipelines using Scala and Apache Spark in Lisbon. Translate business requirements into scalable data aggregations while upholding rigorous software engineering standards.

You will design data pipelines, optimize Spark performance, and collaborate with stakeholders to drive architecture direction, data marts, and analytics initiatives with a strong emphasis on testing and CI/CD.

Qualificações

• Proven experience designing and operating enterprise data pipelines using Scala and Apache Spark.
• Strong CS fundamentals, software design patterns, data structures, algorithms, and code testing practices.
• Deep operational understanding of distributed computing systems, parallel processing, and big data architecture patterns.
• Solid expertise in schema design, ETL/ELT optimization, partition strategy, and query performance tuning.

Responsabilidades

• Design, build, and maintain scalable, robust Big Data pipelines using Scala and Apache Spark to process large-volume batch and real-time datasets.
• Lead the design and implementation of complex big data aggregations, data marts, and performant data structures to support downstream analytics and AI applications.
• Write clean, modular, tested, and maintainable Scala code following best software engineering practices (CI/CD, code reviews, unit/integration testing).
• Monitor, tune, and optimize Spark job performance, memory management, and cluster resource utilization for high-throughput and low-latency processing.
• Collaborate with cross-functional teams to gather technical requirements, translate business needs into scalable architecture designs, and drive pipeline technical direction.

Conhecimentos

Scala

Apache Spark

Distributed computing

CI/CD

Unit testing

Data modeling

Data structures

Ferramentas

Snowflake

AWS

S3

EMR

Glue

Descrição da oferta de emprego

Grid Dynamics seeks a Senior Big Data Engineer to lead architecture and implementation of high-throughput data pipelines using Scala and Apache Spark in Lisbon. Translate business requirements into scalable data aggregations while upholding rigorous software engineering standards.

You will design data pipelines, optimize Spark performance, and collaborate with stakeholders to drive architecture direction, data marts, and analytics initiatives with a strong emphasis on testing and CI/CD.