Senior Big Data Engineer

Grid Dynamics; Lisboa; Presencial; EUR 70 000 - 100 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Competitive salary

Flexible schedule

Professional development opportunities

Resumo da oferta

Grid Dynamics seeks a Senior Big Data Engineer to lead architecture and implementation of high-throughput data pipelines using Scala and Apache Spark in Lisbon. Translate business requirements into scalable data aggregations while upholding rigorous software engineering standards.

You will design data pipelines, optimize Spark performance, and collaborate with stakeholders to drive architecture direction, data marts, and analytics initiatives with a strong emphasis on testing and CI/CD.

Qualificações

• Proven experience designing and operating enterprise data pipelines using Scala and Apache Spark.
• Strong CS fundamentals, software design patterns, data structures, algorithms, and code testing practices.
• Deep operational understanding of distributed computing systems, parallel processing, and big data architecture patterns.
• Solid expertise in schema design, ETL/ELT optimization, partition strategy, and query performance tuning.

Responsabilidades

• Design, build, and maintain scalable, robust Big Data pipelines using Scala and Apache Spark to process large-volume batch and real-time datasets.
• Lead the design and implementation of complex big data aggregations, data marts, and performant data structures to support downstream analytics and AI applications.
• Write clean, modular, tested, and maintainable Scala code following best software engineering practices (CI/CD, code reviews, unit/integration testing).
• Monitor, tune, and optimize Spark job performance, memory management, and cluster resource utilization for high-throughput and low-latency processing.
• Collaborate with cross-functional teams to gather technical requirements, translate business needs into scalable architecture designs, and drive pipeline technical direction.

Conhecimentos

Scala

Apache Spark

Distributed computing

CI/CD

Unit testing

Data modeling

Data structures

Ferramentas

Snowflake

AWS

S3

EMR

Glue

Descrição da oferta de emprego

We are looking for a Senior Big Data Engineer to join our team in Grid Dynamics. In this role, you will lead the architecture, design, and implementation of high-throughput data pipelines using Scala and Apache Spark. You'll work closely with stakeholders to translate business requirements into scalable big data aggregations while applying rigorous software engineering standards to our entire data ecosystem.

Working hours: Overlap until 11 am PST.

Responsibilities

• Design, build, and maintain scalable, robust Big Data pipelines using Scala and Apache Spark to process large-volume batch and real-time datasets.
• Lead the design and implementation of complex big data aggregations, data marts, and performant data structures to support downstream analytics and AI applications.
• Write clean, modular, tested, and maintainable Scala code following best software engineering practices (CI/CD, code reviews, unit/integration testing).
• Monitor, tune, and optimize Spark job performance, memory management, and cluster resource utilization for high-throughput and low-latency processing.
• Collaborate with cross-functional teams to gather technical requirements, translate business needs into scalable architecture designs, and drive pipeline technical direction.

Requirements

• Proven experience designing and operating enterprise data pipelines using Scala and Apache Spark.
• Strong foundation in computer science, software design patterns, data structures, algorithms, and code testing practices.
• Deep operational understanding of distributed computing systems, parallel processing, and big data architecture patterns.
• Solid expertise in schema design, ETL/ELT optimization, partition strategy, and query performance tuning.

Nice to have

• Hands-on experience with cloud platforms (AWS preferred, including services like S3, EMR, or Glue).
• Experience working with Snowflake for large-scale analytical workloads.

We offer

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Professional development opportunities

About Us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.