Lead Cloud Data Engineer — AI-Driven Analytics

Mastercard Inc.; Lisboa; Presencial; EUR 90 000 - 120 000; Tempo integral

Resumo da oferta

Mastercard is seeking a Lead Cloud Data Engineer to design and deliver cloud-native data platforms at scale, enabling AI-driven insights for global brands. You will lead data pipelines, orchestrate batch and streaming workloads, and guide modernization from Hadoop to cloud-native environments.

A strong background in Spark, Databricks, and Lakehouse concepts is essential. You will collaborate with architects, data scientists, and engineers to build secure, scalable, and cost-efficient data

Qualificações

• Hands-on experience with Big Data technologies including Hadoop, HDFS, Hive, and Spark.
• Proficiency in Python, Java, or Scala for large-scale data processing.
• Experience designing scalable data solutions and modern Lakehouse architectures.

Responsabilidades

• Design, develop, and support large-scale cloud-native data solutions on AWS and Databricks.
• Build batch, streaming, and real-time data pipelines for analytics, reporting, ML, and AI workloads.
• Engineer scalable distributed data processing systems using Apache Spark and related Big Data technologies.
• Design and implement modern Lakehouse architectures leveraging Databricks, Delta Lake, and Apache Iceberg.
• Lead modernization initiatives migrating data workloads from Hadoop-based platforms to cloud-native architectures.
• Develop reusable frameworks and engineering patterns to improve scalability, reliability, and operational efficiency.
• Collaborate with Architects, Product Managers, Data Scientists, and Software Engineers to define and implement cloud data solutions.
• Evaluate technologies and approaches for data ingestion, transformation, storage, governance, and consumption.
• Implement best practices around security, governance, performance optimization, observability, and cost management.
• Drive automation using Infrastructure-as-Code, CI/CD pipelines, and modern software engineering practices.

Conhecimentos

Big Data

Hadoop

HDFS

Hive

Spark

Python

Java

Scala

Data modeling

Schema design

Partitioning

Query optimization

Performance tuning

Security

Governance

Agile/Scrum

Communication

Formação académica

BS/MS degree in Computer Science, Software Engineering, Information Systems, or a related field

Ferramentas

Databricks

Spark

Delta Lake

Apache Iceberg

Hadoop

HDFS

Hive

Kafka

Flink

Terraform

Descrição da oferta de emprego

Mastercard is seeking a Lead Cloud Data Engineer to design and deliver cloud-native data platforms at scale, enabling AI-driven insights for global brands. You will lead data pipelines, orchestrate batch and streaming workloads, and guide modernization from Hadoop to cloud-native environments.

A strong background in Spark, Databricks, and Lakehouse concepts is essential. You will collaborate with architects, data scientists, and engineers to build secure, scalable, and cost-efficient data