Lead Data Engineer

Mastercard; Helena (MT); In loco; USD 120.000 - 180.000; Tempo pieno

Descrizione del lavoro

Mastercard is seeking a Lead Data Engineer to build and evolve large-scale backend data systems, real-time and batch pipelines, and AI-enabled services. You will work with Scala, Python, Java, and cloud platforms, focusing on distributed data architecture and data-driven automation.

You will design datamarts, enable analytics tools, and mentor peers while collaborating with cross-functional teams to deliver scalable data solutions at global scale.

Competenze

• Bachelor's degree in a technical field.
• Hands-on experience with Scala, Python/Java in backend or data-intensive systems.
• Experience with CDP and Spark; familiarity with cluster administration and monitoring.

Mansioni

• Design, develop, and maintain backend data pipelines and services.
• Build batch and streaming workloads on CDP using Spark; optimize performance.
• Create data products, datamarts, and curated datasets with strong data integrity.
• Collaborate across teams to enable analytics, visualization, and AI-driven workflows.

Conoscenze

Scala

Python/Java

Spark

CDP

Databricks

AWS

Data modeling

Distributed systems

AI/Automation

Formazione

BS/BA in Computer Science or related field

Strumenti

Cloudera Manager

Cloudera Data Platform (CDP)

Kafka

Hadoop/Hive/Impala

Descrizione del lavoro

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build asustainableeconomy where everyone can prosper. We support a wide range of digital payments choices, making transactionssecure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Lead Data Engineer

Position Overview

Have you ever wanted to be part of something BIG? Now is the time to make an immediate impact at a leading global technology company, Mastercard. This role is part of the AI&DPE Data Engineering platform team, responsible for building and evolving large scale backend data systems, real time and batch pipelines, and AI enabled services that power analytics, decisioning, and automation across the organization. This is a backend engineering role, focused on Scala/Python/Java, distributed data platforms (Cloudera/Spark), and cloud based architectures, with a strong emphasis on AI agent creation and intelligent automation. Visualization tools (e.g., Qlik) are consumers of the platform, not the core focus of this role. You will work with massive transactional datasets, modern big data and cloud platforms, and AI driven workflows to transform how Mastercard processes, enriches, and operationalizes data at global scale.

PRIMARY RESPONSIBILITIES

• Backend & Data Platform EngineeringDesign, develop, and maintain backend services and data pipelines using Scala, Python and JavaBuild and optimize batch and streaming workloads on Cloudera Data Platform (CDP) using SparkWork with Cloudera Manager to support platform configuration, monitoring, performance tuning, and operational stabilityDesign and implement high quality datamarts and curated datasets with strong emphasis on data integrity, performance, and reliability
• AI Agents & Intelligent AutomationDesign, build, and integrate AI powered agents that operate to support:Anomaly detection and operational intelligenceworkflow automationApply generative AI driven, or rule based agents to reduce manual effort and improve scalability across backend systems
• Cloud & Modern Data ArchitectureBuild and support data and compute workloads in AWS environmentsLeverage Databricks for large scale data processing, advanced analyticsContribute to cloud native and hybrid architectures integrating on prem and cloud platforms
• Integration & Downstream EnablementEnable downstream consumers (analytics, visualization, reporting tools such as Qlik) through well designed, reliable backend data interfacesPartner with analytics, fraud, and business teams to ensure backend systems meet evolving needs without compromising platform stability
• Documentation & CollaborationCreate clear technical documentation, including architecture diagrams, data flows, and design specificationsParticipate in Agile/Scrum ceremonies and cross functional design reviewsMentor and upskill team members in backend engineering, big data, and AI agent concepts

KNOWLEDGE AND SKILL REQUIREMENTS

• RequiredBS/BA degree in Computer Science, Engineering, Information Systems, or related fieldStrong handson experience with Scala,Python/Java in backend or data intensive systemsExperience working with Cloudera Data Platform (CDP) and SparkFamiliarity with Cloudera Manager for cluster administration, monitoring, or troubleshootingStrong understanding of data modeling concepts, distributed systems, and large scale data processingExcellent problem solving skills and ability to work independently in complex environments
• GOOD TO HAVE / STRONGLY PREFERREDHands on experience building, integrating, or supporting AI driven agents or intelligent automation solutionsExperience with Databricks for data engineering or ML workloadsExperience working in AWS (e.g., S3, EC2, EMR, Glue, Lambda, IAM, or equivalent services)Knowledge of streaming and big‑data technologies:KafkaHadoop ecosystemHive/ImpalaExposure to model monitoring, or AI platform enablementExperience with ETL tools such as InformaticaExperience working in Agile / Scrum teams within large enterprises

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:Abide by Mastercard’s security policies and practices;Ensure the confidentiality and integrity of the information being accessed;Report any suspected information security violation or breach, andComplete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.