Staff Data Platform Engineer (AWS, Databricks, Azure, IaC, Solution Architect, Devops, Java/Python, 6 - 10 Years)

Visa; Bengaluru; Presencial; INR 3 000 000 - 5 000 000; Tempo integral

Resumo da oferta

Visa is seeking a Staff Cloud Platform Engineer to lead design, build, and operation of a scalable cloud data platform. You will work across AWS, Azure, and Databricks to migrate analytics and AI/ML workloads from Hadoop to modern cloud services.

You will collaborate with Architecture, Security, Product, and Engineering to deliver secure and reliable data pipelines, governance, and cost-efficient operations, while mentoring engineers and advancing platform best practices.

Qualificações

• 8+ years of experience in Cloud Data Engineering, Big Data Engineering, Platform Engineering, or Data Platform Engineering.
• Strong hands-on experience building and operating enterprise-scale cloud data platforms.
• Strong expertise in cloud data platform services across AWS and Azure, including AWS EMR, EMR on EKS, Glue, Athena, S3, IAM, Lake Formation, and Azure services such as ADLS, Azure Databricks, Synapse Analytics, Data Factory, Azure Kubernetes Service, and Azure identity and access management.
• Hands-on experience with Databricks, Delta Lake, Unity Catalog, and Lakehouse architecture patterns.
• Hands-on experience designing and operating data platform solutions on at least one major cloud platform, with preference for AWS, Azure, or both.
• Experience migrating workloads from Hadoop-based platforms to cloud environments.
• Strong proficiency in Python, Scala, or Java.
• Hands-on experience with Infrastructure-as-Code tools such as Terraform or CloudFormation.
• Strong expertise in Spark, Hive, Hadoop ecosystem technologies, and distributed data processing frameworks.
• Experience implementing automation, CI/CD, and platform engineering best practices.
• Strong understanding of data governance, security, compliance, metadata management, and access management controls.
• Strong Linux, operating system, networking, and troubleshooting skills.
• Excellent problem-solving, communication, collaboration, and stakeholder management skills.
• Experience working with architecture teams and translating enterprise architecture designs into scalable, production-ready implementations.

Responsabilidades

• Design, develop, and maintain scalable, secure, and reliable cloud-based data platform services across AWS, Azure, and Databricks.
• Collaborate closely with Architecture, Product, Security, and Engineering teams to implement enterprise-grade cloud data platform solutions.
• Build and enhance reusable platform capabilities, self-service onboarding solutions, and engineering frameworks that accelerate cloud adoption.
• Implement data integration pipelines and processing frameworks to ingest, transform, and manage large volumes of structured and unstructured data.
• Support migration and modernization of data, analytics, and AI/ML workloads from Hadoop-based platforms to AWS, Azure, and Databricks.
• Automate infrastructure provisioning, platform deployment, and operational processes using Infrastructure-as-Code (IaC) tools such as Terraform and CloudFormation.
• Implement and support data governance, metadata management, lineage, security, and access management capabilities across cloud platforms.
• Monitor and optimize platform performance, scalability, reliability, and cost efficiency across AWS, Azure, and Databricks environments.
• Partner with product and application teams to onboard workloads and enable adoption of cloud platform capabilities.
• Troubleshoot complex production issues and drive continuous improvement in platform reliability and operational excellence.
• Stay current with emerging cloud, data, and platform engineering technologies and recommend improvements where appropriate.
• Provide technical guidance and mentorship to engineers while promoting engineering best practices and operational rigor.

Conhecimentos

Python

Java

Scala

Spark

Kubernetes

AWS

Azure

Linux

Databricks

Delta Lake

Unity Catalog

Lakehouse

Ferramentas

Terraform

CloudFormation

Databricks

Delta Lake

Unity Catalog

Descrição da oferta de emprego

About Us Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid. At Visa, you'll have the opportunity to create impact at scale — tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world. Join Visa and do work that matters – to you, to your community, and to the world. Progress starts with you. Job Description Job Summary: We are seeking a highly skilled and experienced Staff Cloud Platform Engineer to join our dynamic Data Platform team. The successful candidate will be responsible for building, implementing, and operating cloud-based data platform capabilities that support Visa's large-scale migration of analytical and AI/ML workloads from open-source Hadoop platforms to AWS, Azure, and Databricks. This role requires strong expertise in cloud infrastructure, data engineering, platform engineering, automation, and data management best practices. The individual will work closely with Enterprise Architecture, Security, Product, and Engineering teams to implement enterprise-grade cloud data platform solutions that are scalable, secure, reliable, and operationally efficient.

Key Responsibilities

• Design, develop, and maintain scalable, secure, and reliable cloud-based data platform services across AWS, Azure, and Databricks.
• Collaborate closely with Architecture, Product, Security, and Engineering teams to implement enterprise-grade cloud data platform solutions.
• Build and enhance reusable platform capabilities, self-service onboarding solutions, and engineering frameworks that accelerate cloud adoption.
• Implement data integration pipelines and processing frameworks to ingest, transform, and manage large volumes of structured and unstructured data.
• Support migration and modernization of data, analytics, and AI/ML workloads from Hadoop-based platforms to AWS, Azure, and Databricks.
• Automate infrastructure provisioning, platform deployment, and operational processes using Infrastructure-as-Code (IaC) tools such as Terraform and CloudFormation.
• Implement and support data governance, metadata management, lineage, security, and access management capabilities across cloud platforms.
• Monitor and optimize platform performance, scalability, reliability, and cost efficiency across AWS, Azure, and Databricks environments.
• Partner with product and application teams to onboard workloads and enable adoption of cloud platform capabilities.
• Troubleshoot complex production issues and drive continuous improvement in platform reliability and operational excellence.
• Stay current with emerging cloud, data, and platform engineering technologies and recommend improvements where appropriate.
• Provide technical guidance and mentorship to engineers while promoting engineering best practices and operational rigor.

Basic Qualifications

• 8+ years of experience in Cloud Data Engineering, Big Data Engineering, Platform Engineering, or Data Platform Engineering.
• Strong hands-on experience building and operating enterprise-scale cloud data platforms.
• Strong expertise in cloud data platform services across AWS and Azure, including AWS EMR, EMR on EKS, Glue, Athena, S3, IAM, Lake Formation, and Azure services such as ADLS, Azure Databricks, Synapse Analytics, Data Factory, Azure Kubernetes Service, and Azure identity and access management.
• Hands-on experience with Databricks, Delta Lake, Unity Catalog, and Lakehouse architecture patterns.
• Hands-on experience designing and operating data platform solutions on at least one major cloud platform, with preference for AWS, Azure, or both.
• Experience migrating workloads from Hadoop-based platforms to cloud environments.
• Strong proficiency in Python, Scala, or Java.
• Hands-on experience with Infrastructure-as-Code tools such as Terraform or CloudFormation.
• Strong expertise in Spark, Hive, Hadoop ecosystem technologies, and distributed data processing frameworks.
• Experience implementing automation, CI/CD, and platform engineering best practices.
• Strong understanding of data governance, security, compliance, metadata management, and access management controls.
• Strong Linux, operating system, networking, and troubleshooting skills.
• Excellent problem-solving, communication, collaboration, and stakeholder management skills.
• Experience working with architecture teams and translating enterprise architecture designs into scalable, production-ready implementations.

Preferred Qualifications

• Experience building and operating multi-tenant enterprise data platforms.
• Experience with Kubernetes, EKS, containerized workloads, and modern platform engineering practices.
• Experience implementing observability, monitoring, logging, alerting, and reliability engineering solutions.
• Experience with cost governance and cloud optimization initiatives.
• Experience supporting AI/ML platforms and data science workloads.
• Experience mentoring engineers and driving engineering best practices across teams.
• Experience working on large-scale cloud transformation or modernization programs.

Visa is an EEO Employer Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.