Data Engineer

EY; Dadri, Kolkata District; Hybrid; INR 3,000,000 - 4,500,000; Full time

Job summary

EY GDS is seeking a Senior Data and AI Engineering professional to design, build and operate cloud-native data platforms and AI/ML solutions on Google Cloud, including Vertex AI and GenAI-enabled automation for enterprise data modernization.

You will collaborate across product, data and platform teams to implement scalable pipelines, manage CI/CD, and drive multi-cloud patterns including Azure while delivering production-grade AI/ML capabilities.

Qualifications

• Experience delivering production-grade AI/ML on Google Cloud.

Responsibilities

• Design, build and optimize scalable data pipelines using Python, Java, Go or Scala on Flink, Spark, and Dataflow.

Skills

Data engineering

Google Cloud

Vertex AI

ETL pipelines

Java

Python

Go

Scala

CI/CD pipelines

Docker/containers

Java Spring Boot

Azure

Kubernetes

Data platform design

GenAI integration

Tools

Apache Flink

Apache Spark

Google Cloud Dataflow

GitHub Actions

Jenkins

Azure DevOps

Docker

Job description

The Opportunity

EY GDS is looking for a Senior Data and AI Engineering professional to design, build and operate cloud-native data platforms and AI/ML solutions, primarily on Google Cloud. In this role, you will work on industry-led use cases that combine scalable data engineering, Vertex AI-based machine learning, GenAI-enabled automation and token-aware solution design to help clients modernize enterprise data platforms and accelerate insight generation.

You will collaborate with cross-functional teams to build robust pipelines, integrate GenAI capabilities into data workflows, optimize token usage and cost efficiency, and support production-grade deployments using modern engineering, containerization and CI/CD practices. While the primary focus will be on Google Cloud, candidates should also understand how similar data engineering, AI/ML and GenAI solution patterns can be deployed on Microsoft Azure, where required by client or enterprise architecture needs.

Your Key Responsibilities

• Design, build and optimize scalable data pipelines and ETL workflows using Python, Java, Go or Scala, and run them on Apache Flink, Apache Spark and Google Cloud Dataflow, with Dataflow experience preferred.
• Build cloud-native AI/ML solutions using Vertex AI for industry-specific use cases. Candidates should be able to describe the business use case they support, the data flow, model lifecycle and measurable outcome enabled by the solution.
• Integrate GenAI capabilities with data pipelines to enhance automation, insight generation, predictive analytics and intelligent data processing across enterprise platforms.
• Apply token economics principles while designing GenAI-enabled solutions, including prompt optimization, token usage monitoring, cost control, latency management and performance trade-off assessment.
• Stay current with emerging AI and GenAI technologies and identify practical ways to embed GenAI-driven intelligence into modern data platforms.
• Design reusable solution patterns for data engineering, AI/ML and GenAI workloads, with primary implementation on Google Cloud and the ability to adapt similar solutions for deployment on Microsoft Azure where required.
• Develop and maintain application components and automation scripts using Java, Python, Go and scripting languages.
• Manage the complete application and container lifecycle, including build, packaging, deployment, monitoring, troubleshooting and support.
• Apply strong public cloud engineering practices, with hands‑on expertise in Google Cloud. Professional or Practitioner certification on GCP is required.
• Demonstrate working knowledge of Microsoft Azure services to support deployment of similar data, AI/ML and GenAI solution architectures across client cloud environments.
• Set up and maintain CI/CD pipelines for application builds and target platform deployments.
• Follow standard procedures for escalating unresolved issues to the appropriate internal engineering teams, with clear problem statements and supporting diagnostics.
• Document technical knowledge, deployment steps, support notes and operating procedures in the form of reusable notes, runbooks and manuals.
• Good to have experience with Apache Beam and Java/Spring Boot for pipeline and application development.
• Responsible for decision‑making, optimizing processes, resource management, and overseeing team management as needed for task execution.
• Accountable for allocating personnel, supervising team members, assigning tasks, ensuring that the team has the necessary tools and support to succeed in their roles and optimizing and evaluating their performance to meet organizational goals.

Skills and attributes for success

• Strong hands‑on experience in data engineering, cloud-native engineering and production‑grade AI/ML solution delivery on Google Cloud.
• Practical experience with Vertex AI, including model development or integration, deployment patterns and use‑case‑driven implementation for enterprise or industry scenarios.
• Ability to build and optimize ETL/data pipelines using Python, Java, Go or Scala, and execute workloads on Spark, Flink and Google Cloud Dataflow.
• Working knowledge of GenAI integration patterns for automation, insight generation, predictive analytics and intelligent data workflows.
• Understanding of token economics for GenAI solutions, including prompt design efficiency, token consumption, latency, cost optimization, usage monitoring and governance considerations.
• Awareness of multi‑cloud deployment patterns, with the ability to translate similar data engineering, AI/ML and GenAI solution architectures from Google Cloud to Microsoft Azure where client environments require it.
• Basic to working knowledge of Microsoft Azure cloud services relevant to data platforms, AI/ML workloads, GenAI‑enabled applications and cloud‑native deployment.
• Strong programming skills in Java, Python, Go and scripting languages, with the ability to troubleshoot and optimize application and data‑processing components.
• Good understanding of application and container lifecycle management, including containerized deployment, operational readiness and supportability.
• Experience setting up CI/CD pipelines for build, validation and target platform deployment.
• Professional or Practitioner‑level Google Cloud certification, with demonstrated hands‑on public cloud delivery experience.
• Strong analytical thinking, problem‑solving ability and a structured approach to escalation, documentation and knowledge sharing.
• Clear communication skills and the ability to collaborate with product, data, platform and engineering teams in a delivery‑focused environment.
• Good to have familiarity with Apache Beam and Java/Spring Boot.

Preferred Qualifications

• Google Cloud certifications such as Professional Cloud Architect / Data Engineer
• Microsoft Certified: Azure Solutions Architect Expert (AZ‑305)
• Experience with CI/CD platforms such as GitHub Actions, Jenkins, or Azure DevOps
• Familiarity with AI‑assisted development tools such as GitHub Copilot or similar