Senior Cloud Administrator (AWS Data Engineer)

ZS; Pune District; Hybrid; INR 3,500,000 - 5,000,000; Full time

Job summary

ZS in Pune seeks a Senior Cloud Administrator / Data Engineer to design, build and optimize scalable data pipelines using Spark, Hadoop, SQL and Python. You will partner with analytics and cloud teams to run AWS-based data platforms securely, cost-effectively, and at scale.

Responsibilities include tuning jobs, troubleshooting production issues, automating operations, and contributing to AI-enabled workflows with LLMs, Bedrock and MCP concepts while maintaining data governance and quality across

Qualifications

• 4+ years of hands-on data engineering with Spark, Hadoop and data warehouses.
• Hands-on Spark/PySpark development and distributed processing.
• Advanced SQL with joins, window functions and data validation.
• Strong Python for data processing, automation and tooling.
• Linux experience, shells, logs and troubleshooting.

Responsibilities

• Design, develop, maintain and support robust data pipelines ingesting, transforming and delivering data for analytics and ML use cases.
• Write production-grade PySpark/SQL code for high-volume transformations.
• Optimize Spark and SQL workloads for performance, scalability and cost.
• Build reusable ingestion components and automation frameworks.
• Monitor pipelines, investigate failures and improve observability and runbooks.
• Troubleshoot data-processing layers and supporting cloud infrastructure.
• Perform root-cause analysis for incidents and implement durable fixes.
• Apply data governance, quality and security standards across the lifecycle.
• Collaborate with data producers/consumers, architects and cloudOps to translate requirements.
• Contribute to design reviews, release validation and production-readiness checks.
• Support PoCs involving LLMs, Bedrock, Agentic AI and MCP-based integrations.

Skills

Spark

Hadoop

SQL

Python

Linux

Data warehousing

Cloud fundamentals

AWS

Data pipelines

Troubleshooting

Education

AWS certification (Associate or higher)

Tools

Amazon EMR

AWS Glue

Amazon Athena

Amazon SageMaker

Amazon Bedrock

Job description

What You'll Do

As a Senior Cloud Administrator at ZS you will combine strong data engineering skills with practical cloud infrastructure knowledge. The role is primarily responsible for designing, developing, supporting, and optimizing scalable data pipelines and distributed data-processing workloads. The successful candidate will work across Spark, Hadoop, SQL, Python and Linux, while supporting analytics services and the underlying AWS environment required to run them reliably, securely and cost-effectively. This is a hands‑on role for a data engineer who can own complex transformations, tune jobs, troubleshoot production issues, automate recurring operational work, and collaborate with product, engineering, operations and architecture teams. The role also contributes to emerging AI‑enabled data workflows through an understanding of Large Language Models, Amazon Bedrock, Agentic AI and Model Context Protocol (MCP).

• Design, develop, maintain and support robust data pipelines that ingest, transform and deliver data for analytics, reporting, machine learning and AI use cases.
• Write and review production‑grade Python, PySpark and SQL for complex, high-volume data transformations.
• Optimize Spark and SQL workloads for performance, scalability and cost by analyzing execution plans, partitioning, skew, caching, memory pressure and I/O patterns.
• Build reusable ingestion and transformation components, common utilities and automation frameworks that reduce manual intervention and improve delivery consistency.
• Monitor scheduled pipelines and distributed jobs, investigate failures, restore services, and improve observability, alerting, and operational runbooks.
• Perform hands‑on troubleshooting across the data‑processing layer and supporting cloud infrastructure, including Linux hosts, storage, access, networking and service configuration.
• Conduct root cause analysis for recurring or high‑impact incidents and implement long‑term remediation to prevent recurrence.
• Apply data‑engineering standards for reliability, maintainability, data quality, security and governance throughout the development lifecycle.
• Partner with data producers, consumers, architects, product teams and cloud operations teams to translate requirements into practical technical solutions.
• Contribute to technical design reviews, code reviews, release validation, capacity planning and production‑readiness assessments.
• Support proofs of concept involving LLMs, Amazon Bedrock, Agentic AI and MCP‑based integrations where they intersect with enterprise data platforms.
• Work with AWS or other technology vendors and subject matter experts for advanced troubleshooting when required.

What You’ll Bring

• 4+ years of hands‑on experience in data engineering, including Apache Spark, Hadoop ecosystems and Data Warehousing.
• Strong expertise in designing, building, testing, and supporting scalable batch or ETL/ELT data pipelines.
• Hands‑on expertise in Spark or PySpark development, including distributed processing concepts and job optimization.
• Advanced SQL skills, including complex joins, window functions, aggregations, query tuning and data validation.
• Strong Python skills for data processing, reusable frameworks, automation and operational tooling.
• In‑depth hands‑on experience with Linux, shell utilities, logs, processes, file systems and troubleshooting.
• Proven ability to implement complex data transformations, including cleansing, deduplication, enrichment, aggregation and slowly changing dimension patterns.
• Sound understanding of data warehousing, data lakes, partitioning, file formats, schema evolution and data quality controls.
• Experience diagnosing and optimizing pipeline performance, including parallelism, partition sizing, data skew, memory usage, shuffle behavior and inefficient SQL patterns.
• Ability to troubleshoot complex production issues, perform root cause analysis and implement sustainable corrective actions.
• Understanding of cloud infrastructure fundamentals relevant to analytics workloads, including compute, storage, identity and access, networking, monitoring and cost management.
• Understanding of Large Language Models, Amazon Bedrock, Agentic AI patterns and Model Context Protocol (MCP) concepts.
• Ability to collaborate with cross‑functional and globally distributed teams and communicate technical findings clearly to stakeholders.
• Hands‑on experience with AWS analytics services such as Amazon EMR, AWS Glue, Amazon Athena, Amazon SageMaker, etc.
• Experience with Amazon S3‑based data lakes and related security, cataloging, partitioning and performance
  considerations.
• Exposure to workflow orchestration or scheduling tools and CI/CD practices for data pipelines.
• Familiarity with infrastructure‑as‑code or cloud automation tools used to provision and manage analytics
  environments.
• Knowledge of streaming data, open table formats, data governance, or metadata and catalog solutions.
• Experience supporting secure, regulated or mission‑critical analytics platforms.
• Associate‑level or higher AWS certification, or relevant data engineering certification.
• Fluency in English.
• Client‑first mentality.
• Intense work ethic.
• Collaborative spirit and problem‑solving approach.

How you’ll grow

• Cross‑functional skills development & custom learning pathways
• Milestone training programs aligned to career progression opportunities
• Internal mobility paths that empower growth via s‑curves, individual contribution and role expansions

Perks & Benefits

At ZS, your growth matters. We offer a comprehensive total rewards package that supports your health and well‑being, financial future, time away, and professional development. With robust skills‑building programs, multiple career progression paths, internal mobility, and a deeply collaborative culture, you’ll have the opportunity to do meaningful work, expand your capabilities, and thrive as part of a global community. For details on total rewards in India, visit ZS India office locations | Where we work | ZS.

Hybrid working model

We are committed to giving our employees a flexible and connected way of working. A flexible and connected ZS allows us to combine work from home and on‑site presence at clients/ZS offices for the majority of our week. The magic of ZS culture and innovation thrives in both planned and spontaneous face‑to‑face connections.

Travel

Travel is a requirement at ZS for client facing ZSers; business needs of your project and client are the priority. While some projects may be local, all client‑facing ZSers should be prepared to travel as needed. Travel provides opportunities to strengthen client relationships, gain diverse experiences, and enhance professional growth by working in different environments and cultures.

Considering applying?

At ZS, we honor the visible and invisible elements of our identities, personal experiences, and belief systems—the ones that comprise us as individuals, shape who we are, and make us unique. We believe your personal interests, identities, and desire to learn are integral to your success here. We are committed to building a team that reflects a broad variety of backgrounds, perspectives, and experiences. Learn more about our inclusion and belonging efforts and the networks ZS supports to assist our ZSers in cultivating community spaces and obtaining the resources they need to thrive.

ZS is an equal opportunity employer and is committed to providing equal employment and advancement opportunities without regard to any class protected by applicable law.