AWS/Databricks Lead

Wipro; Mumbai; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Wipro Limited is seeking an experienced observability specialist to lead the monitoring of Databricks and Apache Spark workloads. You will implement dashboards, alerts, and cost-tracking to ensure high reliability and performance across environments.

The role requires 10–15 years of hands-on experience with Spark, Databricks, and cloud monitoring tools, along with strong SQL and PySpark skills. Join a global digital transformation team.

Qualifications

• 10–15 years of experience in observability and monitoring of Databricks and Spark workloads.
• Hands-on expertise with Databricks system tables, job orchestration, cluster management.
• Experience with data and platform observability frameworks and alerting.
• Proficiency in SQL and PySpark; dashboard development for operational insights.

Responsibilities

• Monitor Spark jobs, clusters, SQL warehouses, and data pipelines for performance and SLA breaches.
• Build dashboards and automated alerts using Databricks system tables for jobs, billing, audits, and query history.
• Troubleshoot, perform root cause analysis, and tune Spark workloads for performance.
• Ensure cost transparency across Databricks environments and optimize resource usage.
• Integrate Databricks monitoring with Azure Monitor, Log Analytics, or similar tools.
• Lead incident triage and provide guidance for permanent fixes.

Skills

Databricks

Apache Spark

SQL

PySpark

Dashboard creation

Tools

Azure Monitor

Log Analytics

Job description

Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. For additional information, visit us at www.wipro.com.

Job Description

Experience: 10 to 15 Years

Role Summary:

The primary responsibility of this role is to implement and manage comprehensive observability and monitoring for Databricks and Apache Spark workloads. The objective is to ensure high platform reliability, optimal performance, and transparent cost tracking.

Key Responsibilities

• Continuously monitor Spark jobs, clusters, SQL warehouses, and data pipelines to identify performance issues, failures, or breaches of service level agreements (SLAs).
• Develop intuitive dashboards and automated alerts by leveraging Databricks system tables, including those for jobs, clusters, billing, audits, and query history.
• Proactively address operational challenges through troubleshooting, root cause analysis, and performance tuning of Spark workloads.
• Facilitate cost, usage, and capacity transparency across Databricks environments, empowering teams to optimise resource allocation.
• Seamlessly integrate Databricks monitoring capabilities with enterprise-grade tools such as Azure Monitor, Log Analytics, or similar platforms.

Required Skills

• Demonstrated, hands-on expertise in working with Databricks and Apache Spark for both batch and streaming processes.
• In-depth understanding of Databricks system tables, job orchestration, cluster management, and workflow automation.
• Experience with data and platform observability frameworks, as well as advanced alerting and monitoring solutions.
• Proficiency in SQL, PySpark, and dashboard creation to deliver actionable operational insights.

Areas of Responsibility

• Triaging and resolution of tickets - Establish incident monitoring protocols and provide technical guidance to enable permanent fixes for recurring issues.
• SLA Monitoring - Review SLA performance across the team, represent the function in meetings, and ensure consistent delivery quality.
• Performance tuning and optimization - Collaborate with technical teams in managing production support activities regarding incident resolution, efficient deployments, and implementation of changes.
• Perform system Improvements - Collaborate with technical teams to review mitigation measures and proposed improvement actions.
• Handling disaster recovery - Develop disaster recovery and data backup plans to ensure system resilience and quick restoration in case of disruptions.
• Reporting and stakeholder engagement - Participate in Quarterly Business Reviews (QBRs), manage client expectations, and ensure alignment with business objectives.
• Team Management - Supervise team members by overseeing daily operations, conducting performance appraisals, and providing feedback.
• Knowledge base and continuous learning Update and maintain documentation for complex issues, share best practices within the team, and support ongoing knowledge enhancement to improve service efficiency.

Mandatory Skills

• Cloud Azure Admin.
• Experience: 8-10 Years.

Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.

We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, caste, creed, religion, gender, marital status, age, ethnic and national origin, gender identity, gender expression, sexual orientation, political orientation, disability status, protected veteran status, or any other characteristic protected by law.

Wipro is committed to creating an accessible, supportive, and inclusive workplace. Reasonable accommodation will be provided to all applicants including persons with disabilities, throughout the recruitment and selection process. Accommodations must be communicated in advance of the application, where possible, and will be reviewed on an individual basis. Wipro provides equal opportunities to all and values diversity.