Senior Associate | Data Engineering | Bengaluru | Engineering as a Service/ Operate

Deloitte & Touche GmbH Wirtschaftsprüfungsgesellschaft; Bengaluru; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

Deloitte & Touche GmbH Wirtschaftsprüfungsgesellschaft in Bengaluru seeks a Senior Associate Data Engineer to help Disney Entertainment be more data-driven by monitoring and improving critical data pipelines. You will collaborate with partner teams, design scalable data models, and contribute to reliability and observability in a fast-paced environment.

The role requires 5+ years in data engineering, hands-on experience with AWS, Hadoop, Hive, Snowflake, and proficiency in Python/Scala/Java/SQL.

Qualifications

• 5+ years experience working on mission critical data pipelines and ETL systems.
• 5+ years of hands-on experience with big data technology, systems and tools such as AWS, Hadoop, Hive, and Snowflake
• Detailed problem-solving approach, coupled with a strong sense of ownership and drive
• A passionate bias to action and passion for delivering high-quality data solutions
• Expertise with common Data Engineering languages such as Python, Scala, Java, SQL and a proven ability to learn new programming languages
• Experience with workflow orchestration tools such as Airflow

Responsibilities

• Work and mentor all members of the Data Reliability Engineering team to manage production incidents, triage and root cause issues and ensure timely resolution via Tier 1 and Tier 2 support.
• Assist in designing and developing a platform to support incident observability and automation.
• Collaborate with engineering teams to improve, maintain, performance tune, and respond to incidents on our big data pipeline infrastructure.
• Build out observability and intelligent monitoring of data pipelines and infrastructure to achieve early and automated anomaly detection and alerting.

Skills

Python

Scala

Java

SQL

Airflow

Problem solving

Education

Degree in Computer Science/Engineering

Tools

Docker

Kubernetes

Hadoop

Hive

Snowflake

Airflow

Job description

Job Title: Senior Associate | Data Engineering | Bengaluru | Engineering as a Service/ Operate

Job Summary

The Senior Data Engineer will help us in the ongoing mission of delivering outstanding services to our usersallowing Disney Entertainment to be more data-driven. You will work closely with our partner teams to monitorand drive improvements for reliability and observability of their critical data pipelines and deliverables. You will be making an outsized impact in an organization that values data as itstop priority. We are a tight and driven team with big goals, so we seek people who are passionate about solvingthe toughest challenges and working at scale, using, supporting, and building distributed systems in a fast-pacedcollaborative team environment. The candidate will be required to work on on-call shifts that will also includeweekend shifts on rotation. We also support a healthy work-life-balance.

Responsibilities and Duties of the Role

• Work and mentor all members of the Data Reliability Engineering team to manageproduction incidents, triage and root cause issues and ensure timely resolution via Tier 1and Tier 2 support. Offer recommendations via RCAs to prevent future incidents toengineering partners.
• Assist in designing and developing a platform to support incident observability andautomation. This team will be required to build high quality data models and products thatmonitor and reports on data pipeline health and data quality.
• Collaborate with engineering teams to improve, maintain, performance tune, and respondto incidents on our big data pipeline infrastructure.
• Build out observability and intelligent monitoring of data pipelines and infrastructure toachieve early and automated anomaly detection and alerting. Present your research andinsights to all levels of the company, clearly and concisely.

Required Education, Experience/Skills/Training

• 5+ years experience working on mission critical data pipelines and ETL systems.
• 5+ years of hands-on experience with big data technology, systems and tools such as AWS, Hadoop,Hive, and Snowflake
• Detailed problem-solving approach, coupled with a strong sense of ownership and drive
• A passionate bias to action and passion for delivering high-quality data solutions
• Expertise with common Data Engineering languages such as Python, Scala, Java, SQL and a provenability to learn new programming languages
• Experience with workflow orchestration tools such as Airflow

JOB DESCRIPTION

• Deep understanding of end-to-end pipeline design and implementation.
• Attention to detail and quality with excellent problem solving and interpersonal skills

Preferred Qualifications

• Advanced degrees are a plus.
• Strong data visualizations skills to convey information and results clearly
• Ability to work independently and drive your own projects.
• Exceptional interpersonal and communication skills.
• Impactful presentation skills in front of a large and diverse audience.
• Experience with DevOps tools such as Docker, Kubernetes, Jenkins, etc.
• Innate curiosity about consumer behavior and technology
• Experience with event messaging frameworks like Apache Kafka
• A fan of movies and television is a strong plus.

Required Education

• Degree in Computer Science, Engineering, or related field.