Sr Data Engineer, Data Eng & Governance

Dolby Laboratories; Bengaluru; On-site; INR 2,600,000 - 3,800,000; Full time

Job summary

Dolby Laboratories is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support data-driven initiatives across the organization.

You will work with large datasets and cutting-edge technologies, collaborating with AI researchers, data scientists, and product teams to deliver insights that shape our products and user experiences. 5+ years of related experience and a Master’s degree are expected.

Qualifications

• 5+ years of experience in data engineering and building large-scale pipelines.
• Proficient in Python, Scala, or Java.
• Distributed data processing with Spark/Beam.
• Hands-on with Data Warehousing/Lakehouse (Databricks, Snowflake, Redshift, BigQuery).
• Strong SQL and data modeling skills.
• Cloud platforms (AWS, GCP, Azure).
• Containerization and orchestration (Docker, Kubernetes) with CI/CD.
• Relational and NoSQL databases knowledge.
• Data modeling and architecture expertise.
• Strong problem solving and communication skills.
• Master’s degree required.

Responsibilities

• Build and optimize data infrastructure with cloud-based services and data pipelines.
• Design and implement scalable ETL/ELT processes for diverse data sources.
• Ensure data quality, metadata, and governance across assets.
• Collaborate with researchers, engineers, and product teams to translate needs into data solutions.
• Mentor junior data engineers and contribute to best practices and innovation.

Skills

Python

Scala

Java

Apache Spark

Apache Beam

Databricks

Snowflake

Redshift

BigQuery

AWS

GCP

Azure

Docker

Kubernetes

CI/CD

SQL

NoSQL

Education

Master's degree in Computer Science, Data Science, or related field

Tools

Python

Scala

Java

Apache Spark

Apache Beam

Databricks

Snowflake

Redshift

BigQuery

AWS

GCP

Azure

Docker

Kubernetes

CI/CD

SQL

NoSQL

Job description

Opportunity:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. In this role, you will be instrumental in designing, building, and maintaining robust and scalable data pipelines and infrastructure that power critical data-driven initiatives across our organization. You will work with vast datasets, cutting-edge technologies, and collaborate closely with AI researchers, other data engineers, data scientists, machine learning engineers, and product teams to deliver insights that shape the future of our products and user experiences.

Opportunity:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. In this role, you will be instrumental in designing, building, and maintaining robust and scalable data pipelines and infrastructure that power critical data-driven initiatives across our organization. You will work with vast datasets, cutting-edge technologies, and collaborate closely with AI researchers, other data engineers, data scientists, machine learning engineers, and product teams to deliver insights that shape the future of our products and user experiences.

What You'll Do:

Build and Optimize Data Infrastructure:

• Develop, construct, test, and maintain large-scale data ingest architecture consisting of diverse cloud-based services (messaging, storage, Kubernetes, persistent data store, serverless functions, etc).
• Create tooling like SDK, APIs to enable user self-service.
• Contribute to the design and evolution of our core data platform,ensuring its scalability, reliability, and cost-effectiveness.
• Implement robust monitoring, alerting, and logging solutions for data pipelines and infrastructure to proactively identify and resolve issues.

Design and Implement Scalable Data Pipelines:

• Design and implement highly reliable and efficient ETL/ELT processes to ingest, transform, and load data from diverse sources (e.g., real-time events, third-party APIs, rich media datasets) into our data lake and data warehouses.
• Utilize distributed data processing frameworks like Spark or similar to handle large scale data volumes with high throughput and low latency.

Ensure Data Quality and Governance:

• Describe and annotate datasets using industry standard schemas and internal specifications
• Cultivate data catalogs and metadata management solutions to improve data discoverability and understanding across the organization.
• Implement data validation, cleansing, and reconciliation processes to ensure the accuracy and integrity of our data assets.

Collaborate and Mentor:

• Work closely with stakeholders (research, engineering, product, and peers more broadly) to translate their data needs into robust data solutions.
• Provide technical leadership and mentorship to junior data engineers, fostering a culture of technical excellence and continuous learning.

Drive Innovation:

• Contribute to the evolution of our data architecture and engineering best practices.

What You'll Bring:

• Extensive Experience: 5+ years of experience in data engineering, with a strong focus on building and maintaining large-scale data pipelines and infrastructure.
• Programming Proficiency: Expert-level proficiency in at least one major programming language such as Python, Scala, or Java.- Distributed Data Processing: Deep experience with distributed data processing frameworks (e.g., Apache Spark, Apache Beam). Strong foundation in event-based approaches and systems including messaging/topics, pub/sub, queues, etc.
• Data Warehousing/Lakes: Hands-on experience with data warehousing solutions (e.g., Databricks, Snowflake, Redshift, BigQuery) and data lake technologies (e.g., S3, HDFS). Deep experience with managing large scale, heterogeneous datasets on Databricks is highly preferred.
• SQL Mastery: Advanced SQL skills for data manipulation, analysis, and optimization.
• Cloud Platforms: Strong experience with one or more major cloud providers (AWS, GCP, Azure) and their data-related services.
• Orchestration and DevOps: Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes). Proficient at CI/CD-based deployment.
• Database Knowledge: Solid understanding of relational and NoSQL databases.
• Data Modeling: Expertise in data modeling, schema design, and data architecture principles.
• Problem-Solving: Excellent analytical and problem-solving skills, with a track record of tackling complex data challenges.
• Communication: Strong communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams.
• Master's degree in Computer Science, Data Science, or a related quantitative field.

Bonus Points:

• Strong theoretical understanding of distributed computing concepts such as concurrency, parallelism, queueing, consistency, coordination protocols, etc.- Experience with machine learning pipelines and MLOps principles.
• Contributions to open-source data projects.