Senior Data Engineer

Proximity Works; Bengaluru; Presencial; INR 2.200.000 - 3.500.000; Jornada completa

Descripción de la vacante

Proximity Works is seeking a Senior Data Engineer to join our Data Platform team in Bengaluru. You will design, develop, and optimize large-scale data pipelines, leveraging Spark, Scala, and AWS services to deliver robust, production-grade data solutions.

You will work with cross-functional teams, design data workflows with Airflow, and ensure data quality, reliability, and performance across batch and streaming workloads.

Formación

• 5+ years of experience in Data Engineering or Big Data roles.
• Hands-on Spark and Scala development experience.
• Experience designing, building, and maintaining large-scale ETL pipelines.
• Strong AWS experience, especially S3.
• Hands-on Airflow for workflow orchestration and scheduling.
• Strong SQL skills with distributed data processing concepts.
• Experience with batch and streaming data pipelines.

Responsabilidades

• Design, develop, and maintain scalable ETL and data pipelines for large-scale datasets.
• Build and optimize distributed data applications using Spark and Scala.
• Develop reliable, high-performance data pipelines for batch and streaming workloads.
• Design and manage data workflows using Airflow.
• Build and operate data workloads on AWS, with strong S3 usage.
• Collaborate with engineering, product, analytics, and other platform teams.
• Troubleshoot production issues and optimize platform performance.

Conocimientos

Apache Spark

Scala

SQL

AWS

Airflow

Databricks

Kafka

Python

Herramientas

Databricks

Airflow

Kafka

Descripción del empleo

We are looking for Senior Data Engineers to join our Data Platform team and build scalable, high-performance data platforms that power data processing, analytics, and downstream applications.

The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Apache Spark and Python Scala.

You will be responsible for designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL and data processing pipelines for large-scale datasets
• Build and optimize distributed data applications using Apache Spark and Python Scala
• Develop reliable, high-performance data pipelines for batch and streaming workloads
• Design and manage data workflows using Apache Airflow
• Build and operate data workloads on AWS, with strong usage of Amazon S3 for large-scale data storage
• Work with large datasets to ensure data quality, consistency, reliability, and performance
• Collaborate with engineering, product, analytics, and other platform teams to deliver robust data solutions
• Optimize data workflows for scalability, reliability, performance, and cost efficiency
• Troubleshoot production issues, identify bottlenecks, and continuously improve platform performance

Requirements

• 5+ years of experience in Data Engineering, Big Data Engineering, or a similar role
• Strong hands-on experience with Apache Spark and Scala
• Experience designing, building, and maintaining large-scale ETL pipelines
• Strong hands-on experience with AWS, particularly Amazon S3
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling
• Strong SQL skills and a solid understanding of distributed data processing concepts
• Experience working with batch and/or streaming data pipelines
• Excellent debugging, problem-solving, and performance optimization skills
• Strong communication and collaboration skills

Good to Have

• Experience with Databricks and the broader Databricks data platform
• Familiarity with streaming technologies such as Apache Kafka
• Experience working on large-scale data platforms handling high-volume data workloads
• Exposure to additional AWS data services and cloud-native data architectures

Benefits

• Best-in-class salary: We hire strong talent and compensate accordingly
• Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners
• Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development
• High-impact work: Build AI-first systems and products used at scale by global clients

About Us

Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We\'re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.

Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.