Sr. Software engineer

SupportFinity™; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

A leading technology firm in Bangalore is looking for a Spark Engineer to design and manage large-scale distributed computing infrastructure. The successful candidate will collaborate with multiple teams to enhance Spark applications and contribute to open source initiatives. Ideal candidates should have 6+ years of experience with a solid understanding of Java and experience with distributed systems like Apache Spark. This position requires in-office work to foster collaboration and maintain company culture.

Qualifications

• 6+ years of experience building large scale distributed software systems.
• Solid understanding of Java for backend / systems software development.
• Experience managing production systems with strong availability SLA.

Responsibilities

• Work with Uber data science and engineering teams to improve Spark applications.
• Design and implement distributed algorithms for large scale batch data processing.
• Collaborate with partner teams to build cross-functional solutions.

Education

Bachelor’s degree in Computer Science or related field

MS / PhD in Computer Science or related field

Tools

Apache Spark

HDFS

Yarn

SQL Compiler

Job description

About the Role

Spark is our main batch big data analytics query engine at Uber. It accounts for the majority of our data compute resources, powers most of our major ingestion, ETL, data science, and machine learning workloads. Spark at Uber is used by over 2.5K weekly active users, who run 3M+ applications every week.

As part of the Spark team, you will design, implement, optimize, and manage large scale distributed computing infrastructure that will impact multiple business use cases at Uber scale. You will also have the opportunity to collaborate with the open source community for Spark and related domains.

What the Candidate Will Do

1. Work with Uber data science and engineering teams to improve Spark applications and use-cases at Uber.
2. Leverage knowledge of spark internals to dramatically help improve customer's Spark job performance.
3. Design and implement distributed algorithms for large scale batch data processing.
4. Work with multiple partner teams within and outside of Uber and build cross-functional solutions in a collaborative work environment.
5. Work on Spark internals so as to continue improving Spark's robustness and performance for Uber's unique use cases.
6. Work with the community to upstream Uber's contributions to open source and also keep our internal fork up to date

Basic Qualifications

1. Bachelor’s degree in Computer Science or related field.
2. 6+ years of experience building large scale distributed software systems.
3. Solid understanding of Java for backend / systems software development.

Preferred Qualifications

1. MS / PhD in Computer Science or related field.
2. Experience managing production systems with a strong availability SLA.
3. Experience working with Apache Spark or similar analytics technologies.
4. Experience working with large scale distributed systems, HDFS / Yarn.
5. Experience working with SQL Compiler, SQL Plan / Runtime Optimization.

Uber's mission is to reimagine the way the world moves for the better. Here, bold ideas create real-world impact, challenges drive growth, and speed fuelds progress. What moves us, moves the world - let’s move it forward, together.

Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.

*Accommodations may be available based on religious and/or medical conditions, or as required by applicable law. To request an accommodation, please reach out to [e-mail].