Big Data Tools Engineer - Software Dev Engineer I

Yahoo Holdings Inc. in; Mountain View (CA); Hybrid; USD 88,000 - 185,000; Full time

Job summary

An innovative firm is seeking a Big Data Tools Engineer to join their dynamic team. This role involves developing and optimizing cloud-native big data platforms, ensuring efficient data processing across vast Hadoop clusters. You will work with cutting-edge technologies, tackling complex challenges while collaborating with cross-functional teams. The position offers a unique opportunity to influence how users connect with data on a massive scale. If you're passionate about big data and eager to make a significant impact, this is the perfect opportunity for you.

Benefits

Healthcare

401k

Backup Childcare

Education Stipends

Qualifications

• Proficiency in Python and familiarity with Java and C++.
• Knowledge of data structures, algorithms, and database systems.

Responsibilities

• Oversee execution of data jobs and ensure SLAs are met.
• Design and optimize queries for high performance on big data platforms.

Skills

Python

Java

C++

SQL

Data Structures

Algorithms

Agile Methodologies

Debugging

Analytical Skills

Problem-Solving

Education

Degree in Computer Science

Tools

AWS

GCP

Airflow

BigQuery

Hadoop

Spark

Jenkins

Linux/Unix

Job description

Big Data Tools Engineer - Software Dev Engineer I (Finance)

It takes powerful technology to connect our brands and partners with an audience of hundreds of millions of people. Whether you're looking to write mobile app code, engineer the servers behind our massive ad tech stacks, or develop algorithms to help us process trillions of data points a day, what you do here will have a huge impact on our business—and the world.

Role Overview: We build, improve, and maintain one of the highest scaling platforms in the world. Our team of engineers works on next-generation Big Data Platforms that transform how users connect daily. Yahoo's Big Data Platform handles billions of requests daily on some of the largest Hadoop clusters, consisting of 50,000+ nodes, tackling complex problems across web services, operating systems, and networking layers. Our main challenge is designing efficient, cloud-native big data platforms.

Responsibilities:

1. Job Monitoring: Oversee the execution of data jobs, ensuring adherence to SLAs and issue resolution.
2. Data Orchestration: Use tools like Airflow for scheduling, executing, and monitoring data workflows across AWS and GCP.
3. Query Optimization: Design and optimize queries on platforms such as BigQuery, Hive, Pig, and Spark for high performance and scalability.
4. Integration and Support: Collaborate with teams to integrate data flows, support query execution, and manage secure data operations.
5. Feature Development: Develop features for advanced query capabilities, including federated queries and lineage tracking.

Minimum Qualifications:

• Degree in Computer Science or equivalent experience.
• Proficiency in Python; experience with Java and C++ preferred.
• Knowledge of data structures, algorithms, and database systems like SQL, HBase, and BigQuery.
• Experience with cloud services such as AWS (EMR, Glue, S3) and GCP (Dataproc, BigQuery).
• Comfortable working in Agile environments.
• Ability to design large-scale, distributed, highly available systems.
• Experience with Linux/Unix operating systems.

Preferred Qualifications:

• Experience with public cloud deployment (AWS, GCP, Azure).
• Experience developing containerized applications and orchestration.
• Knowledge of Hadoop, Presto, Hive, Oozie, Pig, Storm, Spark, Jupyter.
• Understanding of JVM internals and performance tuning.
• Strong debugging, testing, analytical, and problem-solving skills.
• Experience with CI tools like Jenkins and Hudson.
• Excellent communication skills for cross-functional collaboration.

Joining our Big Data Tools Team offers the opportunity to work on complex data systems, engage with cutting-edge technologies, and contribute to Yahoo's strategic projects.

Responsibilities also include adhering to company policies, exercising sound judgment, working safely and inclusively, and safeguarding operations and brand integrity.

We offer flexible hybrid work options, with occasional in-person events or sessions, notified in advance. For questions about work arrangements, contact your recruiter.

Yahoo is an equal opportunity employer. We consider all qualified applicants without regard to age, race, gender, or other protected categories. Accessibility accommodations are available upon request via our contact form or phone number provided.

We value diversity and inclusion, supported by our employee resource groups fostering a culture of belonging.

The salary range for this position is $88,500 - $184,375 annually, depending on location, skills, and experience. Benefits include healthcare, 401k, backup childcare, education stipends, and more.

If currently employed at Yahoo, please apply through our internal career site.