Big Data Tools Engineer - Software Dev Engineer I

Yahoo Holdings Inc. in; Great Falls Crossing (VA); Hybrid; USD 80,000 - 120,000; Full time

Job summary

Join a forward-thinking company as a Big Data Tools Engineer, where you will work with cutting-edge technologies on one of the largest Hadoop clusters. This role involves monitoring data jobs, optimizing queries, and collaborating with teams to integrate data flows. You will have the opportunity to develop advanced features for cloud-native big data platforms, shaping the future of data processing. Enjoy a flexible hybrid work environment, comprehensive benefits, and a culture that values diversity and innovation. If you're passionate about big data and eager to make an impact, this is the perfect opportunity for you!

Benefits

Flexible Hybrid Work Options

Comprehensive Benefits

Supportive Environment

Diversity and Inclusion Programs

Qualifications

• Bachelor's or Master's degree in Computer Science or equivalent experience.
• Proficiency in Python; experience with Java and C++ is preferred.
• Knowledge of SQL, HBase, and BigQuery.

Responsibilities

• Oversee execution of data jobs, ensuring adherence to SLAs.
• Use tools like Airflow to manage workflows across cloud platforms.
• Design and optimize queries on platforms like BigQuery and Spark.

Skills

Python

Java

C++

Data Structures

Algorithms

SQL

Cloud Services (AWS, GCP)

Agile Methodologies

Debugging

Analytical Skills

Education

Bachelor's Degree in Computer Science

Master's Degree in Computer Science

Tools

AWS (EMR, Glue, S3)

GCP (Dataproc, BigQuery)

Hadoop

Jenkins

Docker

Job description

Big Data Tools Engineer - Software Dev Engineer I (Finance)

Yahoo's Big Data Platform handles billions of requests daily on one of the largest Hadoop clusters, supporting demanding applications across web services, operating systems, and networking layers. We seek a talented engineer to join our team, focusing on next-generation cloud-native big data platforms that will shape the future of data processing and analysis.

Responsibilities:

1. Job Monitoring: Oversee execution of data jobs, ensuring adherence to SLAs and troubleshooting issues.
2. Data Orchestration: Use tools like Airflow to manage workflows across cloud platforms such as AWS and GCP.
3. Query Optimization: Design and optimize queries on platforms like BigQuery, Hive, Pig, and Spark for high performance.
4. Integration and Support: Collaborate with teams to integrate data flows and manage secure data operations.
5. Feature Development: Develop features for advanced query capabilities, including federated queries and lineage tracking.

Required Skills and Qualifications:

1. Bachelor's or Master's degree in Computer Science or equivalent experience.
2. Proficiency in Python; experience with Java and C++ is preferred.
3. Knowledge of data structures, algorithms, and database systems like SQL, HBase, and BigQuery.
4. Experience with cloud services such as AWS (EMR, Glue, S3) and GCP (Dataproc, BigQuery).
5. Comfortable working in an Agile environment.
6. Ability to design large-scale, distributed, and resilient systems.
7. Experience with Linux/Unix operating systems.

Preferred Qualifications:

1. Experience with cloud deployment and container orchestration.
2. Familiarity with Hadoop, Presto, Hive, Oozie, Pig, Storm, Spark, Jupyter.
3. Understanding of JVM internals and performance tuning.
4. Strong debugging, testing, and analytical skills.
5. Experience with CI/CD tools like Jenkins and Hudson.
6. Excellent communication skills for cross-functional collaboration.

Join Yahoo's Big Data Tools Team to work on complex data systems, engage with cutting-edge technologies, and contribute to impactful projects that drive our business forward. We value diversity and inclusion, offering flexible hybrid work options, comprehensive benefits, and a supportive environment that fosters innovation and growth.