Hadoop Lead

Ventures Unlimited Inc; Beaverton (OR); On-site; USD 120,000 - 150,000; Full time

Job summary

Ventures Unlimited Inc is seeking a Hadoop Lead with Python for a Full Time position in Beaverton, OR. The ideal candidate will have at least 8 years of experience in large-scale software development and strong expertise in Hadoop and related technologies.

Responsibilities include designing and implementing distributed processing solutions, building libraries, and researching new technologies. A master’s or bachelor’s degree in computer science or a related field is required.

Qualifications

• At least 8 years of experience in large-scale software development.
• Strong development skills around Hadoop, MapReduce, Hive, Pig, and Impala.
• Experience with performance and scalability tuning, algorithms, and computational complexity.

Responsibilities

• Design and implement map reduce jobs to support distributed processing.
• Build libraries and frameworks around Hadoop.
• Research and utilize new technologies around the Hadoop ecosystem.

Skills

Java programming

Python

SQL

Hadoop

ETL development

Data warehousing

MapReduce

Hive

Pig

Shell scripting

Statistical analysis

Education

MS/BS degree in a computer science field or related discipline

Tools

Maven

HBase

Cassandra

Kafka

Storm

Oracle

Job description

Ventures Unlimited is a service centric organization. Our success begins with our ability to apply our relationships, leverage technologies, and identify talent - often in combination - to an endless array of real-world customer needs.

Job Description

Position: Hadoop Lead with Python

Location: Beaverton, OR

Duration: Full Time

Responsibilities

• Design and implement map reduce jobs to support distributed processing using Java, Python, Hive and Pig; Design and implement end‑to‑end solutions.
• Build libraries, user defined functions, and frameworks around Hadoop.
• Research, evaluate and utilize new technologies/tools/frameworks around the Hadoop ecosystem.
• Develop user defined functions to provide custom Hive and Pig capabilities.
• Define and build data acquisition and consumption strategies.
• Define and develop best practices.
• Work with support teams to resolve operational and performance issues.
• Collaborate with architecture and engineering leads and other teams on capacity planning.
• Co‑operate with the Site‑Operations team on configuration and upgrades of the cluster.
• Maintain excellent communication skills.

Qualifications

• MS/BS degree in a computer science field or related discipline.
• At least 8 years of experience in large‑scale software development.
• Experience with data warehousing, dimensional modeling and ETL development.
• Strong Java programming, shell scripting, Python, and SQL skills.
• Strong development skills around Hadoop, MapReduce, Hive, Pig, and Impala.
• Strong understanding of Hadoop internals.
• Good understanding of AVRO and JSON.
• Experience with build tools such as Maven.
• Experience with databases like Oracle.
• Experience with performance and scalability tuning, algorithms, and computational complexity.
• Experience (or familiarity) with data warehousing, dimensional modeling and ETL development.
• Capability to understand ERDs and relational database schemas.
• Proven ability to work cross‑functional teams to deliver effective resolutions.
• Experience with open‑source NoSQL technologies such as HBase and Cassandra.
• Experience with messaging and complex event processing systems such as Kafka and Storm.
• Machine learning framework experience is a plus.
• Statistical analysis skills with Python, R, or similar.