Senior Big Data Engineer - AWS EMR & Spark

Socket.dev; East Palo Alto (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Amazon Web Services (AWS) Elastic MapReduce (EMR) team is hiring experienced software engineers to design and build stable components for massive-scale data processing. You will own end-to-end software development for EMR clusters, including monitoring, reliability, and performance improvements in Hadoop/Spark ecosystems.

The role requires strong programming skills, system design and architecture experience, and collaboration with product managers and open-source communities.

Qualifications

• 3+ years of non-internship professional software development experience.
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• Experience programming with at least one software programming language.

Responsibilities

• Translation of complex functional and technical requirements into detailed architecture and design.
• Deliver systems and features with top-notch quality, on time.
• Develop new technologies for monitoring production clusters.
• Own the software development process end-to-end, including: working with engineers and product managers to develop requirements; designing, architecting, planning, implementing, and testing new systems and features; deploying, and operating the production EMR systems.

Skills

Software development

Design patterns

Distributed systems

Big data

Education

Bachelor's degree in Computer Science or equivalent

Tools

Hadoop

Spark

HBase

Hive

Job description

Amazon Web Services (AWS) Elastic MapReduce (EMR) team is hiring experienced software engineers to design and build stable components for massive-scale data processing. You will own end-to-end software development for EMR clusters, including monitoring, reliability, and performance improvements in Hadoop/Spark ecosystems.

The role requires strong programming skills, system design and architecture experience, and collaboration with product managers and open-source communities.