Sr. Big Data Engineer

Redolent Infotech Pvt. Ltd.; Sunnyvale (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Redolent Infotech Pvt. Ltd. in Sunnyvale, CA seeks a Sr. Big Data Engineer to design, build and operate petabyte-scale data systems, with a strong focus on real-time pipelines and ML infrastructure.

You will collaborate with engineers, product managers, BI developers and architects to deliver scalable, low-latency data solutions using Java and Python. 6–8 years of experience, Hadoop/Spark/Kafka and cloud tech (GCP/Azure) are required.

Qualifications

• Min 6-8 years of Big data development experience.
• Expertise in Data Engineering and complex data pipeline development.
• Proficiency in Java and Python for data processing.
• Experience with Hadoop, Hive, Kafka, Presto, Spark; strong SQL.

Responsibilities

• Design, develop, implement and tune large-scale distributed systems and pipelines for high data volumes with scalability, low latency and fault tolerance.
• Create and optimize real-time data pipelines and streaming analytics.
• Collaborate with engineers, product managers, BI developers and architects to deliver robust data solutions.

Skills

Big data development

Java

Python

SQL

Data pipelines

MapReduce

Hadoop

Hive

Kafka

Presto

Spark

Cloud: GCP

Cloud: Azure

NoSQL/Cassandra

Druid

RESTful APIs

Clickstream data

Data governance

Retail experience

Job description

We have following urgent role with our DIRECT client
Title:Sr. Big Data Engineer
Location: Sunnyvale, CA
Duration: 6+ Months
Rate: Market As part of the International eCommerce Data Engineering team, you'll be responsible for design, development and operations of large-scale data systems operating at petabytes scale. You will be focusing on real-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. You'll interact with the engineers, product managers, BI developers and architects to provide scalable robust technical solutions.

Essential Job Functions & Responsibilities :

• Min 6-8 years of Big data development experience
• Demonstrates up-to-date expertise in Data Engineering, complex data pipeline development
• Design, develop, implement and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low -latency, and fault-tolerance in every system built.
• Experience with Java, Python to write data pipelines and data processing layers
• Experience in writing map-reduce jobs
• Demonstrates expertise in writing complex, highly-optimized queries across large data sets
• Proven, working expertise with Big Data Technologies Hadoop, Hive, Kafka, Presto, Spark, just to name a few
• Highly Proficient in SQL
• Experience with Cloud Technologies ( GCP, Azure)
• Experience with Relational, NoSQL/in memory data stores would be a big plus ( Oracle, Cassandra, Druid)
• Provides and supports the implementation and operations of the data pipelines and analytical solutions
• Performance tuning experience of systems working with large data sets
• Experience with clickstream data processing
• Experience with Data Governance ( Data Quality, Metadata Management, Security, etc.)
• Experience in developing RESTful API data service
• Retail experience is huge plus