Sr Staff Engineer

Sanfoundry; Bengaluru; On-site; INR 4,000,000 - 6,400,000; Full time

Job summary

Sanfoundry seeks a Senior Staff Software Engineer, Data for the Search Discovery organization to design and implement durable software for massive data volumes. You will help optimize navigation and drive growth through ranking and recommendation at Coupang.

You will handle billions of records and petabytes of logs, building scalable data infrastructure and ML pipelines, while mentoring peers and collaborating across Data Scientists, Analysts and Product teams.

Qualifications

• 8+ years of experience in software/data engineering
• BS or MS degree in Computer Science or related field
• Expert with Big Data tools and ETL frameworks
• Experience with cloud platforms (AWS, GCP)
• Strong communication skills and ability to mentor engineers
• Experience with ML platforms for search discovery is a plus

Responsibilities

• Design and implement durable, efficient data solutions for massive data volumes
• Own data logging, processing, and ML model training pipelines
• Scale data infrastructure for batch and real-time processing
• Collaborate with Data Scientists, Analysts, PMs, and engineers
• Evaluate and select technical approaches for complex problems
• Coach and mentor other engineers on processes and methodologies

Skills

End-to-end data platform

Data logging pipelines

ML model training pipelines

Big Data tools

Cloud platforms (AWS, GCP)

Technical mentoring

Data visualization & analysis

Education

BS or MS in Computer Science

Tools

Hadoop

Hive

Presto

Spark

Scala

Apache Airflow

Job description

Role Overview:

As our Senior Staff Software Engineer, Data for Search Discovery organization, you will be in charge of designing and implementing durable efficient software solutions that handle massive volume of structured and unstructured data.

The Search Discovery organization is responsible for optimizing customers navigation experience and driving long-term growth. Every week more than 20 million customers use search in our app or on coupang.com, more than one third of them make purchases. Over half of the Coupang sales are directly attributed to search and recommendation. Many business initiatives and growth strategies are driven or boosted by ranking and recommendation.

This role is an opportunity to build a solid data foundation to support ranking and recommendation, and business decision making. You will be handling billions of records of user interaction data, and petabytes of raw log files, and transform them into data objects that can be easily consumed by engineers and scientists. There are so many things to analyze, to design, and to build. If you are passionate about big data engineering and technology, you want to make real business impact, and you are a true problem solver, this is the right job for you.

What You Will Do:

Basic Qualifications:

• Proactively drive the execution of end-to-end data platform, data quality roadmap for Search Discovery ML efforts


• Take ownership, consolidate and optimize the Search Discovery core data logging, processing, and ML model training pipelines


• Drive design and implementation of durable efficient platform solutions that handles massive amount of structured and unstructured data


• Develop and scale data infrastructure that powers batch and real-time data processing, monitoring, and debugging for ML experiments


• Build strong cross-functional partnerships with Data Scientists, Analysts, Product Managers, Indexing Platform Engineers, Ranking Engineers and Data Platform Engineers to understand data needs and deliver on those needs


• Research, analyze and select technical approaches for solving difficult and challenging development and integration problems, coach and mentor other engineers in process and methodologies


• BS or MS degree in Computer Science or related field, or equivalent practical experience


• 8+ years of experience


• Computer science fundamentals: data structures, algorithms, performance complexity, and implications of computer architecture on software performance (e.g., I/O and memory tuning)


• Software engineering fundamentals: version control systems (i.e Git, Github) and workflows, and the ability to write production-ready code


• Expert experience with Big Data tools and ETL frameworks (Hadoop, Hive, Presto, Spark, Scala, Apache Airflow, etc.)


• Knowledgeable of modern data visualization and exploration platforms such as Apache Superset, Tableau, etc.


• Experience deploying highly robust and scalable data pipelines processing petabytes of data


• Experience with integrating applications and platforms with cloud technologies (i.e, AWS and GCP)


• Strong verbal and written communication skills


• Ability to conduct meetings and make professional presentations, and explain complex concepts and technical material to non-technical users


• Experience with machine learning platform for search discovery related rankings is a strong plus