Staff/Lead Engineer

Eightfold; California; On-site; USD 150,000 - 190,000; Full time

Job summary

Eightfold is seeking a Lead Engineer for the Data Platform team to architect and build large-scale data infrastructure powering AI and Talent Intelligence.

You will own the design of real-time and batch data pipelines, drive reliability and observability, and partner with AI/ML and product teams to enable fast, reliable data access. A high-impact leadership role with global reach and hybrid work expectations.

Qualifications

• 7+ years of experience in distributed systems engineering.
• Strong programming skills in Java, Scala, Python, or Go.
• Experience with Spark, Kafka, Airflow, Flink, or Snowflake.
• Strong understanding of distributed systems, cloud infrastructure, and large-scale data processing.
• Experience operating production systems on AWS or similar cloud platforms.

Responsibilities

• Lead the design and development of scalable data platform services and distributed data systems.
• Build and optimize batch and real-time data pipelines and platform tooling.
• Drive best practices around reliability, observability, CI/CD, and infrastructure automation.
• Partner with AI/ML, analytics, and product engineering teams to enable efficient and reliable data access.
• Mentor engineers and contribute to long-term platform architecture and technical strategy.

Skills

Distributed systems

Programming: Java/Scala/Python/Go

Spark

Kafka

Airflow

Flink

Snowflake

AWS

Tools

AWS

Job description

Eightfold was founded with a vision to solve for employment in our society. For decades, the connection between individuals and opportunities has been based on who the individuals are and the strength of their network, vs. their potential. Eightfold leverages artificial intelligence to transform how to think about skills and capabilities for individuals as well as how jobs and career decisions are made. Eightfold offers the industry's first AI-powered Talent Intelligence Platform to transform how organizations plan, hire, develop and retain a diverse workforce, enabling individuals to transform their careers. To date, Eightfold AI has received more than $410 million in funding and a valuation of over $2B from leading investors to further our mission of finding the right career for everyone in the world. If you are passionate about solving one of the most fundamental challenges of our society - employment, working on hard business problems, and being part of an amazing growth story - Eightfold is the place to be! You may want to refer to the media coverage on TechCrunch and Reuters. Eightfold.ai is led by Ashutosh Garg, a PhD in Machine Learning/IIT Delhi alumni who managed Search and Personalization at Google, and Varun Kacholia- IIT Bombay alumni, who led the News Feed Ranking team at Facebook and developed YouTube Search at Google.

Our customers

Press

About the Role

The Data Platform Team builds and operates the foundational data infrastructure powering Eightfold's AI and Talent Intelligence Platform. The team enables scalable, reliable, and efficient data processing across AI/ML, analytics, and product systems.

As a Lead Engineer , you will drive the architecture and development of large-scale data platforms, real-time and batch processing systems, and cloud-native infrastructure that supports millions of users globally. This is a high-impact technical leadership role with broad ownership across platform scalability, reliability, and engineering excellence.

Key Responsibilities

• Lead the design and development of scalable data platform services and distributed data systems.
• Build and optimize batch and real-time data pipelines and platform tooling.
• Drive best practices around reliability, observability, CI/CD, and infrastructure automation.
• Partner with AI/ML, analytics, and product engineering teams to enable efficient and reliable data access.
• Mentor engineers and contribute to long-term platform architecture and technical strategy.

Qualifications

• 7+ years of experience in distributed systems engineering.
• Strong programming skills in Java, Scala, Python, or Go.
• Experience with technologies like Spark, Kafka, Airflow, Flink, or Snowflake.
• Strong understanding of distributed systems, cloud infrastructure, and large-scale data processing.
• Experience operating production systems on AWS or similar cloud platforms.

We are a team of self-starters who excel in their fields. We believe in giving you responsibility, not a task. We want you to have ownership and pride in the work you are doing and see the positive impact of your work on your colleagues, our customers, and the world. We believe in providing transparency and support, so you can do the best work of your career.

Hybrid Work @ Eightfold

We embrace a hybrid work model that aims to boost collaboration, enhance our culture, and drive innovation through a blend of remote and in-person work. We are committed to creating a dynamic and flexible work environment that nurtures the collaborative spirit of our team. Starting February 1, 2024, employees residing near Santa Clara, California, or our Bangalore and Noida offices in India will return to the office twice a week.

Eightfold.ai provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, or disability.

Experience our comprehensive benefits with family medical, vision and dental coverage, a competitive base salary, and eligibility for equity awards and discretionary bonuses or commissions.

Our customer stories- https://eightfold.ai/customers/customer-stories/ Press- https://eightfold.ai/about/press