AI Software Engineer - Data Platform (SF, Palo Alto or NYC)

Perplexity AI; Palo Alto, San Francisco, New York (CA, CA, NY); On-site; USD 200,000 - 280,000; Full time

Job summary

An innovative firm is seeking an experienced AI Software Engineer to help build a cutting-edge AI Data Platform. This role involves designing and maintaining scalable data pipelines, collaborating with various teams to create reusable data resources, and implementing tools for analytics and AI applications. Join a rapidly growing company that has redefined online search and interaction, serving millions of users worldwide. With significant funding and a strong market presence, this position offers a unique opportunity to contribute to transformative AI solutions in a dynamic environment.

Benefits

Comprehensive health insurance

Dental insurance

Vision insurance

401(k) plan

Equity options

Qualifications

• Proficiency in programming and data engineering with open source tools.
• Experience in cloud-based data services and containerized infrastructure.

Responsibilities

• Designing and maintaining scalable data pipelines and data lakes.
• Implementing tools for analytics and AI product use cases.

Skills

Programming skills

Data engineering skills

Open source data processing

Quantitative skills

Self-motivation

Education

5+ years of industry experience

Tools

AWS

Spark

Flink

Docker

Databricks

Snowflake

Job description

AI Software Engineer - Data Platform - SF

Perplexity is seeking an experienced Software Engineer focusing on building the next-gen AI Data Platform to help revolutionize the way people search and interact online. In this role, you'll help build Perplexity’s end-to-end AI data stack and flywheel which powers all AI products, ML use cases and language models.

Perplexity is rapidly scaling both in number of use cases and number of users. Perplexity’s data stack powers scalable, personalized and fast answers for millions of people worldwide.

Tech Stack: Spark | AWS Data Stack (S3, RDS, DynamoDB, Docker, EKS, Kinesis) | Pytorch | Docker | Databricks | Snowflake

Responsibilities

• Collaborating closely with AI Product, Applied ML, Post-Training and Data Science teams to design, build, and maintain scalable data pipelines and data lakes
• Developing, deploying, and monitoring entire data lifecycle for ingestion, transformation, streaming and storage at high scale
• Implementing tools and abstractions on top of data infrastructure for a variety of analytics, recommendations, AI product and post-training use cases
• Working closely with product and AI teams to develop reusable data resources and design patterns

Qualifications

• Extensive programming and data engineering skills, with proficiency in open source & distributed data processing (AWS, Spark, Flink, Iceberg)
• Familiarity with cloud-based data services (e.g., AWS, RDS, DynamoDB), containerized infrastructure (e.g., EKS, Docker), and data streaming (Flink, Spark streaming, CDC)
• Strong quantitative and engineering skills with experience in estimating performance at high scale
• Experience supporting various ML/AI engineering teams to build scalable platforms to accelerate R&D for frontier models and AI products
• Self-motivated with a strong sense of ownership of systems and designs
• 5+ years of industry experience in distributed systems or AI infrastructure

The cash compensation range for this role is $200,000 - $280,000.

At Perplexity, we've experienced tremendous growth and adoption since publicly launching the world's first fully functional conversational answer engine just over a year ago. Our AI-powered search assistant has amassed 10 million monthly active users as of early 2024, with our mobile apps installed over 1 million times across iOS and Android devices. In 2023 alone, we served over 500 million queries from users around the globe.

To support our rapid expansion, we've raised significant funding from some of the most respected investors in technology. In January 2024, we raised $73.6 million in a Series B round led by IVP, with participation from NVIDIA, Jeff Bezos' investment fund, NEA, Databricks, and other prominent firms. We followed that up with a $62.7 million Series B1 round in April 2024 led by Daniel Gross, valuing Perplexity at over $1 billion.

Equity: In addition to the base salary, equity is part of the total compensation package.

Benefits: Comprehensive health, dental, and vision insurance for you and your dependents. Includes a 401(k) plan.

Apply for this job

First Name *

Last Name *

Email *

Phone *

Resume/CV *

Website

LinkedIn Profile

Will you now or in the future require visa sponsorship for employment? *

Are you open to a hybrid work schedule (In office M-W-F) *

Are you located in the San Francisco Bay or New York Metro Area? *

What are your preferred pronouns?