Remote LLM Platform Engineer — Scale AI Infrastructure

Whatnot; New York (NY); On-site; USD 245,000 - 345,000; Full time

Benefits offered by this job

Health Insurance

Work From Home Support

Parental Leave

401k matching

Job summary

Whatnot is seeking an AI/ML Engineer to design and scale core infrastructure powering large language model applications. You will work with ML scientists to productionize models, grounding responses with Whatnot’s business context and building robust evaluation systems.

You’ll deploy RAG systems, MCP servers, and human-in-the-loop pipelines, ensuring scalable, auditable, and fast AI across Whatnot’s live commerce platform. A strong background in Python and cloud services is required.

Qualifications

• Bachelor's degree in Computer Science, Statistics, Applied Mathematics or related field or equivalent work experience.
• 4+ years of professional experience developing machine learning systems and algorithms.
• 3+ years of software engineering for production systems with consumer-scale loads.
• 1+ years of Python development experience.
• Ability to work autonomously and drive initiatives across multiple product areas.
• Experience with PostgreSQL, DynamoDB, Elasticsearch, Redis.
• Familiarity with DataDog and Grafana for monitoring.
• Cloud experience with AWS services (SageMaker, Lambda, Kinesis, S3, EC2, EKS/ECS).
• Strong documentation and communication within a remote team.

Responsibilities

• Own the infrastructure powering LLMs across critical business surfaces (growth, recommendations, trust and safety, fraud, seller tooling).
• Create scalable LLM evaluation frameworks to measure model performance and guide iteration via CI/CD.
• Deploy RAG systems and MCP servers to ground LLM responses in business context while enforcing PII controls.
• Design human-in-the-loop feedback pipelines for scalable LLM evaluation.
• Bridge research and production to transform ideas into scalable solutions.
• Tackle new technical challenges as AI scales within Whatnot’s ecosystem.

Skills

Python

ML systems

Cloud platforms

Production debugging

Documentation

Collaboration

CI/CD

Education

Bachelor's in CS/Statistics/Math or related field

Tools

PostgreSQL

DynamoDB

Elasticsearch

Redis

DataDog

Grafana

AWS (SageMaker, Lambda, Kinesis, S3, EC2)

Kafka

Job description

Whatnot is seeking an AI/ML Engineer to design and scale core infrastructure powering large language model applications. You will work with ML scientists to productionize models, grounding responses with Whatnot’s business context and building robust evaluation systems.

You’ll deploy RAG systems, MCP servers, and human-in-the-loop pipelines, ensuring scalable, auditable, and fast AI across Whatnot’s live commerce platform. A strong background in Python and cloud services is required.