ML Engineer: Retrieval & Embeddings for AI Platform

Spotlight Partners; United States; Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Hybrid work flexibility in MI area

Competitive compensation

Real ownership of the AI product layer

Job summary

Spotlight Partners seeks an experienced ML Engineer to design, optimize, and deploy retrieval and embedding strategies behind Beacon—their AI platform. You will benchmark models for cost and latency, and collaborate with the evaluation team to measure performance.

You will own model retrieval, tuning, and tooling decisions, balancing fine-tuning with prompt-based approaches while ensuring scalable, cost-efficient production.

Qualifications

• Strong Python skills with experience in ML frameworks and model tuning.
• Experience with retrieval-augmented generation (RAG) and embeddings.
• Hands-on knowledge of vector stores and orchestration frameworks.

Responsibilities

• Design and tune retrieval, ranking, and reranking components.
• Own embedding and chunking strategy for Beacon models.
• Fine-tune or distill models where beneficial over prompts.
• Benchmark model options for cost and latency.
• Build feature and training pipelines for predictive components.

Skills

Python

PyTorch

Hugging Face

scikit-learn

LangChain

LlamaIndex

Vector stores

Transformers

RAG design

SQL

Pandas

Experiment tracking

Tools

LangChain

LlamaIndex

pgvector

Pinecone

Weaviate

Job description

Spotlight Partners seeks an experienced ML Engineer to design, optimize, and deploy retrieval and embedding strategies behind Beacon—their AI platform. You will benchmark models for cost and latency, and collaborate with the evaluation team to measure performance.

You will own model retrieval, tuning, and tooling decisions, balancing fine-tuning with prompt-based approaches while ensuring scalable, cost-efficient production.