Senior Applied Research Engineer 2

Cacheflow; San Francisco (CA); On-site; USD 192,000 - 259,800; Full time

Benefits offered by this job

Stock equity

Health coverage

401(k) plan

Paid parental leave

Professional development stipends

Flexible vacation policy

Job summary

Cacheflow is seeking a Senior Applied Research Engineer to enhance the effectiveness of our AI systems through focused research and experimentation. This role involves designing information retrieval strategies and collaborating with engineers to turn validated approaches into production-ready systems.

The successful candidate will have over 6 years of experience in applied research and a strong foundation in NLP and information retrieval. Benefits include stock equity, health coverage, and flexible vacation policies.

Qualifications

• 6+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems.
• 2+ years of hands-on experience building or contributing to production AI/ML systems.
• Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance.
• Experience with RAG systems: chunking strategies, vector databases, retrieval optimization.
• Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical significance.
• Strong Python skills and comfort with notebook-driven research workflows.
• Experience communicating research findings to engineering teams and translating insights into actionable improvements.

Responsibilities

• Design and evaluate information access and reasoning strategies across RAG, agents, and classic ML.
• Prototype GenAI workflows that map and reason over compliance objects.
• Explore ML and probabilistic approaches for various tasks.
• Build and maintain evaluation frameworks: golden datasets, automated quality metrics.
• Implement and tune ranking/reranking systems.
• Run experiments to validate hypotheses and quantify improvements.
• Debug failure modes and build error taxonomies.
• Collaborate with AI and Software Engineers to hand off validated approaches.

Skills

Applied research

Data science

Machine Learning

Information retrieval

NLP

Python

A/B testing

Statistical significance

Job description

Job Summary

Drata is seeking a Senior Applied Research Engineer to drive the quality and effectiveness of our AI systems through rigorous experimentation, evaluation, and applied research. This is a research-focused role emphasizing experimentation and rigor over production engineering. You'll own the science behind how Drata's AI products retrieve, reason, and respond — and you'll work closely with AI and Software Engineers to turn validated approaches into production‑ready systems. Drata's compliance platform is document‑heavy: VRM Agent, AIQA, Trust Agent, policy‑to‑control mappers, and more all depend on high‑quality information retrieval and reasoning.

Responsibilities

• Design and evaluate information access and reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, semantic routing.
• Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls → risks → requirements → evidence).
• Explore ML and probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and structured prediction.
• Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection.
• Implement and tune ranking/reranking systems: cross‑encoders, LLM‑based rerankers, learning‑to‑rank, custom scoring functions.
• Run experiments to validate hypotheses and quantify improvements before production rollout.
• Debug failure modes and build error taxonomies across retrieval, reasoning, and generation.
• Collaborate with AI and Software Engineers to hand off validated approaches for productionization.
• Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product.

Qualifications

• 6+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems.
• 2+ years of hands‑on experience building or contributing to production AI/ML systems.
• Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance.
• Experience with RAG systems: chunking strategies, vector databases, retrieval optimization.
• Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical significance.
• Strong Python skills and comfort with notebook‑driven research workflows.
• Experience communicating research findings to engineering teams and translating insights into actionable improvements.
• Bonus: Experience with compliance, legal, or document‑heavy domains.
• Bonus: Publications or contributions in IR, NLP, or RAG evaluation.

Benefits

We provide a comprehensive total rewards package that supports well‑being, growth, and work‑life balance.

• Shared Success: stock equity for full‑time employees.
• Health & Wellness: up to 100% employer‑paid premiums for medical, dental, and vision coverage for employees and dependents; comprehensive wellness benefits and healthcare concierge services.
• Financial Well‑being: 401(k) plan, company‑paid life and disability insurance, tax‑advantaged spending accounts, and discounted voluntary offerings.
• Family Support: paid parental leave after six months of employment; access to Kindbody fertility and family‑building benefits and dedicated leave specialists.
• Growth & Development: annual stipends for professional and personal development; internal learning opportunities.
• Time Off & Flexibility: flexible vacation policy, paid holidays, and other perks for rest and recovery.

Competitive base salary and benefits, typically in the form of Restricted Stock Units (RSUs). The salary range for this role is: $192,000 - $259,800. Salary ranges are subject to adjustments and final offers may vary.