Senior ML Specialist

careers-quartile; Brasil; Presencial; BRL 250 000 - 420 000; Tempo integral

Resumo da oferta

Quartile is seeking a Senior ML Specialist to lead model strategy and evaluation for the Sciene AI platform. You will design benchmarks, build classifiers, and fine-tune models in a production codebase, ensuring robust, measurable performance across tasks.

You will collaborate with data and software engineers to deploy agentic AI solutions, maintain observability, and optimize reliability and cost in a fast-moving environment.

Qualificações

• 5+ years of applied ML with pretraining, evaluation, and deployment experience.
• Strong foundation in statistics and probability for rigorous analysis.
• Deep Understanding of LLMs, transformers, decoding and failure modes.
• Hands-on ML/NLP, including embeddings and semantic similarity.
• Experience fine-tuning models (LoRA/PEFT) and data curation for datasets.

Responsabilidades

• Own model strategy across tasks selecting providers and benchmarks.
• Design offline evals, judge calibration, and regression checks.
• Build and fine-tune models for routing, categorization, and detection tasks.
• Develop agentic AI products with reusable skills and tool integrations.
• Implement prompt engineering and structured outputs with schemas.
• Monitor quality and cost; ship and maintain production services.

Conhecimentos

Applied ML

Statistics

LLM theory

NLP

Fine-tuning

Evaluation design

Python

PyTorch

Hugging Face

Model deployment

Databricks

Docker

Azure

FastAPI

MCP/agents

Formação académica

Master's or PhD in ML/CS/Math

Ferramentas

Databricks

Azure

Docker

FastAPI

Pydantic

Descrição da oferta de emprego

WHO WE ARE:

Quartile, the world's largest retail media optimization platform, is a trusted partner for multichannel e-commerce success. Through unmatchedexpertiseand patented AI technology, we fuel growth for 5,300+ brands and sellers worldwide and manage an annual ad spend exceeding$2 billion. The award-winning platform covers major marketplaces and ad channels foroptimalreach. The result is unprecedented granularity, smarter budgeting, and bespoke solutions for retailers.

Quartile is proud to be an equal opportunity employer with employees stemming from a wide range of backgrounds and experiences. As a business, we value the enrichment that diversity brings to our organization and are committed to a culture that creates a sense of inclusion and belonging. We welcome new perspectives and affirm that all employment decisions are made without regard to race, color, ancestry, religion, national origin, age, familial or marital status, sex, sexual orientation, pregnancy, gender identity or expression, disability, genetic information, veteran status, or any other classification protected by federal, state, or local law.

About Sciene

At Sciene, the mission is to empower professional services firms with cutting-edge, customized AI solutions — enhancing automation, analytics, and optimization across industries while prioritizing security, cost efficiency, and state-of-the-art technology.

Our flagship product, the Sciene AI Companion, is an autonomous customer success platform deployed across Quartile — the world's largest retail media optimization platform, managing performance marketing for 1,000+ brands. It automates relationship-heavy enterprise workflows end to end: generating personalized email replies in the CSM's own voice (8x faster), building full presentation decks for client meetings (12x faster), and detecting and diagnosing account fluctuations before anyone has to ask (6x faster). None of this replaces human judgment — it removes the work that was getting in the way of it.

Read more about how we built it: Sciene AI Companion: Building an Autonomous Customer Success Platform on Databricks

OVERVIEW:

Sciene runs a production agentic AI platform: a config-driven engine where every product is an agent with its own identity, skills, tools, and quality gates, executing ReAct loops against real business data across multiple LLM providers. The platform is built; what it needs now is deeper machine learning judgment behind every model decision it makes.

The Senior ML Specialist is the team's authority on how these models actually work and how to measure them. You will decide which models run which tasks and prove it with statistically sound benchmarks, design the evaluation methodology the whole team relies on, build classifiers and fine‑tuned models where they beat prompting, and diagnose model behavior from first principles rather than by trial and error. You will do this inside a production codebase — shipping, operating, and monitoring what you design.

WHO THIS ROLE IS FOR:

This is a machine learning role first and a software engineering role second. We are looking for someone whose core professional background is in machine learning and statistics — not a software engineer who adopted LLM APIs in the last few years. You should be able to explain from first principles how a large language model is trained and why it behaves the way it does, reason about model outputs as probabilistic objects, and tell whether a difference between two models on a benchmark is real or noise.

REQUIREMENTS:

• 5+ years of applied machine learning experience, with a substantial part predating the current generative AI wave — training, evaluating, and deploying models, not only integrating LLM APIs
• Strong foundation in statistics and probability: hypothesis testing, confidence intervals, sampling and sample-size reasoning, bias/variance, calibration, and the ability to state when a measured difference is statistically meaningful
• Deep understanding of LLM and generative AI theory: transformer architecture and attention, tokenization, embeddings, the training pipeline (pretraining, supervised fine-tuning, RLHF/DPO), decoding strategies (temperature, top-p, sampling vs. greedy), scaling behavior, and the known failure modes — hallucination, sycophancy, long-context degradation, prompt sensitivity — and why they occur
• Hands‑on classical ML and NLP: classification, clustering, feature engineering, text classification, embeddings and semantic similarity — and the judgment to know when a small trained model outperforms an LLM on cost, latency, or accuracy
• Experience fine‑tuning models (LoRA/PEFT, supervised fine‑tuning, or full fine‑tuning) and building the datasets for it — and equally, knowing when prompting or retrieval is the better answer
• Experience designing evaluation methodology for AI systems: building offline eval sets, LLM-as-judge and its known biases, inter-rater agreement, regression benchmarks, and significance testing on results
• Production‑grade Python: PyTorch, scikit‑learn, Hugging Face, and modern async Python; comfortable shipping into a FastAPI/Pydantic codebase with tests and CI — this role ships code, not only notebooks
• Hands‑on experience with at least one major LLM provider API (OpenAI, Anthropic, Google), including structured outputs and tool/function calling, and an understanding of the trade‑offs between models and providers
• Master's or PhD in Machine Learning, Statistics, Computer Science, Mathematics, or a related quantitative field — or equivalent demonstrated depth (publications, competition results, released models or research code)
• The autonomy expected of a senior: you own a problem end to end — from framing to shipped, measured outcome — and are accountable for the result

PREFERRED QUALIFICATIONS:

• Publications, research, or open‑source contributions in NLP or machine learning; strong Kaggle or equivalent competition record
• Experience with retrieval systems: embedding model selection, chunking strategies, reranking, and retrieval quality evaluation
• Experience with agent evaluation: trajectory analysis, tool‑use accuracy, multi‑step task success metrics
• Experience with model cost and latency optimization: model routing, distillation, prompt caching, quantization
• Experience with Databricks (MLflow, model serving, Delta Sharing) and with Azure (Container Apps, Key Vault) and Docker
• Experience with observability stacks: OpenTelemetry, Grafana, Loki, structured logging
• Experience with the Model Context Protocol (MCP) or similar agent‑integration standards
• Track record of shipping something from 0 to 1 in a fast‑moving environment where priorities shift often

WHAT YOU’LL DO:

• Own model strategy across the platform: evaluate and select models per task (OpenAI, Anthropic, Google, and emerging providers) using our benchmark and LLM-as-judge harness, and make the decision defensible with proper statistical treatment of the results
• Design and maintain the evaluation methodology the team relies on: offline eval sets, judge calibration and reliability, regression benchmarks, and quality metrics that go beyond "it looks right"
• Build classification and fine‑tuned models where they beat prompting — for routing, categorization, and detection tasks — including dataset construction, training, evaluation, and deployment
• Design, develop, and ship agentic AI products on our platform: agent identities, reusable skills, tool integrations, and structured outputs — often config‑driven, with platform‑level code changes when the engine needs to grow
• Do prompt and context engineering grounded in how models actually process context: system prompt assembly, context injection from live data sources, thread/memory management, and structured output design with Pydantic schemas
• Own output quality: design deterministic enforcers, validation quality gates, and LLM-as-judge evaluators, and measure whether the judges themselves are reliable
• Diagnose model behavior from first principles — hallucination, inconsistency, drift across model versions — and turn the diagnosis into fixes in prompts, retrieval, guardrails, or model choice
• Build and extend agent tools that query Databricks, MongoDB, and external systems, and integrate agents with the broader ecosystem via the Model Context Protocol (MCP)
• Operate what you ship: instrument services with OpenTelemetry, monitor quality, latency, and cost in Grafana, and improve them over time
• Collaborate with data engineers, software engineers, and product to deliver end‑to‑end AI solutions, and raise the team's ML fundamentals through reviews and knowledge sharing
• Follow the research literature critically and bring in techniques that hold up on our data — not every paper does
• Document and maintain the codebase, ensuring code quality and adherence to best practices

*This is a PJ contract based in Brazil.