Applied AI Data Analyst

Confido Health; New York (NY); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Equity

Fully paid health coverage

Dental & vision coverage

Fertility & family-forming support

Parental leave

Unlimited PTO

401(k)

Relocation support

Home-office setup stipend

Team meals & snacks

Job summary

Confido Health in New York City seeks an Applied AI Data Analyst to bridge unstructured financial and operational data to the R&D team. You will own the evaluation platform for agentic forecasting pipelines and improve customer-facing insights used to drive mission-critical decisions.

This role sits in the Applied AI Research department with opportunities to advance into senior data science or AI/ML engineering roles, and relocation is supported for in-person work.

Qualifications

• Graduate degree in a STEM field (MS/PhD).
• 2–4 years in quantitative data analytics or AI evaluation roles.
• Proficiency in Python and hands-on work with AI/ML models.
• Strong SQL and enterprise data infra skills (PostgreSQL, Snowflake, dbt, AWS).
• Ability to translate unstructured data into rigorous evaluation insights.

Responsibilities

• Analyze edge cases, model predictions, and extraction accuracy for LLM-driven document processing systems.
• Investigate root causes of model successes and failures across complex datasets.
• Define and maintain gold-standard evaluation datasets for AI agents and forecasts.
• Build dashboards, data transformations, and evaluation harnesses to monitor model drift and performance.
• Collaborate with applied AI engineers, product managers, and business ops to derive quantitative product insights.

Skills

Python

SQL

Analytical thinking

Cross-functional collaboration

Education

Master's or PhD in CS/DS/Statistics/Mathematics/Physics

Tools

PostgreSQL

Snowflake

dbt

AWS

LangSmith

Arize AI

TruLens

Job description

Confido is the AI infrastructure powering modern CPG — the platform that 200+ brands like OLIPOP, Simple Mills, Dr. Squatch, and Tropicana use to run everything from deductions to production planning. Finance, accounting, sales, and operations, unified in one system for the first time.

We’re growing 5x year over year with a small team in New York City; the people who join now will shape the product, the culture, and the company itself.

If you want your work on shelves everywhere — and outsized ownership while you build — we'd love to meet you.

The Role

As an Applied AI Data Analyst, you will bridge the gap between our massive, unstructured financial and operational datasets to our core R&D team. You will own the evaluation platform of our agentic workflows forecasting pipelines, as well as improve customer facing insights leading to mission-critical decisions.

This role is part of the Applied AI Research department, with options to advance to senior data science and AI/ML engineering roles.

Location

New York, NY (In-Person | Relocation supported)

What you'll do

• Algorithm & LLM Evaluation: Analyze edge cases, model predictions, hallucination rates, and extraction accuracy for our LLM-driven document processing systems and agentic workflows.
• Root Cause Analysis: Dig deep into complex datasets (inconsistent financial documents, fragmented enterprise systems) to determine why an extraction or time-series model succeeded or failed.
• Golden Dataset Curation: Define, curate, and maintain gold-standard evaluation datasets ("golden sets") for automated backtesting of AI agents and demand forecasting models.
• Performance Monitoring & Evals: Build dashboards, data transformations, and evaluation harnesses to monitor system health, model drift, and AI performance metrics across release cycles.
• Cross-Functional Collaboration: Partner directly with applied AI engineers, product managers and business operations managers to turn qualitative real-world problem definitions into quantitative product defining insights.

What we're looking for

• RequiredEducation: Graduate degree (M.S. or Ph.D.) in Computer Science, Data Science, Statistics, Mathematics, Physics, or a related STEM field.
• Experience: 2–4 years of experience in a quantitative data analyst or AI evaluation role, working closely alongside AI/ML R&D teams.
• Programming & Agentic AI: Proficiency in Python and hands-on works with AI/ML models.
• Data Stack: Strong capabilities with SQL and enterprise data infrastructure (PostgreSQL, Snowflake, dbt, AWS).
• Analytical Mindset: Exceptional ability to translate noisy, unstructured document data into rigorous evaluation insights for engineers.
• Nice to have: Experience with LLM observability, tracing, and eval frameworks (LangSmith, Arize AI, or TruLens).
• Nice to have: Proficiency with agentic workflows or multi-agent orchestration.
• Nice to have: Familiarity with demand forecasting, time-series models, or statistical modeling.
• Nice to have: Comfort in fast-moving, high-growth NYC startup environments.

Perks + Benefits

• Equity — own a meaningful piece of the company you’re helping build.
• Fully paid health coverage through Aetna — we cover 100% of employee premiums.
• Top-tier dental and vision coverage through Guardian.
• Carrot Fertility Pro — comprehensive fertility and family-forming support.
• 12 weeks of paid parental leave.
• Unlimited PTO — plus regular 4-day holiday weekends we actually take.
• 401(k) through Vestwell.
• Paid relocation support — we’ll help you make the move to NYC.
• Fully equipped workspace from day one — laptop, monitor, keyboard, and a $200 stipend to personalize your setup.
• Team perks — catered Friday lunches, team dinners, and unlimited coffee + snacks featuring products from the brands we work with.

Confido provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.