AI product analyst

MEDvidi, Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Remote work

Health insurance

English lessons

Vacation days

Job summary

MEDvidi, Inc. представляет собой платформу психического здоровья на базе ИИ, занимающуюся безопасной, эффективной и масштабируемой помощью в США. Ваша роль – анализ производственных сценариев AI и построение мониторов качества и влияния на бизнес.

Задачи включают разработку фреймворков оценки, создание дашбордов и мониторинга, а также взаимодействие с командами Product, Engineering, QA и Operations для быстрого принятия решений в быстро меняющейся среде.

Qualifications

• 5+ лет опыта в Product Analytics, Data Analytics или сопутствующих аналитических ролях.
• Сильные навыки SQL и Python для анализа больших текстовых наборов.
• Опыт работы с ML пайплайнами или AI‑продуктами, понимание ограничений LLM.
• Умение строить мониторинг и аналитические дашборды, коммуникация результатов.
• Владение русским языком; уровень английского не ниже B2; работа в CET‑перекрытие.

Responsibilities

• Анализ поведения Voice AI Agents и рабочих процессов AI.
• Разработка рамок оценки качества AI, надежности и бизнес‑эффекта.
• Создание систем мониторинга и дашбордов видимости производительности AI.
• Измерение влияния AI на эффективность операций, результаты и опыт пациентов.
• Обнаружение сбоев, аномалий и новых режимов отказов в продакшене.
• Тесное сотрудничество с Product, Engineering, QA и Operations.

Skills

SQL

Python

Russian language

English (B2+)

Product analytics

Tools

ML pipelines

Job description

Описание

MEDvidi is an AI-powered mental healthcare platform setting a new standard for safe, effective, and scalable psychiatric care in the United States. We combine licensed providers with proprietary AI tools to deliver consistent, outcomes-driven treatment for conditions like ADHD, anxiety, depression, and more. MEDvidi's technology automates charting, follow-ups, and treatment planning, freeing providers to focus on patient care while improving efficiency and clinical quality.

Задачи

Analyze production behavior of Voice AI Agents and conversational AI workflows. Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations. Work directly with call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights. Design evaluation frameworks for measuring AI quality, reliability, and business impact. Build monitoring systems and dashboards for AI performance visibility. Measure the impact of AI systems on operational efficiency, business outcomes, and patient experience. Detect quality shifts, anomalies, and emerging failure modes in production environments. Partner closely with Product, Engineering, QA, and Operations teams on AI initiatives. Support rapid decision-making in fast-changing and ambiguous environments.

Требования

5+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics roles within complex production systems. Strong SQL skills. Python for analytical investigations and working with large-scale text datasets. Experience working with ML pipelines or AI-powered products Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes. Experience building monitoring systems, analytical frameworks, or operational dashboards. Strong communication skills and ability to translate analysis into product decisions. Ability to work effectively in ambiguous and fast-changing environments. Fluent Russian; English level B2 or higher. Ability to work CET-overlapping hours.

Будет плюсом: Experience with conversational AI or AI agents; Experience evaluating LLM outputs; Experience with AI quality metrics and evaluation methodologies; Experience with prompt iteration and AI evaluation frameworks

Условия

High-impact environment: A chance to influence how AI is used in real patient interactions. Growth & compensation: Competitive compensation and clear growth opportunities. Remote-first: Fully remote, long-term B2B collaboration. Health & wellness: Health insurance after the probation period, plus sports & wellness compensation. Career development: Career growth, including personalized English lessons via Preply. Time off: 19 working days of vacation per year, 4 additional wellness days, and paid sick leave for the first 5 working days. Culture: Thoughtful gifts for important life events.