AI Product Analyst - Automation team

MEDvidi, Inc.; Województwo mazowieckie; À distance; PLN 120 000 - 180 000; Plein temps

Avantages offerts par ce poste

Health insurance after probation

Sports & wellness compensation

Fully remote, long-term B2B

19 working days of vacation per year

3 additional wellness days per year

Paid sick leave for the first 5 days

Gifts to employees for importantevents

Career Development

Personalized English lessons via Prepy

Résumé du poste

MEDvidi, Inc. seeks a Product Analyst to own analytics for the team end-to-end as AI-powered and conventional features expand. You will translate production behavior into evidence to guide decisions, combining product analytics, AI evaluation, and operational analysis.

You will work across logs, prompts, dashboards, and data to assess AI quality, measure impact, and drive improvements, collaborating with Product, Engineering, MedOps, Clinical, and Legal teams in a fast-paced startup environment.

Qualifications

• 3+ years of experience in Product Analytics/Operational Analytics/Data Analytics
• Understanding of AI, ML, and LLM systems
• Hands-on experience evaluating LLM or AI output
• Effective, structured written and verbal communication
• Comfort in ambiguous, fast-changing environments
• Strong product / operational analytics background, writing SQL and Python against production data
• Fluent Russian; English level B2 or higher
• Ability to work CET-overlapping hours in a B2B format

Responsabilités

• Own the full product analytics stack for features the team ships - discovery and sizing, experiment design and analysis, funnel and adoption analysis, post-release impact assessment
• Analyze production behavior of AI and automation workflows across clinical and operational journeys
• Investigate hallucinations, silent failures, and quality degradation in AI-generated output
• Work directly with logs, transcripts, generated artifacts, and operational data to find patterns dashboards miss
• Design evaluations for AI quality - both automated and human review - and close the loop into prompt and product changes
• Build monitoring and dashboards for AI and automation health, and the operational metrics they roll up to
• Measure business and operational impact of automation initiatives and surface risks
• Partner with Product, Engineering, Clinical, MedOps, and Legal to support fast decisions under ambiguity

Connaissances

Product analytics

Operational analytics

Data analytics

SQL

Python

LLM / AI evaluation

Communication

Outils

SQL

Python

Tableau

Description du poste

At MEDvidi, the Automation team builds and operates a mix of AI and conventional product features that automate clinical and operational workflows - from AI-drafted medical documentation to automated resolution of patient issues, intake verification flows, provider tooling, and the backend pipelines behind them. We free clinicians and operations from manual work a machine can do reliably and safely, in a domain where mistakes have real clinical and compliance consequences.

It's a full-stack product team, not an AI lab: we ship UI, backend workflows, integrations, and AI features side by side, and we measure them the same way.

The team works closely with Product, Engineering, MedOps, Clinical people. We're hiring an Product Analyst to own analytics for the team end-to-end as our automated surface area grows.

The role

You'll be the Product Analyst for the team - across everything we ship, both classical product features and AI-powered ones. You'll turn the production behavior of our systems into evidence the team can act on: which parts work, where they fail silently, and what to fix next. The work blends product analytics, AI evaluation, and operational analysis.

Responsibilities

• Own the full product analytics stack for features the team ships - discovery and sizing, experiment design and analysis, funnel and adoption analysis, post-release impact assessment - across both AI-powered and conventional product features.
• Analyze production behavior of AI and automation workflows across clinical and operational journeys.
• Investigate hallucinations, silent failures, and quality degradation in AI-generated output.
• Work directly with logs, transcripts, generated artifacts, and operational data to find patterns dashboards miss.
• Design evaluations for AI quality - both automated and human review - and close the loop into prompt and product changes.
• Build monitoring and dashboards for AI and automation health, and the operational metrics they roll up to.
• Measure business and operational impact of automation initiatives and surface the risks they introduce.
• Partner closely with Product, Engineering, Clinical, MedOps, and Legal to support fast decisions under ambiguity.

Requirements

• 3+ years of experience in Product Analytics/Operational Analytics/Data Analytics, or analytics roles within complex production systems.
• Understanding of AI, ML, and LLM systems.
• Hands-on experience evaluating LLM or AI output - prompt iteration, eval frameworks and human review pipelines is a strong plus.
• Effective, structured written and verbal communication - able to turn analysis into crisp, decision-ready writeups that non-analysts can act on.
• Comfort in ambiguous, fast-changing environments where ground truth often has to be constructed.
• Strong product / operational analytics background, comfortable owning metrics end-to-end and writing SQL and Python code against messy production data.
• Fluent Russian; English level B2 or higher,
• Ability to work CET-overlapping hours in a B2B format.

Nice to Have

• Experience evaluating LLM outputs or AI systems.
• Previous experience in Data Science or Machine Learning.

What we offer:

• Opportunity to work on products at the intersection of Product Analytics and AI.
• Real influence on product direction and engineering practices
• High-impact work in a product-driven medical tech company
• Startup environment: fast iteration, high ownership, minimal bureaucracy
• Competitive compensation and growth opportunities

Benefits

• Health insurance after probation period
• Sports & wellness compensation
• Fully remote, long-term B2B collaboration
• 19 working days of vacation per year
• 3 additional wellness days per year
• Paid sick leave for the first 5 working days
• Gifts to employees for important events
• Career Development
• Personalized English lessons via Preply