AI Product Data Analyst

Remote from: Serbia
Annual salary: Undisclosed
Salary information is not provided for this position. Check our Salary Directory to estimate the average compensation for similar roles.
Department: Data Science & Analytics
Employment type: Full Time,
Job posted: 8 May 2026
Apply before: 8 Jun 2026
Experience level: Midweight
Views / Applies: 421 / 118

We are an online mental health treatment center aiming to make professional care accessible and affordable for everyone.

Mental Health Care
2019

AI Summary

MEDvidi is seeking an AI Product Data Analyst to join their AI product team, focusing on measuring and improving the impact of AI agents and LLM-based systems in production. The role involves analyzing performance, building dashboards, investigating issues, and collaborating with product and engineering teams. Candidates need 3+ years of data analysis experience, strong Python and SQL skills, and familiarity with LLM systems. Fluency in Russian and B2+ English are required. The position offers remote work, competitive compensation, and benefits like health insurance and wellness compensation.

Job Complexity

Easy Hard

AI Insight The role requires a rare combination of advanced data analysis skills, AI/LLM knowledge, and Russian fluency, plus the ability to work in a fast-paced startup environment, making it challenging.

Salary Analysis

AI Insight The job listing does not specify a salary range, but based on US market data for an AI Product Data Analyst with 3+ years experience, the median is approximately $115,000. This role offers competitive compensation typical for high-impact roles in medical tech startups.

Key Skills

Data Analysis Python SQL Tableau LLM AI Product Dashboarding Product Analytics Russian Language Healthcare Technology

Cover Letter Sample

Dear Hiring Manager,

I am excited to apply for the AI Product Data Analyst position at MEDvidi. With over three years of experience as a Data Analyst working on product data and a strong proficiency in Python and SQL, I have a proven track record of turning ambiguous data into actionable insights. My understanding of LLM-based systems and their evaluation aligns perfectly with your need to measure and improve AI agent performance.

In my previous role, I built production-grade dashboards in Tableau and collaborated closely with product and engineering teams to drive data-informed decisions. I thrive in fast-paced, ambiguous environments and am eager to contribute to MEDvidi's mission of transforming mental healthcare through AI. I am fluent in Russian and have B2+ English, meeting your language requirements.

I look forward to the opportunity to discuss how my skills can help scale your AI initiatives. Thank you for your consideration.

Sincerely,
[Your Name]

Possible Interview Questions

Can you describe a time when you had to analyze the performance of an AI system in production? What metrics did you use and how did you present your findings?

In my previous role, I evaluated a chatbot's performance by tracking metrics like response accuracy, user satisfaction scores, and task completion rate. I built a dashboard in Tableau that highlighted these metrics over time, and identified a drop in accuracy due to a model update. I presented this to the product team with recommendations for rollback and retraining.

How would you approach evaluating the impact of a new LLM-based feature on user experience and business outcomes?

I would define key metrics such as user engagement (e.g., session length, feature adoption), quality metrics (e.g., response relevance, error rates), and business metrics (e.g., conversion rates). I'd set up A/B testing or pre-post analysis, control for confounding factors, and use statistical methods to assess significance. Then I'd create a dashboard to monitor these metrics continuously.

Describe your experience with SQL complex queries. Can you give an example of a challenging query you wrote and what it accomplished?

I once wrote a query to join multiple tables (user sessions, chatbot logs, and support tickets) to identify patterns leading to escalations. It involved window functions to calculate time between events, conditional aggregations to flag issues, and subqueries to filter relevant data. The result was a dataset that revealed common user paths causing frustration, which led to product improvements.

How do you handle ambiguous problems where data is incomplete? Give a specific example.

When tasked with measuring AI agent success without explicit ground truth, I used proxy metrics like user re-engagement and manual sampling to validate. I collaborated with product to define success criteria and used statistical imputation for missing data. This approach provided actionable insights despite data gaps.

Why are you interested in working at MEDvidi specifically, and how does your background align with our mission?

I am passionate about using data to drive positive impact in healthcare. MEDvidi's mission to make mental healthcare accessible resonates with me. My experience in AI product analytics and my ability to work in fast-paced environments align well with your needs. I am eager to apply my skills to help scale your AI initiatives and improve patient outcomes.

Description

At MEDvidi, we are transforming mental healthcare by making it more accessible and affordable across the United States. Our mission is to remove barriers to mental health support and ensure that everyone can access the care they need.

As our AI initiatives scale, high-quality analytics becomes a critical bottleneck – and this role is designed to solve it.

About the Role

We are hiring a Data Analyst to play a key role within the AI product team, ensuring data-driven accuracy in decision-making while evaluating, improving, and scaling AI-powered solutions in production.

This is a highly strategic, fast-paced role focused on measuring the real impact of AI agents and LLM-based systems, turning ambiguous problems and incomplete data into accurate decisions

Requirements

3+ years of hands-on experience as a Data Analyst working with product or platform data
Strong proficiency in Python for analytical work and ability to independently investigate data on the level of datapoints, build structural analyses, and present results
Confident SQL skills with experience working on complex queries
Experience building production-grade dashboards in BI tools, preferably in Tableau
Clear understanding of how LLM-based systems operate, including their limitations, failure modes, and trade-offs
Strong problem-solving and ownership mindset: you take responsibility for analytical outcomes, not just task execution
Ability to work in fast-changing, ambiguous environments and quickly learn new AI-related technologies
Proven experience linking product quality metrics to business impact and decision-making
Fluent Russian; English level B2 or higher
Ability to work CET-overlapping hours

Nice to Have

Experience evaluating or analyzing AI / ML / LLM products
Experience with dialog systems or conversational agents
Understanding of model quality metrics and evaluation approaches

Responsibilities

Analyze the performance and impact of AI initiatives already running in production
Develop and maintain dashboards that provide transparency into AI agent performance
Investigate issues, inconsistencies, and performance drops across automated user journeys
Design and implement evaluation approaches to measure system quality and user experience
Identify opportunities where automation improves outcomes, as well as areas where it introduces friction
Track adoption and effectiveness of automation across the product and support its further evolution
Partner closely with Product, Engineering, and QA teams to support data-informed decisions

Why Join Us

Work on a modern AI product where quality is a strategic function, not a formality
Real influence on product direction and engineering practices
High-impact work in a product-driven medical tech company
Startup environment: fast iteration, high ownership, minimal bureaucracy
Competitive compensation and growth opportunities

Benefits