Principal Data Scientist

autodesk; San Francisco (CA); On-site; USD 135,000 - 242,000; Full time

Benefits offered by this job

Health benefits

Financial benefits

Wellness programs

Job summary

Ultimate.ai is looking for a Principal Data Scientist to evaluate AI systems and enhance user interactions. The ideal candidate will have 8+ years in data science with a focus on predictive modeling and AI products.

This role requires collaboration with product and engineering teams to develop frameworks for analyzing complex, AI-driven user experiences. Competitive compensation, including a base salary range of $135,000 to $242,000 and comprehensive benefits, is offered.

Qualifications

• 8+ years in data science or applied ML.
• Experience with LLMs and related technologies.
• Ability to translate ambiguous questions into structured programs.

Responsibilities

• Design and implement predictive models for user behavior.
• Develop frameworks for analyzing AI-driven user experiences.
• Guide experimentation strategies for agent performance.

Skills

Predictive modeling

User behavior modeling

Data architecture

Technical writing

Collaboration

Tools

AI/ML technologies

Vector databases

Job description

Position Overview

At Autodesk, we do what no other company can: we help our customers design and make anything. The Experience Foundations team at Autodesk plays a critical role in designing the experiences that make that mission a reality, especially in this transformative moment where seamless digital experiences and AI‑powered innovation will empower customers and teams to achieve meaningful outcomes faster.

The Principal Data Scientist will work within an Engineering team focused on building frameworks to evaluate AI Systems. This is a critical data science role for our agentic product platform—a next‑generation framework that combines domain‑specific AI agents into a unified harness enabling complex, multi‑step, personalized work across product surfaces. As Principal Data Scientist, you will help build fundamental evaluation techniques, create algorithms to assess quality scores, as well as define and predict what success for users and the product looks like.

You’ll work at the intersection of predictive modeling, behavioral analytics, and AI system design — partnering closely with product, engineering, and platform teams to build the measurement and intelligence tools and where appropriate, infrastructure, that this product needs to learn, adapt, and grow.

To deliver our vision, we need exceptional contributors and leaders who are curious, adaptable, customer‑focused, and excited to help shape the future of work at Autodesk.

Responsibilities

• Design and implement predictive models to analyze and anticipate user behavior, intent, and outcomes across multi‑agent workflows
• Define and establish data instrumentation, telemetry, and observability standards for agent‑based systems and user interactions
• Develop frameworks and prototypes for analyzing and optimizing non‑deterministic user experiences (including user interfaces) driven by AI agents
• Collaborate with product and engineering teams to integrate predictive intelligence into agent orchestration and decision‑making systems
• Create analytical models and reporting frameworks to generate business intelligence, forecasting, and performance insights
• Guide experimentation strategies, including A/B testing, causal inference, and evaluation methodologies for agent performance
• Provide technical leadership and recommendations on data architecture, model selection, and system scalability
• Translate ambiguous, early‑stage product questions into structured analytical programs with clear hypotheses, methods, and business impact

Minimum Qualifications

• 8+ years in data science or applied ML, with significant time in product analytics or user behavior modeling
• Deep experience with predictive modeling — classification, survival analysis, sequence models, or LTV/propensity frameworks
• Fluency in designing instrumentation and event schemas for complex, stateful systems
• Demonstrated ability to define metrics and measurement frameworks for new product spaces
• Experience working on or adjacent to AI/ML‑powered products — especially those with nondeterministic outputs
• Experience with LLMs and related technologies, including frameworks, embedding models, vector databases, and Retrieval‑Augmented Generation (RAG) systems, in production settings
• Deep understanding of data modeling, system architectures, and processing techniques, including 2D/3D geometric data representations
• Proven ability to translate theoretical concepts into practical solutions and prototype implementations
• Ability to work autonomously while effectively collaborating across teams, bridging the gap between research and practical implementation
• Excellent technical writing and communication skills for documentation, presentations, and influencing cross‑functional teams

Benefits

From health and financial benefits to time away and everyday wellness, we give Autodeskers the best, so they can do their best work. Learn more about our benefits in the U.S. by visiting https://benefits.autodesk.com/

Salary Transparency

Salary is one part of Autodesk’s competitive compensation package. For U.S.-based roles, we expect a starting base salary between $135,000 and $242,000. Offers are based on the candidate’s experience and geographic location, and may exceed this range. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.

Equal Employment Opportunity

At Autodesk, we’re building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.

Diversity & Belonging

We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging