Data Scientist

Ascendion; Teletrabalho; BRL 160.000 - 200.000; Tempo integral

Resumo da oferta

Uma empresa de soluções digitais está em busca de um Cientista de Dados Sênior para se concentrar na avaliação de produtos baseados em LLMs. O candidato ideal deve ter de 6 a 7 anos de experiência em Ciência de Dados e forte conhecimento em Inteligência Artificial Generativa. A função envolve revisar saídas de IA quanto à qualidade e segurança, colaborando com equipes de produto e engenharia para aprimorar processos de avaliação. A vaga é remota e oferece oportunidades para inovação e aprendizado.

Qualificações

• 6-7 anos de experiência em Ciência de Dados, Machine Learning ou funções de IA.
• Experiência prática avaliando saídas de LLMs (manual ou automatizado).
• Familiaridade com métricas de avaliação de IA ou frameworks de qualidade.

Responsabilidades

• Revisar e avaliar saídas de LLM quanto à qualidade, precisão e segurança.
• Trabalhar com SMEs em avaliações manuais e automatizadas.
• Definir critérios de avaliação e diretrizes de pontuação.

Conhecimentos

Experiência com LLMs e Inteligência Artificial Generativa

Habilidades sólidas em Python para análise e automação

Habilidades de comunicação fortes

Mentalidade colaborativa

Ferramentas

AWS (Lambda, EC2, S3, SQS, SNS)

Descrição da oferta de emprego

Data Scientist Ascendion•viamão, estado do rio grande do sul, br

Descrição da vaga

About Ascendion

Ascendion is a full-service digital engineering solutions company. We make and manage software platforms and products that power growth and deliver captivating experiences to consumers and employees. Our engineering, cloud, data, experience design, and talent solution capabilities accelerate transformation and impact for enterprise clients. Headquartered in New Jersey, our workforce of 6,000+ Ascenders delivers solutions from around the globe. Ascendion is built differently to engineer the next.

Ascendion | Engineering to elevate life

We have a culture built on opportunity, inclusion, and a spirit of partnership. Come, change the world with us:

• Build the coolest tech for the world’s leading brands
• Solve complex problems – and learn new skills
• Experience the power of transforming digital engineering for Fortune 500 clients
• Master your craft with leading training programs and hands-on experience.

Experience a community of change-makers!

Join a culture of high-performing innovators with endless ideas and a passion for tech. Our culture is the fabric of our company, and it is what makes us unique and diverse. The way we share ideas, learning, experiences, successes, and joy allows everyone to be their best at Ascendion.

About the Role

Job Title: Data Scientist

Contract Type: Contract in Brazil

Office model: Remote

We’re looking for a Senior Data Scientist focused on evaluating LLM-based products. The role is all about making sure AI outputs are high-quality, accurate, safe, and useful, using a mix of human evaluation and LLM-based evaluation approaches (LLM-as-a-Judge and LLM-as-a-Jury). You’ll work closely with SMEs, product, and engineering teams to build and scale evaluation frameworks that help improve AI products.

What You’ll Do:

• Review and evaluate LLM outputs for quality, accuracy, safety, and relevance.
• Work with SMEs on manual and human-in-the-loop evaluations.
• Use LLMs to evaluate other LLMs (LLM-as-a-Judge / LLM-as-a-Jury)
• Define simple evaluation criteria and scoring guidelines.
• Analyze results and share clear feedback with product and engineering teams.
• Help improve evaluation processes and best practices over time.

What We’re Looking For:

• 6-7 years of experience in Data Science, ML, or AI roles.
• Strong experience with LLMs and Generative AI.
• AWS: Core AWS - Lambda, EC2, S3, SQS, SNS, etc.
• Hands-on experience evaluating LLM outputs (manual or automated).
• Solid Python skills for analysis and automation.
• Experience with LLMs like ChatGPT, Claude, Mistral, or similar.
• Strong communication skills and a collaborative mindset.

Nice to Haves:

• Experience with AI evaluation metrics or quality frameworks.
• Familiarity with LLMOps or AI monitoring.
• Interest in AI safety, reliability, and responsible AI processes.