Senior Applied AI Engineer (GenAI & LLM Evaluation)

Acuity Analytics; Brasil; On-site; BRL 180,000 - 360,000; Full time

Job summary

Acuity Analytics in Brazil is seeking a Senior AI & Data Engineer (GenAI & LLM Evaluation) to help build and improve AI-powered data products. You will work with Data Scientists, Engineers, and Delivery teams to develop scalable data pipelines and deploy production-ready AI solutions.

This hands-on role involves designing solutions end-to-end, evaluating GenAI and LLMs, creating golden datasets, and continuously improving data quality and automation in agile, cross-functional teams.

Qualifications

• 5+ years of experience in data engineering, ML, AI engineering, or data science.
• Strong Python and SQL skills.
• Hands-on GenAI/LLM experience and model evaluation.
• Experience building production ML solutions.
• Strong understanding of data pipelines, validation, and quality frameworks.
• Experience integrating APIs and cloud-based AI services.
• Excellent communication and stakeholder management.
• Experience working in Agile, cross-functional teams.
• Consulting or customer-facing experience is a plus.

Responsibilities

• Build and maintain scalable data pipelines and workflows.
• Develop evaluation frameworks for GenAI and LLM applications.
• Create and maintain golden datasets for model testing.
• Evaluate and benchmark LLM performance.
• Build ML-assisted tools to improve delivery and automation.
• Deploy production-ready AI solutions.
• Work closely with delivery teams to solve technical challenges.
• Continuously improve data quality, automation, and operational efficiency.

Skills

Data Engineering

Machine Learning

Python

SQL

Generative AI

LLMs

Model Evaluation

Production ML

Data Pipelines

Agile

Communication

Consulting

Job description

Ascent has recently been acquired by Acuity Analytics. This is both a significant milestone for us and a tremendous opportunity for you. Acuity Analytics is a business with a strong global reputation, an impressive client base and ambitious growth plans. We deliver deep insights and domain-led digital transformation to high-growth and heavily regulated organisations. To our customers, we bring a partnership that provides the talent, technology and capability to enhance performance and operational efficiency.

About the role

We're looking for a Senior AI & Data Engineer (GenAI & LLM Evaluation) to help build and improve AI-powered data products.

Working alongside Data Scientists, Engineers, and Delivery teams, you'll develop scalable data pipelines, build GenAI evaluation frameworks, and deploy production-ready AI solutions. This is a hands-on role where you'll own solutions from design through to deployment while working on cutting-edge Generative AI technologies.

If you enjoy solving real-world problems and building practical AI solutions that make an impact, we'd love to hear from you.

Skills and experience required

Core skills

• 5+ years' experience in Data Engineering, Machine Learning, AI Engineering, or Data Science.
• Strong Python and SQL skills.
• Hands-on experience with Generative AI and Large Language Models (LLMs).
• Experience evaluating LLMs using golden datasets, model benchmarking, or LLM-as-a-Judge techniques.
• Experience building and deploying production-ready ML solutions.
• Strong understanding of data pipelines, validation, and quality frameworks.
• Experience integrating APIs and cloud-based AI services.
• Strong problem-solving and analytical skills.
• Excellent communication and stakeholder management.
• Experience working in Agile, cross-functional teams.
• Consulting or customer-facing experience is a plus.

What you will do

• Build and maintain scalable data pipelines and workflows.
• Develop evaluation frameworks for GenAI and LLM applications.
• Create and maintain golden datasets for model testing.
• Evaluate and benchmark LLM performance.
• Build ML-assisted tools to improve delivery and automation.
• Deploy production-ready AI solutions.
• Work closely with delivery teams to solve technical challenges.
• Continuously improve data quality, automation, and operational efficiency.

People are at the Heart of our Business. By investing in people, we achieve exceptional results for our clients and create new opportunities for our teams to thrive. Check out this page for more details.