Machine Learning Engineering Intern

Zendesk GmbH (Germany); Germany (OH); Hybrid; USD 2,755,000 - 4,959,000; Full time

Job summary

Zendesk GmbH is seeking a Machine Learning Engineering Intern to work on both back-end and front-end of the Zendesk Explore platform. You will join a team building features for AI Agents and be responsible for creating high-quality datasets to evaluate agent performance.

You will develop evaluation logic using Python and LLM-based techniques, iterating prompts and metrics to align automated scores with human feedback and improve agent behavior.

Qualifications

• Student pursuing a degree in a relevant field.
• Comfort with Python or TypeScript/JavaScript.
• Interest in AI, NLP, LLMs and evaluating AI agents.

Responsibilities

• Join a team building product features for AI Agents on Zendesk Explore.
• Build datasets and frameworks to measure AI agent quality.
• Create golden datasets from real-world conversations for benchmarking.
• Develop evaluation logic using LLM-as-Judge techniques and compare with human feedback.

Skills

Python

TypeScript/JavaScript

NLP / LLMs

Communication

Education

Student status

Job description

Job Description

At Zendesk, we geek out about building software that delivers the ultimate customer experience. Your favorite brands (like Airbnb, Uber, Slack, and Disney) use our products to engage you as a customer, and we lead the industry with beautifully simple software. And we’re constantly innovating - no really, we can’t wait to outdo ourselves in the near future.

We're looking for a passionate Machine Learning Engineering Intern to work on both the back-end and front-end of the Zendesk Explore platform.

What you will do:

• Be part of a team of engineers building product features for AI Agents
• Your primary mission is to ensure our evals are a true reflection of human judgment. You will be responsible for building the datasets and frameworks that tell us how good our AI agents really are.
• Create Golden Datasets: You will dive into real-world conversations to meticulously annotate and curate "golden datasets." These high-quality, benchmark datasets will be the source of truth for training and testing our agents.
• Develop LLM-as-Judge Evaluations: Using your Python skills, you will write and refine sophisticated evaluation logic within the Braintrust platform. You'll be using cutting-edge "LLM-as-Judge" techniques to assess the quality of our AI's conversations at scale.
• Iterate to Perfection: You'll constantly compare our automated evaluation scores with human feedback. Your goal is to iterate on the evaluation prompts and logic until we are confident that our automated metrics are directly improving the quality and behavior of our agents.

What you bring to the role:

• Our main stack is Python and TypeScript / JavaScript - you don’t need to be proficient in all of these but you should be comfortable in at least one.
• You're ambitious, highly motivated to understand complex AI systems.
• You have a genuine interest in Large Language Models (LLMs), NLP, and the challenges of evaluating AI agents.
• You can explain your views well to others and, just as importantly, actively listen to ensure a good discussion.
• Confident working in multiple languages both in the front-end and back-end of a SaaS application.
• You enjoy learning new technologies and growing your knowledge and capabilities.
• You are a student

Nice to have:

• Knowledge of AWS and other cloud services
• Experience in back-end data pipelines
• Any knowledge and/or experience in the analytics industry

#LI-MK12

The intelligent heart of customer experience

Zendesk software was built to bring a sense of calm to the chaotic world of customer service. Today we power billions of conversations with brands you know and love.

Zendesk believes in offering our people a fulfilling and inclusive experience. Our hybrid way of working, enables us to purposefully come together in person, at one of our many Zendesk offices around the world, to connect, collaborate and learn whilst also giving our people the flexibility to work remotely for part of the week.

As part of our commitment to fairness and transparency, we inform all applicants that artificial intelligence (AI) or automated decision systems may be used to screen or evaluate applications for this position, in accordance with Company guidelines and applicable law.

Zendesk is an equal opportunity employer, and we’re proud of our ongoing efforts to foster global diversity, equity, & inclusion in the workplace.

Individuals seeking employment and employees at Zendesk are considered without regard to race, color, religion, national origin, age, sex, gender, gender identity, gender expression, sexual orientation, marital status, medical condition, ancestry, disability, military or veteran status, or any other characteristic protected by applicable law.

We are an AA/EEO/Veterans/Disabled employer.

If you are based in the United States and would like more information about your EEO rights under the law, please click here.

Zendesk endeavors to make reasonable accommodations for applicants with disabilities and disabled veterans pursuant to applicable federal and state law.

If you are an individual with a disability and require a reasonable accommodation to submit this application, complete any pre-employment testing, or otherwise participate in the employee selection process, please send an e-mail to [e-mail] with your specific accommodation request.