Applied Scientist (AI Evaluation)

JR United Kingdom; Kingston upon Hull; Hybrid; GBP 50,000 - 80,000; Full time

Job summary

A dynamic startup is seeking an Applied Scientist specializing in AI evaluations to contribute to the development of adversarial tests for Large Language Models (LLMs). This role offers the chance to work closely with a fresh team of innovators, providing significant input on the project roadmap and a competitive compensation package that includes salary, bonuses, and share options. Flexibility in work arrangements is encouraged, catering to both on-site and remote applicants within the UK or EU.

Benefits

Competitive salary

Bonus and share options

Flexible/remote work environment

Company issued laptop

Tech allowance

Travel allowance for meet-ups

Qualifications

• PhD focused on Natural Language Processing required.
• Experience with LLM benchmarking and data set creation needed.
• Publication in top-tier journals is a plus.

Responsibilities

• Design, develop and deploy evaluation technologies for LLMs.
• Communicate results effectively to both technical and non-technical teams.
• Collaborate with engineers and product partners on projects.

Skills

Natural Language Processing

Deep Learning Model Architecture Design

Programming in Python

Algorithm Development

AWS Deployment

Strong Communication Skills

Education

PhD in Computer Science or related field

Tools

Git

HuggingFace

Job description

Social network you want to login/join with:

Applied Scientist (AI Evaluation), kingston upon hull, east yorkshire

col-narrow-left

Client:

Trismik

Location:

kingston upon hull, east yorkshire, United Kingdom

Job Category:

Other

-

EU work permit required:

Yes

col-narrow-right

Job Views:

7

Posted:

06.06.2025

Expiry Date:

21.07.2025

col-wide

Job Description:

At Trismik we're a team of tech geeks from the University of Cambridge, Salesforce, and Amazon looking to push the boundaries of AI through science-led evaluations. If you're ambitious to make a difference to the future of AI, have a PhD in NLP, and like to turn ideas into reality, we'd love to hear from you.

Role

We are developing adversarial tests for Large Language Models (LLMs). We are looking for a passionate, talented, and innovative applied scientist with a strong background in algorithms to help build an industry-leading evaluation engine for LLMs and help us bring this to market. As one of our first hires this is a high-impact and high-ownership role. You will have a strong say in how our science and product roadmap evolves.

Our mission is to provide the fastest and most accurate testing environment to add value to AI engineers wishing to deploy AI applications using LLMs. We do this to push forward the SoTA in Artificial Intelligence and to achieve the best possible chance of a human-aligned AGI.

Key job responsibilities

As an Applied Scientist in a startup you will have a key role in our team. You will work with the Chief Scientist to design, develop and deploy evaluation technologies that will create high value insights for AI engineers. These will involve providing assessments for several technologies that involve Large Language Models, including retrieval augmented systems (RAGs), recommender systems, and agentive systems. You will:

• Invent, experiment with, and launch new features, products and systems based on machine learning and MLLM.
• Perform hands-on construction and analysis of large-scale multi-modal datasets to be deployed as part of our product offering.
• Build algorithms, perform offline and A/B test experiments, optimise and deploy your models into production, working closely with software engineers.
• Establish automated processes for large-scale data analysis and generation, machine-learning model development, model validation and serving.
• Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports, and publish your work for internal and external audiences, e.g. through white papers and conference papers.
• Collaborate with engineers, product partners locally and abroad, and join conversations with customers led by our marketing team.

Essential skills

• PhD in Computer Science or related field (Natural Language Processing focus only)
• Experience in state-of-the-art deep learning models architecture design and deep learning training and optimization and model pruning
• Experience with LLM benchmarking
• Strong programming skills in Python, Java, C++, or a related language
• Strong skills in algorithm development to solve optimisation problems
• Familiarity with core tools for a typical ML-focused work environment (e.g. Git, IDE, common libraries, HuggingFace).
• Ability to work as part of a team
• Be comfortable with a fast-paced, high-risk, and a high reward environment
• Experience creating data sets
• Experience coordinate development of products with multiple stakeholders
• Publications in top-tier peer-reviewed journals and conferences (e.g. NeurIPS, EMNLP, ACL)
• Experience deploying solutions to AWS or other cloud platforms
• Excellent communication skills, solid work ethic, a strong desire to write production-quality code

Our offer

• Competitive salary, bonus, and share options in our startup;
• Company issued laptop, workstation, and tech allowance;
• Flexible/remote work environment. We are based in Cambridge/London but open to outstanding remote work candidates based in the UK or EU. We offer travel allowance to enable meet ups;