Research Scientist - Multi-modal LLMs

Encord; London; On-site; GBP 55,000 - 80,000; Full time

Job summary

A leading company in AI infrastructure is seeking a Research Scientist to enhance their multi-modal LLM platform. The role involves problem-solving, developing scalable AI solutions, and improving user experiences. This is a unique opportunity to be part of an innovative team, grow professionally, and directly impact the future of AI technologies.

Benefits

Competitive salary and equity options

Collaborative culture with in-office work

25 days annual leave

Annual learning & development budget

Regular company lunches and socials

Qualifications

• PhD or strong academic background in machine learning.
• 2+ years experience in LLM fine-tuning and prompt engineering.
• Ambitious and self-motivated with communication skills.

Responsibilities

• Build and fine-tune multi-modal LLMs for data analysis.
• Develop ways to personalize LLMs from our data platform.
• Implement latest ML research to enhance user experience.

Skills

LLM fine-tuning

RAG systems

prompt engineering

machine learning

data analysis

problem-solving

Education

PhD or equivalent in machine learning

Tools

PyTorch

Tensorflow

JAX

Pandas

OpenCV

Job description

23 hours ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

About Us

At Encord, we're building the AI infrastructure of the future. One of the biggest challenges AI companies face today is data quality. The success of any AI application relies heavily on the quality of its training data, yet for most teams, this crucial step is both the most costly and time-consuming. We’re here to change that.As former computer scientists, physicists, and quants, we’ve experienced firsthand how a lack of tools to prepare quality training data impedes progress in building AI. We believe AI is at a stage similar to the early days of computing or the internet—where the potential is clear, but the surrounding tools and processes are still catching up. That's why we started Encord.

About Us

At Encord, we're building the AI infrastructure of the future. One of the biggest challenges AI companies face today is data quality. The success of any AI application relies heavily on the quality of its training data, yet for most teams, this crucial step is both the most costly and time-consuming. We’re here to change that.As former computer scientists, physicists, and quants, we’ve experienced firsthand how a lack of tools to prepare quality training data impedes progress in building AI. We believe AI is at a stage similar to the early days of computing or the internet—where the potential is clear, but the surrounding tools and processes are still catching up. That's why we started Encord.

We are a talented and ambitious team of 70, working at the cutting edge of computer vision and deep learning. Backed by $30M in Series B funding from top investors like CRV and Y Combinator, we’re one of the fastest-growing companies in our space. Our platform is consistently rated the best by our customers, and we have big plans ahead. We’re looking for a Research Scientist to help our customers get the right data faster, easier, and cheaper.

The Role

As a Research Scientist focusing on multi-modal LLMs, you'll be allowing all the data, metadata, and embeddings that live in our system to be explored, used, and analyzed in ways no one thought possible. Although starting narrow with “smaller” multi-modal problems like, e.g., improving similarity searches via metadata, we have high ambitions for this role. You'll progressively work on harder problems that will improve user experience, surface the right (personalized) analytics to every customer, and put our users in the driver's seat of a data development platform that can do things much beyond today’s standards. Tasks can be i) fine-tuning models to understand how our platform is used by customers, ii) employing LLM reasoning to assist customers in their data analysis tasks, and iii) Building tools for customers to interface naturally with our platform. All to put the power in the hands of anyone using Encord.

Role

You'll follow the latest research and accelerate state-of-the-art technologies to enrich customers’ data journeys. This role offers a great growth opportunity, with the potential to lead a bigger team of scientists over time in our efforts to build the ultimate data development platformWhat you will be doing:

• Building, fine-tuning, and experimenting with multi-modal LLMs to surface potential actions and analytical conclusions in a data-driven manner.
• Developing scalable and novel ways to personalize LLMs based on information from our data development platform.
• Build sophisticated RAG systems on other types of data than the usual text documents.
• Follow the latest machine learning research to identify and apply new methods that improve our processes or the user experience.
• Ensure our customers have the world’s most powerful AI-powered data development platform.
  
  

Skills For The Job

• A PhD or similarly strong academic background in machine learning, with 2+ years of hands-on experience in with LLM fine-tuning, RAG systems, and prompt engineering.
• Proficiency with frameworks like PyTorch, Tensorflow, JAX, Pandas, and OpenCV.
• A solid understanding of transformer models and their common variants, loss functions, and pitfalls.
• A quick learner with a structured, organized approach to problem-solving.
• Excellent communication skills with an ability to uncover use cases and solve problems efficiently.
• Ambitious and self-motivated, with a proven track record of top performance in academic or professional settings.
  
  

Bonus Skills

• Experience working with data in the order of millions.
• Familiarity with using (and adapting) models like LLaMa and LLaVa.
• Experience with image-to-text embedding models like CLIP and SigLIP.
• Familiarity with cloud-based model training and inference.
  
  

We encourage people from all backgrounds, cultures and skill levels to apply. It is okay to not meet all requirements listed as we are looking for individuals who are passionate, eager to learn.What We Offer- Competitive salary, commission, and equity in a high-growth business.- A collaborative, in-person culture with most of the team working in the office 3+ days a week (engineers typically work on-site Wednesdays).- 25 days annual leave + public holidays.- An annual learning and development budget to help you grow your skills.- Company lunches twice a week and regular socials, including bi-annual off-sites.

At Encord, you’ll have the unique opportunity to be part of a fast-growing startup with a clear mission and vision. You’ll work on real-world AI use cases across a variety of industry verticals and get hands-on experience with cutting-edge computer vision and deep learning technologies. This is a role where you'll grow quickly, take ownership of projects, and help shape the future of our company.

Seniority level

• Seniority level

  Entry level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Other

• Industries

  Software Development

Referrals increase your chances of interviewing at Encord by 2x

Get notified about new Research Scientist jobs in London, England, United Kingdom.

Enfield, England, United Kingdom 5 days ago

London, England, United Kingdom £29.44-£29.44 1 month ago

London, England, United Kingdom 1 day ago

Research Scientist (Chemistry) - AI Trainer

North Mymms, England, United Kingdom 2 weeks ago

London, England, United Kingdom 6 days ago

London, England, United Kingdom 4 days ago

London, England, United Kingdom 2 weeks ago

Teddington, England, United Kingdom 1 week ago

London, England, United Kingdom 3 days ago

Hampton, England, United Kingdom 1 week ago

Greater London, England, United Kingdom 1 week ago

Senior Biomedical Scientist (Night Shift Supervisor) - Biochemistry

London, England, United Kingdom 3 days ago

Greater London, England, United Kingdom 2 weeks ago

London, England, United Kingdom 1 week ago

London, England, United Kingdom 1 month ago

Croydon, England, United Kingdom 2 weeks ago

Greater London, England, United Kingdom 1 week ago

Senior Clinical Scientist Radiation Protection

London, England, United Kingdom 5 days ago

Harlow, England, United Kingdom 1 week ago

Research Scientist (Quantum Chemistry and Machine Learning), London

London, England, United Kingdom 6 days ago

Senior Clinical Scientist Radiation Protection

London, England, United Kingdom 2 weeks ago

Scientific Editor, Trends in Cancer and Trends Reviews

Croydon, England, United Kingdom 1 week ago

Scientific Editor, Joule and Matter, Cell Press

Hammersmith, England, United Kingdom 5 days ago

London, England, United Kingdom 6 days ago

Scientific Editor, Joule and Matter, Cell Press

London, England, United Kingdom 1 week ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.