Senior Software Engineer, AI Benchmarking

Remotedxb; Dubai; On-site; AED 240,000 - 420,000; Full time

Job summary

SecureBio in Dubai is seeking an experienced software engineer to join a mission-driven team focused on biosecurity evaluations of AI models. You will build adapters, scale evaluation suites, and improve tooling for scalable capability assessments.

The role emphasizes Python proficiency, Docker and AWS experience, and a strong commitment to coding quality within a small, fast-moving team and mission-aligned work.

Qualifications

• 5+ years of software engineering experience (industry or open-source).
• Fluency in Python.
• Proficient with Docker and AWS, especially ECS.
• Experience building tools that use or evaluate LLMs or AI agents.
• Self-directed and comfortable working on small, fast-moving teams.
• Commitment to good coding practices and code quality.
• Alignment with SecureBio's mission of preventing catastrophic pandemics.

Responsibilities

• Perform pre-release and post-release assessments of AI models using SecureBio's suite of evaluations.
• Build adapters for models, identify and troubleshoot unexpected agent behaviours, and ensure methodological rigor.
• Build, scale, maintain, and continuously improve SecureBio's suite of biosecurity evaluations.
• Improve evaluations to use state-of-the-art agent frameworks, jailbreaking methods, and elicitation strategies.
• Develop and improve internal tools that enable researchers to run capability assessments at scale.
• Contribute to and maintain cloud infrastructure to run evaluations and analyses.
• Contribute analyses to the public dashboard of trends in AI biology capabilities.

Skills

Python

Software engineering

LLMs / AI agents

Self-directed

Code quality

Open-source collaboration

Mission alignment

Tools

Docker

AWS ECS

Terraform

AWS CDK

OpenAI APIs

Codex / Claude

React + TypeScript

Job description

Responsibilities

• Perform pre-release and post-release assessments of frontier and open-source AI models using SecureBio's suite of biosecurity evaluations
• Build adapters for models, identify and troubleshoot unexpected agent behaviors, and ensure methodological rigor
• Build, scale, maintain, and continuously improve SecureBio's suite of biosecurity evaluations
• Improve evaluations to use state-of-the‑art agent frameworks, jailbreaking methods, and elicitation strategies
• Develop and improve internal tools that enable researchers to run capability assessments at scale
• Contribute to and maintain cloud infrastructure to run evaluations and analyses
• Contribute analyses to the public dashboard of trends in AI biology capabilities

Requirements

• 5+ years of experience in software engineering (industry or open‑source)
• Fluency in Python
• Proficiency with Docker and AWS (particularly ECS)
• Experience building software tools that use or evaluate LLMs or AI agents
• Self‑directed and comfortable working on small, fast‑moving teams
• Commitment to good coding practices and code quality
• Alignment with SecureBio's mission of preventing catastrophic pandemics

Preferred Qualifications

• Experience with AI evaluations (agent systems, scorer/grader design, trajectory analysis, jailbreaking, or red‑teaming)
• Experience with Terraform, Tofu, or AWS CDK
• Experience with modern, component‑based web frontends (React with TypeScript)
• Experience with Inspect AI, OpenAI‑compatible APIs, Anthropic, or Together
• Experience with agent frameworks like Codex or Claude Code

About the Company

SecureBio is a mission‑driven team dedicated to protecting the world from biological threats. Through research and development, SecureBio builds the tools and technologies needed to secure the world against future biotechnology risks, including pandemic detection and reducing biological risks from advanced artificial intelligence.