Senior ML Engineer: AI-Native Search & Evaluation

Atlassian; San Francisco, Northern (CA, KY); Hybrid; USD 185,000 - 242,000; Full time

Job summary

Atlassian seeks a Senior Machine Learning Engineer to join the AI-Native Evaluation & Search team in the US. You will build production ML models for search, retrieval, ranking, and AI agents, owning projects end-to-end from concept to deployment.

You will define evaluation datasets and benchmarks, analyze model behavior, and guide product decisions with results. You will collaborate with product managers, software engineers, data scientists, researchers, and platform teams while mentoring peers

Qualifications

• Bachelor's or Master's degree in Computer Science, ML, Data Science, or related field or equivalent experience.
• 4+ years of producing ML/AI systems in production.
• Strong Python and production languages such as Java, Kotlin or TypeScript.
• Experience with SQL and large-scale data processing like Spark.
• Experience building, evaluating, deploying, and scaling models with large datasets.
• Design evaluation strategies and use results to guide improvements.
• Familiarity with cloud-based ML/data platforms (AWS, Databricks).

Responsibilities

• Build and productionize ML models and AI systems for search and intelligence products.
• Own projects from concept through deployment, monitoring, evaluation, and iteration.
• Develop AI-native experiences across search, retrieval, ranking, and conversational systems.
• Design evaluation-driven development processes with quality metrics and benchmarks.
• Build evaluation datasets, regression suites, and pipelines for search, RAG, chat, and agents.
• Conduct offline and online evaluations, human assessments, and error analysis.
• Design scalable architectures for quality, reliability, latency, privacy, and cost.
• Apply information retrieval, embeddings, NLP, deep learning and LLMs.
• Communicate decisions clearly and mentor other engineers.

Skills

Python

Java

Kotlin

TypeScript

SQL

Spark

Production ML

Communication

Agile mindset

Education

Bachelor's or Master's in CS/ML/Data Science

Tools

Databricks

AWS

Job description

Atlassian seeks a Senior Machine Learning Engineer to join the AI-Native Evaluation & Search team in the US. You will build production ML models for search, retrieval, ranking, and AI agents, owning projects end-to-end from concept to deployment.

You will define evaluation datasets and benchmarks, analyze model behavior, and guide product decisions with results. You will collaborate with product managers, software engineers, data scientists, researchers, and platform teams while mentoring peers