Senior Backend Engineer, AI Evaluation Systems (Remote)

United States Digital Space LLC; United States; On-site; USD 120,000 - 170,000; Full time

Job summary

United States Digital Space LLC is seeking a Senior Software Engineer to join the AI Assistant Product Evaluation team. You will design scalable data systems and evaluation platforms for agentic AI products, turning interactions and model outputs into reliable signals for product and model development.

You will partner with modeling teams, product managers, data scientists and operations to ship trustworthy AI experiences, owning architecture, implementation, testing, and observability across

Qualifications

• 5+ years of industry experience as a software engineer, backend engineer, platform engineer, or data-focused software engineer building production systems.
• Strong programming skills in Python, with experience building production-quality software, libraries, frameworks, or data processing systems.
• Experience designing and operating scalable data pipelines using Airflow or similar orchestration frameworks; experience with Spark, Flink, Kafka, Trino/Presto, Hive, Iceberg, or similar big data technologies is a strong plus.
• Strong understanding of data modeling, schema design, data quality, partitioning, indexing, storage formats, and tradeoffs for large-scale analytical and operational data systems.
• Experience building data layers, evaluation systems, feedback loops, experimentation platforms, observability tools, or AI/ML platform capabilities.
• Proven ability to work cross-functionally with modeling engineers, product managers, data scientists, infrastructure teams, and operations partners to deliver end-to-end solutions.

Responsibilities

• Design and productionize scalable data systems that support AI evaluation, metric computation, observability, and feedback loops for agentic AI products.
• Build data models, schemas, and processing pipelines for agentic AI interactions, supporting reliable logging, retrieval, metric computation, and long-term evaluation dataset management.
• Work closely with Core Modeling engineers to understand pain points in the LLM evaluation process, and develop LLM-as-a-judge solutions and data pipelines to address metric-related challenges in a scalable and efficient way.
• Collaborate with machine learning infrastructure engineering teams to evolve how we build and test evaluation framework for the company Conversational AI products.
• Lead all phases of software development including architecture design, implementation and testing.
• Work collaboratively with cross-functional partners including product managers, operations and data scientists, identify opportunities for business impact, understand and prioritize requirements for machine learning systems and data pipelines, drive engineering decisions and quantify impact.
• Foster a culture of engineering excellence by supporting teammates in writing high-quality code, ensuring operational reliability, and sharing knowledge across the team.

Skills

Python programming

Data pipelines

System design

Backend engineering

Airflow

Spark

Kafka

Cross-functional collaboration

Observability

Education

BS/MS/PhD in CS or Engineering

Tools

Airflow

Spark

Kafka

Trino/Presto

Hive

Iceberg

Job description

United States Digital Space LLC is seeking a Senior Software Engineer to join the AI Assistant Product Evaluation team. You will design scalable data systems and evaluation platforms for agentic AI products, turning interactions and model outputs into reliable signals for product and model development.

You will partner with modeling teams, product managers, data scientists and operations to ship trustworthy AI experiences, owning architecture, implementation, testing, and observability across