Senior AI Product Engineer, Backend Engineering (Remote)

Arize AI, Inc; Northern (KY); Hybrid; USD 125,000 - 225,000; Full time

Benefits offered by this job

Remote-first

Office options NYC/SF

Equity opportunity

Wellness benefits

Job summary

Arize AI is seeking an experienced Backend Engineer to help scale our ML observability platform. You will build scalable services in Go, with Python and Java, and may touch TypeScript/Node.

You’ll design APIs, implement analytics pipelines, and contribute to in-house tools powering AI product teams. This remote-first role offers a salary range of $125k–$225k plus equity; offices in NYC and SF are available.

Qualifications

• 5+ years of experience building high-performance backend systems.
• Strong experience with Go, Python, TypeScript/Node, or Java.
• Interest in AI/LLM ecosystems and SaaS platforms.
• Experience with public clouds and container orchestration (AWS, GCP, Azure, Kubernetes).

Responsibilities

• Write maintainable, scalable backend code in Go, Java, Python, with TypeScript as needed.
• Build high-volume analytics systems and design customer APIs for ML/LLM workflows.
• Prototype and maintain scalable backend services powering the core platform.
• Contribute to open-source OLAP databases and distributed frameworks.
• Collaborate with product, design, and customer engineering teams.

Skills

Go

Python

TypeScript/Node

Java

Backend systems

Tools

Kubernetes

AWS

GCP

Azure

Docker

Prometheus

Job description

AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. That’s where we come in. Arize AI is the leading AI & Agent Engineering observability and evaluation platform, empowering AI engineers to ship high-performing, reliable agents and applications. From first prototype to production scale, Arize AX unifies build, test, and run in a single workspace—so teams can ship faster with confidence.

We’re a Series C company backed by top-tier investors, with over $135M in funding and a rapidly growing customer base of 150+ leading enterprises and Fortune 500 companies. Customers like Booking.com , Uber, Siemens, and PepsiCo leverage Arize to deliver AI that works.

The Opportunity

Our Backend Engineering team builds all of the highly scalable distributed services that power Arize’s ML observability platform. While Go is our primary language for these distributed systems, the team also maintains services and tools written in Python, Java, and TypeScript. The expectation and scope of every individual on this team is high, whether it’s finding the most efficient way to compute model evaluation metrics across billions of data points, designing the next generation of our OLAP database architecture, or researching and implementing the latest dimensionality reduction techniques – you will never lack a technical challenge.

You will be a part of the core team that drives product innovation at Arize. You will be challenged with understanding how some of the most impactful engineering teams are developing AI and LLM-powered applications, and how to build the right tools to enable them to do their best work. Our product solutions range from clean APIs that magically instrument applications, interactive playgrounds for prompt engineering and agent development, or scaling up real-time evaluation infrastructure to handle millions of annotations per second.

What You’ll Do

• Write maintainable, scalable, and performant backend code primarily in Go, Java, and Python, with opportunities to work in TypeScript.
• Build high-volume and highly available analytics systems.
• Design and build APIs specific to our customers’ Machine Learning and LLM workflows.
• Prototype, optimize, and maintain scalable backend services that power the Arize core platform.
• Extend, and contribute back to, open source OLAP databases and distributed message queue frameworks.
• Develop and integrate collection tools for robust monitoring of ML and LLM pipelines.
• Research and implement cutting-edge visualization & dimensionality reduction algorithms in a distributed environment.
• Collaborate with our product, design, and directly with customer engineering teams to enhance and expand our product offerings.
• Contribute to the build our own in-house AI Agents

What We’re Looking For

• 5+ years of experience working with high-performance backend systems.
• Strong experience writing Go, Python, TypeScript/Node, Java, or similar server programming languages.
• Enthusiasm and interest in the AI and LLM ecosystem, with a desire to learn and stay updated on emerging technologies.
• Previous work building and operating highly complex SaaS platforms/systems.
• Knowledge of working with public clouds & container orchestration - AWS, GCP, Azure, Kubernetes, etc.

Bonus Points, But Not Required

• Experience with distributed stream processing - Kafka, Gazette, or similar.
• Experience with OLAP systems.
• Familiarity with system observability tooling like Prometheus.
• Working knowledge of Machine Learning and/or Data Science.
• First‑hand experience working with large language models (LLMs) or developing AI products.

The estimated annual salary for this role is between $125,000 - $225,000, plus a competitive equity package. Actual compensation is determined based on a variety of job-related factors that may include transferable work experience, skill sets, and qualifications. Total compensation also includes a comprehensive benefits package, including medical, dental, vision, a 401(k) plan, unlimited paid time off, a generous parental leave plan, and additional support for mental health and wellness.

While we are a remote‑first company, we have opened offices in New York City and the San Francisco Bay Area, as an option for those in those cities who wish to work in-person. For all other employees, there is a WFH monthly stipend to pay for co‑working spaces.

More About Arize

Arize’s mission is to make the world’s AI work—and work for people.
Our founders came together through a shared frustration: while investments in AI are growing rapidly across every industry, organizations face a critical challenge—understanding whether AI is performing and how to improve it at scale.

Arize’s mission is to make AI work and make AI work for the people, we hope to make an impact in bias industry‑wide and that's a big motivator for people who work here. We actively hope that individuals contribute to a good culture

• Regularly have chats with industry experts, researchers, and ethicists across the ecosystem to advance the use of responsible AI
• Culturally conscious events such as LGBTQ trivia during pride month
• We have an active Lady Arizers subgroup

U.S. Equal Opportunity Employment Information (Completion is voluntary)

Individuals seeking employment at Arize AI are considered without regards to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation. You are being given the opportunity to provide the following information in order to help us comply with federal and state Equal Employment Opportunity/Affirmative Action record keeping, reporting, and other legal requirements. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

How would you describe your gender identity?

Male Female Decline to Self Identify

Yes No Decline To Self Identify

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

How do you know if you have a disability?

You are considered to have a disability if you have a physical or mental impairment or medical condition that substantially limits a major life activity, or if you have a history or record of such an impairment or medical condition.

Disabilities include, but are not limited to:

• Autism
• Autoimmune disorder, for example, lupus, fibromyalgia, rheumatoid arthritis, or HIV/AIDS
• Blind or low vision
• Cancer
• Cardiovascular or heart disease
• Celiac disease
• Cerebral palsy
• Deaf or hard of hearing
• Depression or anxiety
• Diabetes
• Epilepsy
• Gastrointestinal disorders, for example, Crohn's Disease, or irritable bowel syndrome
• Missing limbs or partially missing limbs
• Nervous system condition for example, migraine headaches, Parkinson’s disease, or Multiple sclerosis (MS)
• Psychiatric condition, for example, bipolar disorder, schizophrenia, PTSD, or major depression

1 Section 503 of the Rehabilitation Act of 1973, as amended. For more information about this form or the equal employment obligations of Federal contractors, visit the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp .

PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete.