AI Observability Engineer

Moring AI

Town of Newark (WI)

Vor Ort

USD 130.000 - 190.000

Vollzeit

Vor 2 Tagen
Sei unter den ersten Bewerbenden

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Moring.ai is building an AI observability platform for a large financial services enterprise. You will own the observability architecture across logs, traces, metrics, and AI-specific signals, and stand up LangSmith and related tools inside the customer environment.

This is a hands-on IC role in Newark, NJ. You will instrument the platform with OpenTelemetry, build pipelines, dashboards for cost and latency, and ensure on-call readiness with runbooks and SLOs.

Qualifikationen

  • 6+ years in software or platform engineering with a deep focus on observability.
  • Strong fundamentals in logs, traces, and metrics and how they are produced, collected, stored, queried, and used.
  • Deep OpenTelemetry experience: SDKs, collector, instrumentation, semantic conventions, sampling, context propagation.
  • Experience building observability platforms for cloud-native systems: Kubernetes, microservices, and related tools.

Aufgaben

  • Own the observability architecture for the AI platform: logs, traces, metrics, and AI-specific signals.
  • Stand up and operate LangSmith and related tools inside the customer environment and integrate with existing observability stack.
  • Instrument the platform and agents with OpenTelemetry across traces, LLM calls, tool usage, and retrieval steps.
  • Build OTel collector pipelines: collection, processing, sampling, redaction, and export to backends.
  • Develop tooling around observability: SDKs and decorators for tracing, dashboards for cost and latency, alerting on errors and spend.
  • Connect observability to evaluation: produce production traces for eval datasets and surface quality regressions.
  • Build cost and usage accounting: tokens and spend per model, team, feature, agent.
  • Ensure observability works in restricted enterprise networks: private endpoints, self-hosted backends, data retention rules.
  • Define SLOs and runbooks for on-call engineers and improve them from incidents.
  • Educate team and customer engineers on how to use the system.

Kenntnisse

Observability
OpenTelemetry
Kubernetes
Python
AWS
On-call
Enterprise IT
Explain traces to non-engineers

Tools

Prometheus
Grafana
Jaeger
Loki
Datadog
Tempo
Arize Phoenix
LangSmith

Jobbeschreibung

Newark, United States | Posted on 11/15/2026

Moring.ai builds enterprise AI that runs in production for regulated industries: financial services and other compliance-heavy organizations. We ship AI applications and agents inside our customers' own cloud environments, wired into the systems they already run. Not slideware, not demos.

We're early-stage, lean, and profitable, so what you ship reaches serious enterprises fast.

The Role

We are building an AI platform for a large financial services enterprise. Developers inside that organization use it to build and deploy agents: Python applications that call LLMs, use tools, and run on Kubernetes. When one of those agents is slow, expensive, wrong, or misbehaving, someone has to be able to see exactly what happened: which model it called, what it sent, which tools it used, how long each step took, and what it cost. That is the job.

We are hiring an AI Observability Engineer to own observability for the platform. You are an observability platform person first. You have very strong fundamentals in logs, traces, and metrics, you know OpenTelemetry in depth, and you have built or run observability for cloud-native systems at real scale. You will apply that to AI: standing up LangSmith and related tools, instrumenting agents and the platform with OTel, and building the tooling around it so that engineers, the customer's operations teams, and auditors can all see what the AI is doing.

This is a hands-on IC role.

What You'll Do
  • Own the observability architecture for the AI platform: logs, traces, metrics, and AI-specific signals, end to end.
  • Stand up and operate LangSmith (and related tools such as Langfuse or Arize Phoenix where they fit) inside the customer's environment, and integrate them with the customer's existing observability stack.
  • Instrument the platform and the agents with OpenTelemetry: traces of agent reasoning, LLM calls, tool and MCP calls, and retrieval steps, with consistent semantic conventions across services.
  • Build the OTel collector pipelines: collection, processing, sampling, redaction of sensitive data, and export to the right backends, within the customer's data-handling rules.
  • Build the tooling around observability: SDK helpers and decorators so engineers get tracing for free, dashboards for cost, latency, token usage, and quality, alerting on error rates and spend, and reports an audit team can read.
  • Connect observability to evaluation: capture production traces so they can be turned into eval datasets, and surface quality regressions.
  • Build the cost and usage accounting: tokens and spend per model, per team, per feature, per agent.
  • Make observability work in a restricted enterprise network: private endpoints, no-egress environments, self-hosted backends, data retention rules.
  • Define the SLOs for the platform and the runbooks for the on-call engineers, and improve them from real incidents.
  • Teach the team and the customer's engineers how to use what you built.
What We're Looking For
  • 6+ years in software or platform engineering with a deep focus on observability.
  • Very strong fundamentals in logs, traces, and metrics: how they are produced, collected, stored, queried, and correlated, and what each is good and bad for.
  • Deep OpenTelemetry experience: the SDKs, the collector, instrumentation libraries, semantic conventions, sampling, and context propagation. You have designed OTel pipelines, not just enabled an agent.
  • You have built or run observability platforms for cloud-native systems: Kubernetes, microservices, and the tools around them (Prometheus, Grafana, Loki or Elastic, Jaeger or Tempo, Datadog, Splunk, or comparable).
  • Hands‑on experience with LangSmith or a comparable LLM observability tool (Langfuse, Arize Phoenix, Braintrust, Weights and Biases Weave), and an understanding of what is different about observing LLM applications: non‑determinism, token cost, prompt and response capture, tool calls, and evaluation.
  • Strong Python; you will build tooling, instrumentation, and collector components.
  • Good working knowledge of AWS (EKS, CloudWatch, IAM, networking) and Kubernetes.
  • You have been on call for systems you instrumented, and your dashboards and alerts got better because of it.
  • Experience inside large enterprises is a plus, regulated ones (financial services, insurance, healthcare) more so; you understand data redaction and retention requirements.
  • You can explain to a non‑engineer what a trace shows and why it matters.
  • Location: Newark, New Jersey(on‑site)
  • Years of experience: 3-6years
  • Travel: Up to 10% to customer sites
  • Work authorization: Must be authorized to work in the United States (GC/EAD/US Citizen)
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

AI Observability Engineer
AI Observability Engineer

Moring AI • Texas

Vor Ort
USD 120.000 - 160.000
AI Observability Engineer
AI Observability Engineer

Moring AI • Raleigh (NC)

Vor Ort
USD 120.000 - 180.000
AI Observability Engineer
AI Observability Engineer

Moring AI • Alpharetta (GA)

Vor Ort
USD 140.000 - 180.000
AI Observability Engineer
AI Observability Engineer

Moring, inc. • Alpharetta (GA)

Vor Ort
USD 120.000 - 180.000
AI Observability Engineer Engineering Alpharetta, GA, USA (on-site) →
AI Observability Engineer Engineering Alpharetta, GA, USA (on-site) →

Moring, inc. • Alpharetta (GA)

Vor Ort
USD 60.000 - 90.000
Stock options
AI Observability Engineer — OpenTelemetry & LangSmith
AI Observability Engineer — OpenTelemetry & LangSmith

Moring AI • Texas

Vor Ort
USD 120.000 - 160.000
AI Engineer
AI Engineer

Moring AI • Town of Newark (WI)

Vor Ort
USD 60.000 - 80.000
Site Reliability Engineer
Site Reliability Engineer

TQL • Cincinnati (OH)

Vor Ort
USD 110.000 - 180.000
AI Engineer
AI Engineer

Moring AI • Raleigh (NC)

Vor Ort
USD 60.000 - 80.000
AI Engineer
AI Engineer

Moring AI • Texas

Vor Ort
USD 60.000 - 80.000