AI Observability Engineer — Platform & FinServ

Moring, inc.; Alpharetta (GA); In loco; USD 120.000 - 180.000; Tempo pieno

Descrizione del lavoro

Moring.ai is building an AI platform in Alpharetta, GA for a large financial services enterprise. We seek an AI Observability Engineer to own the platform’s observability, instrument agents and the platform with OpenTelemetry, and stand up LangSmith and related tools within a customer environment.

This hands-on IC role focuses on logs, traces, metrics, and cost-aware dashboards. You will work with LangSmith/Langfuse, cloud-native tooling, and AWS/Kubernetes, enabling auditors and ops teams to

Competenze

• 6+ years in software or platform engineering with a deep focus on observability.
• Very strong fundamentals in logs, traces, and metrics: how they are produced, collected, stored, queried, and correlated, and what each is good and bad for.
• Deep OpenTelemetry experience: the SDKs, the collector, instrumentation libraries, semantic conventions, sampling, and context propagation. You have designed OTel pipelines, not just enabled an agent.
• You have built or run observability platforms for cloud-native systems: Kubernetes, microservices, and the tools around them (Prometheus, Grafana, Loki or Elastic, Jaeger or Tempo, Datadog, Splunk, or comparable).
• Hands‑on experience with LangSmith or a comparable LLM observability tool (Langfuse, Arize Phoenix, Braintrust, Weights and Biases Weave), and an understanding of what is different about observing LLM applications: non-determinism, token cost, prompt and response capture, tool calls, and evaluation.
• Strong Python; you will build tooling, instrumentation, and collector components.
• Good working knowledge of AWS (EKS, CloudWatch, IAM, networking) and Kubernetes.
• You have been on call for systems you instrumented, and your dashboards and alerts got better because of it.
• Experience inside large enterprises is a plus, regulated ones (financial services, insurance, healthcare) more so; you understand data redaction and retention requirements.
• You can explain to a non-engineer what a trace shows and why it matters.
• Years of experience: 3-6years
• Travel: Up to 10% to customer sites
• Work authorization: Must be authorized to work in the United States (GC/EAD/US Citizen)

Mansioni

• Own the observability architecture for the AI platform: logs, traces, metrics, and AI-specific signals, end to end.
• Stand up and operate LangSmith (and related tools such as Langfuse or Arize Phoenix where they fit) inside the customer's environment, and integrate them with the customer's existing observability stack.
• Instrument the platform and the agents with OpenTelemetry: traces of agent reasoning, LLM calls, tool and MCP calls, and retrieval steps, with consistent semantic conventions across services.
• Build the OTel collector pipelines: collection, processing, sampling, redaction of sensitive data, and export to the right backends, within the customer's data-handling rules.
• Build the tooling around observability: SDK helpers and decorators so engineers get tracing for free, dashboards for cost, latency, token usage, and quality, alerting on error rates and spend, and reports an audit team can read.
• Connect observability to evaluation: capture production traces so they can be turned into eval datasets, and surface quality regressions.
• Build the cost and usage accounting: tokens and spend per model, per team, per feature, per agent.
• Make observability work in a restricted enterprise network: private endpoints, no-egress environments, self-hosted backends, data retention rules.
• Define the SLOs for the platform and the runbooks for the on-call engineers, and improve them from real incidents.
• Teach the team and the customer's engineers how to use what you built.

Conoscenze

Observability fundamentals

OpenTelemetry

Kubernetes

Python

Cloud-native systems

LLM observability

Strumenti

LangSmith

Langfuse

Arize Phoenix

Descrizione del lavoro

Moring.ai is building an AI platform in Alpharetta, GA for a large financial services enterprise. We seek an AI Observability Engineer to own the platform’s observability, instrument agents and the platform with OpenTelemetry, and stand up LangSmith and related tools within a customer environment.

This hands-on IC role focuses on logs, traces, metrics, and cost-aware dashboards. You will work with LangSmith/Langfuse, cloud-native tooling, and AWS/Kubernetes, enabling auditors and ops teams to