Senior Software Engineer - IT

Dell Inc.; Karnataka; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

Dell Technologies in Bangalore is seeking an Agent Reliability Engineer to lead the engineering standard for production security workflows. You will own the hardest implementation challenges on the Security Resiliency team and ensure agentic security processes run reliably at scale.

In this senior IC role, you will build observability, cost-tracking, and real-time dashboards, employing OpenTelemetry, Grafana, Datadog, Splunk, Kubernetes, Docker, PostgreSQL, Elasticsearch, and LangFuse to monitor

Qualifications

• 5+ years in site reliability engineering, production operations or platform engineering.
• Hands-on knowledge of performance-measuring and observability tools (OpenTelemetry, Grafana, Datadog, Splunk).
• Python proficiency to build monitoring tooling and automate operational tasks.
• Working knowledge of Kubernetes and Docker for container ops, with IaC familiarity, plus PostgreSQL/Elasticsearch.
• Cost and consumption tracking for metered services, including LLM token accounting and forecasting.

Responsibilities

• Operate agentic security workflows in production, sustaining availability, completion rate, latency and cost within defined service levels.
• Build and operate the observability layer: traces, logging, dashboards, backed by PostgreSQL and Elasticsearch.
• Measure token consumption and cost per agent and workflow using LangFuse and report actual costs.
• Detect agent-specific failure modes and respond to restore service on Kubernetes/Docker deployments.
• Triage production issues to root cause and maintain runbooks for knowledge sharing.

Skills

SRE experience

Python proficiency

Observability mindset

Tools

OpenTelemetry

Grafana

Datadog

Splunk

Kubernetes

Docker

PostgreSQL

Elasticsearch

LangFuse

Job description

Agent Reliability Engineer

The Security Resiliency Organization is rebuilding fragmented, manually-run security processes into consolidated, AI-native workflows that run in production at scale. As the most senior IC on the applied AI team, you will set the engineering standard the rest of the team builds against and own the hardest implementation problems.

Join us as an Agent Reliability Engineer on our Security Resiliency team in Bangalore to do the best work of your career and make a profound social impact.

What you’ll achieve

• You will keep agentic security workflows operating reliably in production, restoring service quickly when performance degrades.
• You will build the observability and performance-measurement layer that shows, in real time, how every workflow is running and what it costs.
• You will make monitoring a routine step for every new workflow rather than a bespoke effort each time.

You will:

• Operate agentic security workflows in production, sustaining availability, completion rate, latency and cost within defined service levels, using performance-measuring tools such as OpenTelemetry, Grafana, Datadog or Splunk.
• Build and operate the observability layer: execution traces across agent steps and tool calls, structured logging, health signals and dashboards, backed by PostgreSQL and Elasticsearch for metrics and log storage.
• Measure token consumption and cost per agent, workflow and run using tools such as LangFuse, reporting actual cost against expectation so overruns are visible early.
• Detect agent-specific failure modes – stalled or looping executions, tool call failures, credential and upstream API changes, silent quality degradation – and respond to restore service on Kubernetes/Docker-based deployments.
• Triage production issues to root cause, elevate defects to the building team with traces and reproduction detail, and maintain runbooks so operational knowledge does not depend on one person.

Take the first step towards your dream career

Every Dell Technologies team member brings something unique to the table. Here’s what we are looking for with this role:

Essential Requirements

• 5+ years in site reliability engineering, production operations or platform engineering, with direct responsibility for systems in production.
• Explicit, hands-on knowledge of performance-measuring and observability tools such as OpenTelemetry, Grafana, Datadog or Splunk – this is mandatory, not preferred, for this role.
• Python (or equivalent) proficiency to build monitoring tooling, automate operational tasks and read the code of the systems being operated.
• Working knowledge of Kubernetes and Docker for container operations, with infrastructure-as-code familiarity, plus PostgreSQL and Elasticsearch for metrics/log storage and querying.
• Cost and consumption tracking for metered services, including LLM token accounting, usage attribution and forecasting against quota or budget.

Desirable Requirements

• Direct experience with LLM tracing tools such as LangFuse for agent-run performance measurement, in addition to general observability platforms.
• Experience with LLM provider rate limits, quota management and multi-provider fallback, and runbook automation or failure-injection testing.

Who we are

We believe that each of us has the power to make an impact. That’s why we put our team members at the center of everything we do. If you’re looking for an opportunity to grow your career with some of the best minds and most advanced tech in the industry, we’re looking for you.

Dell Technologies is a unique family of businesses that helps individuals and organizations transform how they work, live and play. Join us to build a future that works for everyone because Progress Takes All of Us.

Dell Technologies is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. Read the full Equal Employment Opportunity Policy here.