Senior Software Engineer - IT

Dell Inc.; Bengaluru; On-site; INR 2,600,000 - 5,200,000; Full time

Job summary

Dell Technologies is seeking an Agent Reliability Engineer in Bengaluru to lead reliability of agentic security workflows and set engineering standards. You will craft observability layers, monitor performance and cost, and ensure readiness of deployments in Kubernetes/Docker environments, while driving fast resolution of production issues to protect service levels.

You will build real-time traces, logs, and dashboards, using OpenTelemetry, Grafana, Datadog or Splunk, and manage token accounting

Qualifications

• 5+ years in site reliability engineering, production operations or platform engineering.
• Hands-on knowledge of performance-measuring and observability tools (OpenTelemetry, Grafana, Datadog or Splunk) is mandatory.
• Python proficiency to build monitoring tooling and automate operational tasks.
• Working knowledge of Kubernetes and Docker, plus PostgreSQL/Elasticsearch for metrics and logs.
• Cost and consumption tracking for metered services, including LLM token accounting and forecasting.

Responsibilities

• Operate agentic security workflows in production, sustaining availability, completion rate, latency and cost within defined service levels.
• Build and operate the observability layer with traces, structured logs and dashboards.
• Measure token consumption and cost per workflow using LangFuse and related tools.
• Detect failure modes and respond to restore service on Kubernetes/Docker deployments.
• Triage production issues to root cause and maintain runbooks for knowledge sharing.

Skills

Python proficiency

Cost and consumption tracking

Root cause analysis

SRE/production operations experience

Observability mindset

Tools

OpenTelemetry

Grafana

Datadog

Splunk

Kubernetes

Docker

PostgreSQL

Elasticsearch

LangFuse

Job description

Agent Reliability Engineer

The Security Resiliency Organization is rebuilding fragmented, manually-run security processes into consolidated, AI-native workflows that run in production at scale. As the most senior IC on the applied AI team, you will set the engineering standard the rest of the team builds against and own the hardest implementation problems.

Join us as an Agent Reliability Engineer on our Security Resiliency team in Bangalore to do the best work of your career and make a profound social impact.

What you’ll achieve

• You will keep agentic security workflows operating reliably in production, restoring service quickly when performance degrades.
• You will build the observability and performance-measurement layer that shows, in real time, how every workflow is running and what it costs.
• You will make monitoring a routine step for every new workflow rather than a bespoke effort each time.

You will:

• Operate agentic security workflows in production, sustaining availability, completion rate, latency and cost within defined service levels, using performance-measuring tools such as OpenTelemetry, Grafana, Datadog or Splunk.
• Build and operate the observability layer: execution traces across agent steps and tool calls, structured logging, health signals and dashboards, backed by PostgreSQL and Elasticsearch for metrics and log storage.
• Measure token consumption and cost per agent, workflow and run using tools such as LangFuse, reporting actual cost against expectation so overruns are visible early.
• Detect agent-specific failure modes – stalled or looping executions, tool call failures, credential and upstream API changes, silent quality degradation – and respond to restore service on Kubernetes/Docker-based deployments.
• Triage production issues to root cause, elevate defects to the building team with traces and reproduction detail, and maintain runbooks so operational knowledge does not depend on one person.

Take the first step towards your dream career

Every Dell Technologies team member brings something unique to the table. Here’s what we are looking for with this role:

Essential Requirements

• 5+ years in site reliability engineering, production operations or platform engineering, with direct responsibility for systems in production.
• Explicit, hands-on knowledge of performance-measuring and observability tools such as OpenTelemetry, Grafana, Datadog or Splunk – this is mandatory, not preferred, for this role.
• Python (or equivalent) proficiency to build monitoring tooling, automate operational tasks and read the code of the systems being operated.
• Working knowledge of Kubernetes and Docker for container operations, with infrastructure-as-code familiarity, plus PostgreSQL and Elasticsearch for metrics/log storage and querying.
• Cost and consumption tracking for metered services, including LLM token accounting, usage attribution and forecasting against quota or budget.

Desirable Requirements

• Direct experience with LLM tracing tools such as LangFuse for agent-run performance measurement, in addition to general observability platforms.
• Experience with LLM provider rate limits, quota management and multi-provider fallback, and runbook automation or failure-injection testing.

Who we are

We believe that each of us has the power to make an impact. That’s why we put our team members at the center of everything we do. If you’re looking for an opportunity to grow your career with some of the best minds and most advanced tech in the industry, we’re looking for you.

Dell Technologies is a unique family of businesses that helps individuals and organizations transform how they work, live and play. Join us to build a future that works for everyone because Progress Takes All of Us.

Dell Technologies is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. Read the full Equal Employment Opportunity Policy here.