Technical Engineer (Production Liability Engineer)

M&T Bank; South Dakota; On-site; USD 97,000 - 162,000; Full time

Job summary

M&T Bank is seeking a Technical Engineer to ensure availability, stability, and performance of critical applications in Buffalo, NY. This senior role emphasizes SRE practices, incident management, and automation to drive reliability and operational excellence.

You will collaborate with Engineering, Infrastructure, Security, and Product teams, applying modern observability, cloud operations, and AI-assisted tooling to reduce toil and improve incident resolution and customer outcomes.

Qualifications

• Bachelor’s degree or equivalent experience in a technical field.
• 5+ years supporting enterprise applications, platforms, or infrastructure in production.
• Strong incident management and operational best practices.
• Experience with monitoring, observability, logging, and alerting.
• Automation and scripting proficiency across multiple platforms.

Responsibilities

• Serve as escalation point for critical incidents and outages.
• Lead troubleshooting and RCA across apps, infra, cloud, and APIs.
• Coordinate incident response with cross-functional teams and vendors.
• Drive automation to reduce toil and improve reliability.
• Design and optimize monitoring, dashboards, and SLOs/SRIs.
• Mentor junior engineers and contribute to playbooks and runbooks.
• Participate in disaster recovery, failover, and business continuity testing.

Skills

Incident management

SRE

Observability

Automation

Scripting

PowerShell

Python

Bash/Shell

SQL

REST APIs

CI/CD

AI-assisted operations

Education

Bachelor’s degree in Computer Science, Information Technology, Engineering, or equivalent experience

Tools

Dynatrace

Splunk

Datadog

Azure Monitor

Grafana

Prometheus

OpenTelemetry

AppDynamics

Job description

Overview

The Technical Engineer serves as a senior production support and reliability engineering professional responsible for ensuring the availability, stability, performance, and operational excellence of critical business applications and platforms.

This role combines strong troubleshooting expertise with modern Site Reliability Engineering (SRE), observability, automation, cloud operations, and incident management practices. The Technical Engineer partners with Engineering, Architecture, Infrastructure, Security, Product, and Vendor teams to proactively identify operational risks, improve system resilience, accelerate incident resolution, and continuously enhance customer and employee experiences.

The ideal candidate possesses deep technical knowledge of application support, distributed systems, cloud technologies, monitoring platforms, automation tools, and modern operational practices. They are passionate about eliminating repetitive work through automation and leveraging AI-powered tools to improve operational efficiency and support outcomes.

Primary Responsibilities

Production Support & Incident Management

• Serve as a technical escalation point for critical production incidents, outages, and service degradation events.
• Lead troubleshooting and root cause analysis efforts across applications, integrations, infrastructure, cloud services, APIs, and supporting technologies.
• Coordinate incident response activities involving application teams, infrastructure teams, vendors, and business stakeholders.
• Restore service quickly while ensuring long-term corrective actions are identified and implemented.
• Participate in major incident management processes and post-incident reviews.

Site Reliability Engineering (SRE)

• Apply Site Reliability Engineering principles to improve platform reliability, scalability, resilience, and operational efficiency.
• Define and support Service Level Indicators (SLIs), Service Level Objectives (SLOs), and operational performance metrics.
• Drive reduction of operational toil through automation and process improvement.
• Support production readiness reviews and operational acceptance processes.
• Participate in disaster recovery, resiliency, failover, and business continuity testing.

Troubleshooting & Problem Management

• Analyze complex system behavior using logs, metrics, traces, performance data, and monitoring tools.
• Perform deep technical investigations across application, infrastructure, data, network, and cloud environments.
• Identify recurring issues, trends, and systemic problems to reduce future incidents.
• Lead root cause analysis (RCA) activities and implement preventive solutions.
• Develop technical recommendations that improve system stability, performance, and reliability.

Observability & Monitoring

• Design, implement, and optimize monitoring, alerting, logging, and observability solutions.
• Develop dashboards and health indicators providing visibility into application and platform performance.
• Partner with engineering teams to improve observability through instrumentation, distributed tracing, synthetic monitoring, and telemetry collection.
• Continuously refine alerting strategies to reduce false positives and alert fatigue.
• Establish operational health metrics and reliability reporting.

Automation & Scripting

• Develop and maintain automation solutions that improve operational efficiency and service reliability.
• Create scripts, tools, and workflows to automate diagnostics, health checks, remediation activities, and routine support tasks.
• Leverage Infrastructure as Code (IaC) and automation frameworks where appropriate.
• Drive continuous improvement through operational automation and self-healing capabilities.
• Partner with engineering teams to integrate automation into deployment and operational workflows.

Desired Scripting Technologies:

• PowerShell
• Python
• Bash/Shell
• SQL
• REST APIs
• Workflow automation platforms

AI-Assisted Operations & Innovation

• Leverage AI and Generative AI tools to improve incident analysis, troubleshooting, knowledge management, and operational efficiency.
• Utilize AI-powered operational insights to identify patterns, anomalies, and emerging risks.
• Contribute to development of intelligent support capabilities including chatbots, operational copilots, automated RCA generation, and knowledge recommendations.
• Evaluate opportunities to improve production support through AI-enabled automation and predictive analytics.
• Promote responsible AI practices aligned with enterprise governance and security requirements.

Support Playbooks & Knowledge Management

• Develop, maintain, and continuously improve support runbooks, operational procedures, troubleshooting guides, and recovery playbooks.
• Ensure support documentation remains accurate, actionable, and aligned with production environments.
• Establish standardized operational processes supporting incident response and service recovery.
• Capture lessons learned from incidents and incorporate improvements into support practices.
• Build and maintain operational knowledge repositories to improve support consistency and reduce resolution times.

Cloud Operations

• Support cloud-hosted and hybrid application environments, including Azure-based platforms and services.
• Assist engineering teams in implementing resilient and observable cloud architectures.
• Monitor cloud resource health, performance, utilization, and operational readiness.
• Support cloud deployments, platform upgrades, and operational change activities.
• Partner with cloud engineering teams on modernization and reliability initiatives.

Collaboration & Technical Leadership

• Partner closely with Engineering, Product, Architecture, Infrastructure, Security, QA, and Vendor teams.
• Review operational readiness of new systems and platform enhancements.
• Mentor junior support engineers and provide technical guidance during incident response activities.
• Promote operational excellence, reliability engineering principles, and continuous improvement practices.
• Serve as a subject matter expert within assigned technology domains.

Governance, Risk & Compliance

• Ensure production support activities comply with enterprise risk, security, regulatory, and audit requirements.
• Identify operational risks and escalation issues appropriately.
• Support implementation of internal controls and operational governance standards.
• Participate in audit, compliance, and regulatory review activities as required.

Education & Experience Required

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or equivalent experience.
• Minimum 5+ years of experience supporting enterprise applications, platforms, or infrastructure in production environments.
• Experience troubleshooting complex application, infrastructure, integration, or cloud-related issues.
• Strong knowledge of incident management, production support processes, and operational best practices.
• Experience with monitoring, observability, logging, and alerting platforms.
• Experience with scripting and automation technologies.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

Preferred Qualifications

Reliability Engineering & Operations

• Experience with Site Reliability Engineering (SRE) practices.
• Experience defining and managing SLIs, SLOs, and operational metrics.
• Experience supporting distributed systems, APIs, microservices, and cloud-native applications.
• Experience performing production readiness reviews and operational assessments.

Observability & Monitoring

Experience with platforms such as:

• Dynatrace
• Splunk
• Datadog
• Azure Monitor
• Grafana
• Prometheus
• OpenTelemetry
• AppDynamics

Cloud Technologies

• Experience supporting Azure cloud environments.
• Familiarity with Azure App Services, AKS, Functions, Storage, Event Hub, Service Bus, and monitoring services.
• Understanding of cloud security and operational best practices.

Automation & Scripting

• Advanced PowerShell scripting.
• Python development and automation.
• REST API integration.
• Infrastructure as Code concepts.
• CI/CD tools and deployment pipelines.

AI & Modern Operations

• Experience using Microsoft Copilot, Azure AI services, Microsoft Foundry, or similar AI-enabled platforms.
• Familiarity with AI-assisted troubleshooting and operational analytics.
• Experience implementing AI-enabled support workflows or knowledge management solutions.
• Understanding of intelligent automation and operational copilots.

What Great Looks Like

A successful Production Reliability Engineer:

• Resolves complex incidents quickly and effectively.
• Proactively identifies and eliminates recurring production issues.
• Uses automation and scripting to eliminate manual support work.
• Builds comprehensive support playbooks and operational runbooks.
• Leverages AI to accelerate troubleshooting and operational insights.
• Establishes strong observability and monitoring capabilities.
• Partners effectively with engineering teams to improve reliability.
• Drives a culture of operational excellence through SRE principles.
• Continuously improves customer experience through system stability, resilience, and performance.

M&T Bank is committed to fair, competitive, and market-informed pay for our employees. The pay range for this position is $97,100.00 - $161,800.00 Annual (USD). The successful candidate’s particular combination of knowledge, skills, and experience will inform their specific compensation.

Location

Buffalo, New York, United States of America