Lead Site Reliability Engineer

Luxoft; Buffalo (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Luxoft in Buffalo, NY, seeks a senior Site Reliability Engineer to ensure reliability, scalability, performance, and operational excellence of critical banking platforms. You will design, implement, and improve SRE practices across the software development lifecycle, partnering with development, infrastructure, and security teams to boost resiliency through automation and observability.

The role emphasizes coaching, incident response leadership, RCA, and capacity planning while driving IaC using

Qualifications

• Must have proven expertise in observability, monitoring, and incident response.
• Experience with cloud-native architectures on Azure and Terraform IaC.
• Strong background in SRE practices, SLOs/SLIs, and RCA.
• Hands-on skills with Dynatrace, OTel, metrics, logs, and dashboards.

Responsibilities

• Design and support highly available, scalable cloud applications and infrastructure.
• Define and monitor SLOs/SLIs and error budgets for critical services.
• Lead incident response and post-incident RCA to prevent recurrence.
• Develop automated regression tests and improve reliability through automation.
• Build and maintain IaC with Terraform for cloud provisioning and standardization.
• Collaborate with engineering, security, and platform teams to improve resiliency.

Skills

Observability

Dynatrace

OpenTelemetry

Distributed tracing

Terraform

Azure

Incident management

Automation

Tools

Terraform

Azure Monitor

OpenTelemetry

Dynatrace

Log Analytics

Application Insights

Job description

Project description

Responsible at the expert level for ensuring the reliability, scalability, performance, and operational excellence of critical banking platforms and applications. Serves as a senior individual contributor responsible for designing, implementing, and improving Site Reliability Engineering (SRE) practices across the software development lifecycle. Works closely with application development, infrastructure, platform engineering, and business teams to enhance system resiliency through automation, observability, testing, and proactive operational management while coaching and influencing others.

Responsibilities

• Design, implement, and support highly available, scalable, and resilient applications and cloud infrastructure following enterprise technology standards and SRE best practices.Lead initiatives to improve system reliability, availability, performance, and operational maturity through automation and engineering excellence.Define, implement, and monitor Service Level Objectives (SLOs), Service Level Indicators (SLIs), and error budgets for critical business services.Develop comprehensive observability strategies leveraging Dynatrace, OpenTelemetry (OTel), distributed tracing, metrics, logging, dashboards, and alerting solutions.Design and maintain end-to-end monitoring solutions that provide actionable insights into application, infrastructure, and customer experience health.Analyze production telemetry to proactively identify performance bottlenecks, reliability risks, and capacity constraints.Lead incident response activities for high-severity production events, coordinating cross-functional teams to restore services and minimize customer impact.Perform and facilitate Root Cause Analysis (RCA) activities, ensuring corrective and preventive actions are identified, prioritized, and implemented.Drive operational excellence through automation of repetitive tasks, operational workflows, deployments, recovery procedures, and reliability controls.Partner with development teams to build reliable and observable services throughout the Software Development Lifecycle (SDLC).Design, develop, and execute automated regression testing strategies to validate application stability, reliability, and performance following deployments and infrastructure changes.Review test coverage and reliability validation approaches to ensure comprehensive testing and risk mitigation.Create, maintain, and improve Infrastructure as Code (IaC) solutions using Terraform for cloud infrastructure provisioning, configuration management, and environment standardization.Support and optimize Microsoft Azure environments, including Azure App Services, resource management, scaling strategies, deployment automation, and application lifecycle management.Utilize Azure-native tools such as Azure Monitor, Application Insights, Log Analytics, and related services to improve platform visibility and reliability.Drive implementation of performance testing, resiliency testing, fault tolerance validation, and disaster recovery preparedness within assigned domains.Establish operational readiness standards and ensure applications meet reliability, scalability, observability, and supportability requirements before production deployment.Review architectural designs and provide recommendations to improve platform resiliency, operational efficiency, and cloud optimization.Lead capacity planning, performance tuning, and workload optimization efforts across production environments.Develop and maintain operational runbooks, incident playbooks, knowledge articles, and standard operating procedures.Serve as a key partner with engineering, infrastructure, cybersecurity, architecture, and support teams to identify and implement continuous process improvements spanning organizational boundaries.Communicate system health, reliability trends, operational risks, and remediation strategies to technical and business stakeholders.Present reliability initiatives, operational metrics, and engineering recommendations at architecture reviews, technical forums, and leadership meetings.Mentor engineers on observability, cloud engineering, automation, SRE principles, and operational best practices.Understand and adhere to the Company's risk and regulatory standards, policies, and controls in accordance with the Company's Risk Appetite.Identify reliability, operational, and technology risks requiring escalation to management.Promote an environment that supports a culture of belonging and reflects the Client brand.Maintain Client internal control standards, including timely implementation of internal and external audit findings and regulatory requirements as applicable.Complete other related duties as assigned.

SKILLS

Must have

• Strong experience in observability and monitoring, including hands-on expertise with:DynatraceOpenTelemetry (OTel)Distributed tracingMetrics collection and analysisCentralized logging and log aggregationAlerting and dashboard developmentProven experience designing and executing automated regression testing frameworks and test suites to ensure application and platform stability following deployments.Strong proficiency in Infrastructure as Code (IaC) using Terraform.Experience with CI/CD pipelines, deployment automation, and operational tooling.Expert knowledge of production systems monitoring, incident management, and operational troubleshooting.Strong understanding of application performance management, distributed systems, and modern cloud-native architectures.Cloud & Platform ExpertiseStrong experience with Microsoft Azure, including:Azure App ServicesResource GroupsAzure networking conceptsScaling and performance optimizationDeployment and release managementApplication lifecycle managementExperience leveraging Azure-native operational tooling such as:Azure MonitorApplication InsightsLog AnalyticsAzure dashboards and alertingExperience supporting cloud-native and hybrid infrastructure environments.Reliability & Engineering PracticesDemonstrated experience implementing and operating SRE practices, including:Service Level Objectives (SLOs)Service Level Indicators (SLIs)Error budgetsIncident managementProblem managementRoot Cause Analysis (RCA)Reliability automationAbility to improve system reliability through:Performance tuningCapacity planningObservability-driven insightsProactive issue detectionReliability engineering initiativesExperience developing automated recovery mechanisms and self-healing solutions.Knowledge of resiliency engineering patterns, disaster recovery planning, and high-availability architectures.

Nice to have

Experience supporting large-scale enterprise applications in regulated environments.Strong analytical and troubleshooting skills related to production systems and distributed architectures.Experience working in Agile and DevOps operating models.Ability to work autonomously and lead complex reliability initiatives.Strong organizational and time management skills.Advanced verbal and written communication skills.Experience driving project milestones and delivery commitments.Proven experience leading major incident response and post-incident improvement efforts.Experience partnering with architecture, infrastructure, cybersecurity, and application development teams.Experience with scripting and automation using PowerShell, Python, Bash, or similar technologies.Industry certifications in Azure, Terraform, Cloud Engineering, or Site Reliability Engineering preferred.