Principal Software Engineering - Reliability and FinOps Engineer

AT&T; Bothell (WA); On-site; USD 150,000 - 190,000; Full time

Job summary

AT&T Technology Services seeks a Principal System Engineer to lead reliability, FinOps, and automation initiatives across multicloud and hybrid environments. You will shape strategies, drive optimization, and deliver measurable savings while maintaining secure, scalable platforms.

You will mentor engineers, influence cross-functional roadmaps, and partner with internal teams to accelerate adoption of policy-driven, automated solutions in a fast-paced enterprise setting.

Qualifications

• 7+ years of progressive experience in systems engineering.
• Experience leading FinOps, cloud optimization, and automation initiatives.
• Strong skills in scripting and automation (Python, PowerShell, Bash).

Responsibilities

• Provide principal-level technical leadership across Azure, AWS, Kubernetes, and hybrid-cloud environments.
• Lead FinOps and infrastructure optimization activities including cost allocation, anomaly detection, and waste reduction.
• Develop policy-driven, closed-loop optimization workflows from identification to reporting.
• Apply automation-by-default using Python, PowerShell, Bash, REST APIs, and cloud-native services.
• Design generative AI and automation solutions for engineering operations and cost analysis.
• Establish governance, security, and change-management patterns with policy-as-code controls.
• Engineer secure identity models and RBAC across platforms.
• Integrate optimization with Terraform, GitHub, and CI/CD pipelines while preventing drift.
• Build executive dashboards communicating cloud spend, reliability, and savings.

Skills

Cloud optimization

FinOps

Python

PowerShell

Terraform

Kubernetes

Tools

Azure

AWS

GitHub

Job description

This is a full-time onsite role based in the location(s) posted and is performed from the designated work location five days per week. No relocation is offered.

AT&T will not hire any applicants for this position who require employer sponsorship now or in the future.

Join AT&T and reimagine the communications and technologies that connect the world.The Technology Services organization is responsible for advancing information technology performance and delivering solutions with a focus on maximizing ROI, increasing efficiency and enhancing the experience of end users. Guided by experienced leaders, Corporate Systems seamlessly integrate with advanced Technology and Operations to drive our enterprise forward. Our Systems Reliability and Software Delivery teams are unwavering in their commitment to excellence, ensuring every solution is robust and efficient. When you step into a career with AT&T, you won't just imagine the future, you'll create it.

About The Tea"m

AT&T Technology Services is modernizing the technology ecosystem through secure, scalable, cloud-enabled, and AI-powered platforms. Within Reliability & FinOps Engineering, our purpose is to elevate the customer experience by strengthening the reliability and resiliency of critical systems, maximizing the value of technology investments, and accelerating responsible automation.

Our organization brings together reliability engineering, FinOps and infrastructure optimization, AI and automation enablement, and customer-centric business intelligence. We build reusable engineering patterns, policy-driven controls, self-service capabilities, and intelligent optimization services that improve operational outcomes at enterprise scale.

What You'll Do

As a Principal System Engineer, you will serve as a senior technical leader across FinOps, cloud optimization, system reliability, and AI-driven automation. You will shape and execute enterprise strategies that improve the reliability of critical services, reduce cloud waste, strengthen financial accountability, and automate operational workflows across Azure, AWS, Kubernetes, and hybrid-cloud environments.

In this role, you will translate business priorities into durable technical solutions, influence cross-organizational roadmaps, and move optimization opportunities from insight through governed execution and measurable value realization.

Key responsibilities include:

• Provide principal-level technical leadership for resilient, secure, scalable, and cost-efficient platforms across Azure, AWS, Kubernetes, and hybrid-cloud environments.
• Lead FinOps and infrastructure optimization capabilities, including cost allocation, showback, anomaly detection, forecasting, rightsizing, workload scheduling, commitment optimization, waste reduction, and validated savings realization.
• Develop policy-driven, closed-loop optimization workflows that move recommendations through identification, application-owner engagement, approval, ServiceNow change control, execution, validation, evidence capture, and reporting.
• Apply an automation-by-default approach using Python, PowerShell, Bash, REST APIs, Azure Functions, Power Automate, and event-driven services to reduce manual handoffs and operational toil.
• Design practical generative AI, AI-fueled coding, and agentic automation solutions for engineering operations, cost analysis, anomaly investigation, remediation guidance, executive reporting, knowledge retrieval, and decision support.
• Establish cloud-governance guardrails, policy-as-code controls, exception processes, auditability, and risk-based automation patterns aligned with security, architecture, compliance, and change-management requirements.
• Engineer secure identity and authorization models using role-based access control, managed identities, service principals, privileged identity management, just-in-time access, least privilege, and time-bound permissions.
• Integrate optimization and automation capabilities with Terraform, GitHub, Azure DevOps, CI/CD pipelines, ServiceNow, monitoring platforms, cost-management tools, and enterprise data sources while preventing configuration drift.
• Advance reliability engineering practices, including observability, service-level objectives, capacity management, performance analysis, incident prevention, operational readiness, post-change validation, rollback planning, and continuous improvement.
• Build executive dashboards and operational scorecards that communicate cloud spend, reliability, automation coverage, optimization pipeline health, risk, realized savings, and customer or business outcomes.
• Lead proofs of concept, production readiness, phased adoption, and complex technical issue resolution in partnership with internal teams and strategic vendors.
• Mentor engineers, create reusable reference architectures and standards, and influence priorities across Reliability, FinOps, Security, Architecture, Finance, Operations, and application teams.
• Demonstrate AT&T leadership behaviors by serving customers first, moving faster, acting boldly, winning as one, and developing talent.

What You'll Need

• 7+ years of progressive experience in systems engineering