Senior Software Engineer - AI Platform Engineering

Aimlroles; City of Rochester (NY); Hybrid; USD 150,000 - 230,000; Full time

Job summary

Aimlroles is seeking a senior platform engineer to architect high-performance hosting environments for AI models and autonomous agents, emphasizing scalable back-end systems and container orchestration. The role is hybrid, located within 100 miles of Mayo Clinic for occasional on-site expectations in Rochester, NY.

You will drive real-time monitoring, distributed tracing, and production evaluation of agents and models, and join a rotating platform operations team to ensure reliability and

Qualifications

• Bachelor's degree in Computer Science/Engineering or related field with 5 years of experience; OR an Associate’s degree with 7 years of experience.
• Deep knowledge of software engineering practices including SDLC, code reviews, version control, build, test, and operations.
• Experience with containerized environments and orchestration, preferably Kubernetes.
• Strong understanding of LLM lifecycles, inference optimization, and RAG architectures.

Responsibilities

• Architect the future of runtimes: design and implement high-performance hosting environments for models and autonomous agents.
• Scale hybrid-cloud infrastructure: build multi-tenant platform capabilities for cross-cloud and on-prem environments.
• Drive operational excellence: implement real-time monitoring, distributed tracing, and automated evaluation of agents & models in production.
• Operational stewardship: be part of a rotating platform operations team ensuring reliability and uptime of AI services.
• Collaborate & advocate: partner with cross-functional AI teams to reduce friction and accelerate AI-driven value delivery.

Skills

Deep systems expertise

AI/ML fluency

Operational rigor

Problem-solving mindset

Education

Bachelor's degree in CS/Engineering or related field

Associate’s degree in CS/Engineering or related field

Tools

Kubernetes

Python

GCP

AWS

Azure

Job description

This is a hybrid position and must be located within 100 miles of a Mayo Clinic campus for occasional on-site expectations based on business needs.

What You’ll Do

• Architect the Future of Runtimes: Design and implement high-performance hosting environments for both specialized models and autonomous agents.
• Scale Hybrid-Cloud Infrastructure: Build multi-tenant platform capabilities that support seamless agent and model execution across diverse cloud and on-prem environments.
• Drive Operational Excellence: Implement robust solutions for real-time monitoring, distributed tracing, and automated evaluation of agents & models in production.
• Operational Stewardship: Act as a member of our rotational platform operational team, ensuring the reliability and uptime of critical AI services.
• Collaborate & Advocate: Partner with cross-functional AI teams to identify system friction, streamlining their workflows and accelerating the delivery of AI-driven value.

What You’ll Bring

• Deep Systems Expertise: Proven experience building scalable back-end systems, ideally with a focus on containerization and orchestration (Kubernetes).
• AI/ML Fluency: Strong understanding of the lifecycle of LLMs and autonomous agents, including inference optimization and RAG architectures.
• Operational Rigor: A track record of implementing observability (logging, metrics, tracing) in complex, distributed environments.
• Problem-Solving Mindset: The ability to turn abstract AI research into production-grade, hardened platform features.

Why this team?

We are moving beyond simple model serving. We are building the foundational "nervous system" for autonomous agents in a high-stakes environment where performance and governance are non-negotiable.

Required qualifications for this position include: Bachelor's Degree in Computer Science/Engineering or related field with 5 years of experience as noted below; OR an Associate’s degree in Computer/Science/Engineering or related field with 7 years of experience.

Have in-depth knowledge of software engineering with experience coding applications or services in a high-level programming language, preferably Python, and a basic knowledge of related fields. Demonstrated problem solving and time management skills. Possesses strong technical aptitude for designing and implementing software solutions. Experience with modern application development frameworks. Knowledge of professional software engineering practices & best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations. Deep hands-on technical expertise, excellent verbal and written communication skills. Experience with Agile software development techniques.

Preferred qualifications:

• Master's degree in Computer Science/Engineering or related field.
• Ability to use a wide variety of open-source technologies and cloud-based services.
• Experience writing software for the cloud (GCP, AWS, Azure).
• Experience in databases, analytics, big data systems or business intelligence products.
• Experience building high-performance, highly available and scalable distributed systems.
• Experience developing software for healthcare related industries.

Authorization to work and remain in the United States, without necessity for Mayo Clinic sponsorships now, or in the future (for example, be a U.S. Citizen, national, or permanent resident, refugee, or asylee). Mayo Clinic does not participate in the F-1 STEM OPT extension program.