Principal AI Architect for Scalable LLM Platforms

Insight Global; Town of Niskayuna (NY); On-site; USD 103,000 - 138,000; Full time

Job summary

Insight Global is seeking a Principal AI Software Engineer to drive the technical vision and architect enterprise AI applications across multiple initiatives. You will integrate LLMs, AI agents, NLP, CV, and predictive analytics to deliver secure, scalable AI solutions, mentoring teams and raising standards in Agile, testing, security, and documentation.

You will collaborate with product, data science, and engineering leaders, architect cloud-native platforms in AWS/Azure/GCP, and lead

Qualifications

• 10+ years of software engineering experience building scalable production applications.
• Strong backend experience with Python and cloud-native deployments in AWS, Azure or GCP.
• Experience designing scalable, highly available distributed systems.
• Experience building production RAG systems and vector databases.
• Proven track record delivering enterprise platforms with uptime and performance.
• Experience with AI observability, evaluation, and cost optimization.
• Experience with multi-agent systems and AI orchestration.
• Experience with containerized environments and Kubernetes/EKS.
• 13+ years of software development experience.
• Principal/Staff/Distinguished Engineer or Technical Architect background.

Responsibilities

• Define AI architecture and technical strategy across multiple products and initiatives.
• Design and build enterprise AI apps using LLMs, RAG, AI agents, and orchestration.
• Architect cloud-native AI platforms focusing on scalability, latency and cost.
• Lead architecture reviews and critical design tradeoffs across AI products.
• Design highly available distributed systems using Kubernetes and cloud services.
• Build and optimize end-to-end RAG pipelines and retrieval/response orchestration.
• Establish AI governance, monitoring, testing, and production support standards.
• Mentor engineers and influence technical direction across teams.
• Support post-deployment operations, incident readiness and performance optimization.

Skills

Leadership experience

Mentoring engineers

Architecture decisions

Production ownership

RAG architecture

Team collaboration

Education

Bachelor's degree in Computer Science/related

Tools

Python

Kubernetes

Cloud platforms (AWS/Azure/GCP)

AI platforms (Bedrock/OpenAI)

Vector databases (Pinecone/Weaviate/Milvus)

Containerization

Java/Go/Node.js

Job description

Insight Global is seeking a Principal AI Software Engineer to drive the technical vision and architect enterprise AI applications across multiple initiatives. You will integrate LLMs, AI agents, NLP, CV, and predictive analytics to deliver secure, scalable AI solutions, mentoring teams and raising standards in Agile, testing, security, and documentation.

You will collaborate with product, data science, and engineering leaders, architect cloud-native platforms in AWS/Azure/GCP, and lead