Sr. AI Platform Engineer

TechWize; New York, Northern (NY, KY); Hybrid; USD 170,000 - 260,000; Full time

Job summary

TechWize in New York, NY is seeking a Sr. AI Platform Engineer to design and run a scalable AI platform supporting LLM orchestration, vector search, and multi-agent workloads.

You will build APIs and infrastructure for conversational apps, implement inference pipelines on AWS with Kubernetes, and mentor engineers while upholding security and governance standards.

This role offers a fast-paced environment with cutting-edge technologies and opportunities for professional growth.

Qualifications

• 8+ years as Platform Engineer with 3+ years in AI/ML platform development (MLOps).
• Deep expertise in Python with strong design and debugging skills.
• Ability to lead complex projects and communicate effectively.
• Proficiency with AWS, GCP, or Azure and MLOps tools (MLflow, Kubeflow).
• Experience with CI/CD, and infrastructure as code (Terraform/CloudFormation).
• Hands-on with CI/CD pipelines, model observability, and AI service incident response.

Responsibilities

• Platform Design and Architecture: build and operate a scalable AI platform for LLM orchestration, vector search, and multi-agent frameworks.
• Core Infrastructure Development: create APIs and infrastructure for conversational apps, AI agents, and analytics tools.
• LLM Operational Solutions: implement workflows for inference pipelines, fine-tuning, caching, and evaluation.
• Deployment & Performance Optimization: deploy AI services on AWS with Kubernetes (EKS), Lambda, and ECS; optimize vector DBs and runtimes.
• Collaboration, Governance, & Mentorship: partner with teams to deliver production-grade services and mentorship.

Skills

Python

Platform Engineering

AI/ML Platform Dev

Cloud Platforms

CI/CD

Infrastructure as Code

Model Observability

Incident Response

Kubernetes

Tools

Docker

Kubernetes

Helm

Terraform

MLflow

Kubeflow

Ray.io

Job description

• 501, Fifth Avenue, Suite 805 New York, NY 10017

Job Title : Sr. AI Platform Engineer

What We’re Looking for (Minimum Qualifications):

• 8+ years of experience as Platform Engineer ( Site Reliability / DevOps Engineer) , with at least 3+ years in AI/ML platform development ( MLOps ).
• Deep expertise in Python, with strong design and debugging skills.
• Ability to work independently and lead complex projects with Excellent problem-solving, analytical, and communication skills.
• Proficiency working with cloud platforms such as AWS, GCP, or Azure and familiarity with MLOps/AI DevOps tools like MLflow or Kubeflow, proficient in CI/CD , infrastructure as code (Terraform / CloudFormation).
• Hands-on expertise with CI/CD pipelines, model observability, and incident response for AI/ML services.

Preferred Qualification:

• Experience implementing and optimizing Platforms supporting large language model (LLM) pipelines with frameworks such as LangChain, LlamaIndex, Hugging Face Transformers, or similar.
• Hands-on knowledge of Scaling & Setting up Vector DB platforms such as Qdrant (or other vector DBs like Pinecone, Weaviate) for semantic search and embeddings management.
• Exposure to MLOps tools, Ray.io , Anyscale or other distributed orchestration & inference frameworks.
• Experience with developing and deploying containerized applications using Docker and Kubernetes, including Helm charts and automated scaling.
• Understanding of LLMOps patterns — model registry, prompt versioning, and feedback loops.

Responsibilities

Responsibilities/What You’ll Do

• Platform Design and Architecture: building and operating a highly available, scalable, modular AI platform using technologies such as Qdrant, Anyscale, and Ray to support LLM orchestration, vector search, and multi-agent frameworks.
• Core Infrastructure Development: Build essential APIs and infrastructure to power conversational applications, AI agents, and analytics tools.
• LLM Operational Solutions: Implement workflows for Large Language Models, including inference pipelines, fine-tuning, caching, and evaluation for open-weight and hosted models.
• Deployment & Performance Optimization: Deploy AI services on AWS with Kubernetes (EKS), Lambda, and ECS, ensuring scalability and resilience while optimizing vector databases and model runtimes for cost and performance.
• Collaboration, Governance, & Mentorship: Partner with engineering teams, research teams to deliver production-grade, self-healing, and performance-optimized services for AI/RAG pipelines , establish governance/security standards, and mentoring junior engineers in AI infrastructure best practices & reviews.

Join the TechWize Team

Shape the future of IT

Are you a passionate techwize looking to push the boundaries of IT?

Do you thrive in a collaborative environment, solving complex problems with cutting-edge solutions?

If you answered yes, then TechWize is the place for you! We're seeking talented individuals to join our dynamic team and shape the future of IT.

At TechWize, you'll have the opportunity to work alongside industry experts on challenging projects, using the latest technologies to deliver innovative solutions for our clients. We offer a fast-paced environment that fosters continuous learning and professional growth.