AI Gateway Engineer

Jobtailor

Town of Florida (NY)

On-site

USD 180,000 - 240,000

Full time

14 days+

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Jobtailor in New York seeks a senior Enterprise AI Gateway Architect to lead the architecture, engineering, security, and operations of the enterprise AI Gateway platform, primarily using Kong AI Gateway. You will enable secure, scalable consumption of LLMs and AI services across multi-cloud environments.

The role requires 8–12 years of experience in enterprise API platforms and cloud security, with hands-on Kong expertise and preferred certifications.

Qualifications

8 to 12 years of experience
Bachelor's degree in Computer Science, Cyber Security, Information Technology, Engineering, or related discipline
8+ years of experience designing and operating enterprise API, application, or cloud platforms
5+ years of experience in cloud architecture and security
3+ years working with AI/ML platform technologies or enterprise AI deployments
Hands-on experience implementing and managing Kong Gateway and Kong Enterprise solutions
Experience within regulated industries such as financial services, banking, insurance, or capital markets strongly preferred
Kong Certified Professional certification preferred
AWS Solutions Architect certification preferred
Microsoft Azure Solutions Architect certification preferred
CISSP, CCSP, or equivalent security certification preferred
Experience implementing AI security controls and governance frameworks preferred
Familiarity with NIST AI RMF, NIST 800-53, NYDFS 500, PCI DSS, SOC 2, and other financial services regulatory requirements preferred
Experience with DSPM, DLP, and enterprise data protection controls preferred

Responsibilities

Lead architecture and operations of the enterprise AI Gateway platform using Kong
Enable secure, scalable consumption of LLMs, AI Agents, and related services
Design and evolve gateway solutions with Kong Enterprise
Develop onboarding patterns for apps, AI agents, and business services
Engineer integrations for OpenAI, Azure OpenAI, AWS Bedrock, Anthropic, and more
Implement governance, policy enforcement, and AI security measures
Define scalable control and data plane architectures across multi-cloud environments
Architect identity controls and IAM integrations
Implement OAuth 2.0, OIDC, JWT, mTLS, RBAC, ABAC and workload identities
Establish policy-as-code, logging, monitoring, and audit controls
Collaborate with Risk, Compliance, Legal, and AI Governance teams
Design highly available, multi-region deployments with DR/BC controls
Implement rate limiting, load balancing, and traffic shaping
Define and manage SLOs, SLIs, and error budgets
Conduct architecture reviews, resilience testing, and failure-scenario exercises
Design AI access patterns across Azure, AWS, and Snowflake
Integrate gateway with cloud AI services and observability tooling
Build observability for AI workloads: metrics, tracing, audit logs
Analyze token consumption, latency, cost, and security events
Create dashboards for engineering, security, risk, and executives
Support incident response and platform troubleshooting

Skills

Kong AI Gateway Implementation

Cloud Architecture and Security

AI Security Controls and Governance

Enterprise API Management

Regulatory Compliance Experience

Education

Bachelor's degree in Computer Science or related discipline

Tools

Kong Enterprise Solutions

OpenAI

Azure OpenAI

AWS Bedrock

Snowflake Cortex

Observability Tooling

Identity & Access Management Platforms

Job description

Lead the architecture, engineering, security, and operational management of the enterprise AI Gateway platform, primarily using Kong AI Gateway
Enable secure, resilient, compliant, and scalable consumption of LLMs, AI Agents, RAG services, MCP services, and A2A communications
Design, implement, and evolve enterprise AI Gateway solutions using Kong AI Gateway and Kong Enterprise
Develop standardized onboarding patterns for applications, AI agents, and business services consuming AI
Engineer reusable integrations for OpenAI, Azure OpenAI, AWS Bedrock, Anthropic, Snowflake Cortex, and internal/external AI services
Implement intelligent model routing, failover, traffic shaping, provider abstraction, custom Kong plugins, and AI governance integrations
Define scalable control plane and data plane architectures across hybrid and multi-cloud environments
Architect identity controls and integrate the gateway with enterprise identity providers and IAM platforms
Implement OAuth 2.0, OIDC, JWT, mTLS, RBAC, ABAC, non-human identities, workload identities, and agent identities
Establish fine-grained authorization at model, agent, tool, prompt, and data-source levels
Partner with security and compliance teams on AI governance and Zero Trust controls
Implement AI security guardrails, policy enforcement, prompt-injection protection, DLP, PII detection/redaction, content safety, filtering, and model access governance
Establish policy-as-code, logging, monitoring, audit controls, and regulatory compliance support
Collaborate with Risk, Compliance, Legal, Data Protection, and AI Governance teams
Design highly available architectures, multi-region deployments, provider failover, cross-cloud recovery, active-active architectures, disaster recovery, and business continuity controls
Implement rate limiting, circuit breakers, load balancing, traffic throttling, semantic caching, and capacity management
Define and manage SLOs, SLIs, error budgets, RTOs, and RPOs
Conduct architecture reviews, resilience testing, and failure-scenario exercises
Design AI access patterns across Azure, AWS, and Snowflake
Integrate gateway services with Azure OpenAI, AWS Bedrock, Snowflake Cortex, vector databases, data protection platforms, and observability tooling
Build observability for AI workloads, including monitoring, metrics, tracing, and audit logging
Analyze token consumption, latency, model utilization, cost optimization, and security events
Create operational dashboards for engineering, security, risk, and executive stakeholders
Support incident response and platform troubleshooting

Requirements

8 to 12 years of experience
Bachelor's degree in Computer Science, Cyber Security, Information Technology, Engineering, or related discipline
8+ years of experience designing and operating enterprise API, application, or cloud platforms
5+ years of experience in cloud architecture and security
3+ years working with AI/ML platform technologies or enterprise AI deployments
Hands-on experience implementing and managing Kong Gateway and Kong Enterprise solutions
Experience within regulated industries such as financial services, banking, insurance, or capital markets strongly preferred
Kong Certified Professional certification preferred
AWS Solutions Architect certification preferred
Microsoft Azure Solutions Architect certification preferred
CISSP, CCSP, or equivalent security certification preferred
Experience implementing AI security controls and governance frameworks preferred
Familiarity with NIST AI RMF, NIST 800-53, NYDFS 500, PCI DSS, SOC 2, and other financial services regulatory requirements preferred
Experience with DSPM, DLP, and enterprise data protection controls preferred

Core Competencies

Demonstrates expertise in designing and implementing enterprise AI Gateway solutions, with a strong focus on security, compliance, and scalable architectures. Proficient in integrating AI services and managing cloud environments while ensuring adherence to regulatory standards.

Highest-signal resume keywords