VP, Enterprise AI Safety Platforms & Governance

Socket.dev; New York (NY); On-site; USD 130,000 - 250,000; Full time

Job summary

Socket.dev is seeking a Vice President to lead AI safety platforms in New York. You will design a company-wide agentic evaluation framework and guardrail infrastructure, aligning regulatory and risk requirements with scalable enterprise solutions.

The role requires deep expertise in LLMs, tool integration, and platform engineering, with a track record of building high-performance microservices and leading ML teams. Strong collaboration with risk committees and executives is essential.

Qualifications

• VP-level experience in financial services or large-scale enterprise software environments.
• Bachelor’s or Master’s degree in CS/AI/Systems Engineering or related field.
• 8+ years of hands-on software development experience (Python, Go, Java, or C++) building microservices or enterprise platforms.
• 4+ years leading applied ML or software engineering teams for platform infra/microservices.
• Strong fluency with LLMs, RAG systems, function calling / tool integration, and agentic execution paradigms.

Responsibilities

• Design, build, and deploy a company-wide agentic evaluation framework to benchmark AI agent performance before production deployment.
• Implement evaluation methods for autonomous planning, tool-calling precision, multi-turn state retention, and error recovery.
• Integrate automated evaluation pipelines into runtime environments to detect drifting reasoning and tool failures in production.
• Lead the development of low-latency guardrail engines and real-time control mechanisms for safe agent operation.

Skills

LLMs & agentics

Platform leadership

Software development

Microservices

Tool integration

Governance

Agentic evaluation

Guardrail concepts

Education

Bachelor's or Master's in CS/AI/Systems Engineering

Tools

LangSmith

OpenTelemetry

Phoenix

MLflow

NeMoGuardrails

Guardrails AI

Job description

Socket.dev is seeking a Vice President to lead AI safety platforms in New York. You will design a company-wide agentic evaluation framework and guardrail infrastructure, aligning regulatory and risk requirements with scalable enterprise solutions.

The role requires deep expertise in LLMs, tool integration, and platform engineering, with a track record of building high-performance microservices and leading ML teams. Strong collaboration with risk committees and executives is essential.