Senior AI Engineer - Generative AI for Legal Tech (Azure)

Onward Legal Innovation; New York (NY); Hybrid; USD 176,000 - 264,000; Full time

Benefits offered by this job

Comprehensive benefits

Paid time off

Retirement plan

Learning budget

Job summary

Onward Legal Innovation seeks a Senior AI Software Engineer to design and deploy production-grade AI for a top US law firm. You will build secure AI products, leverage LLMs, RAG, and AI agents within Azure, delivering scalable solutions for document analysis, diligence, and legal research.

You will work in a hybrid setup across NY/Chicago, collaborating with attorneys, data scientists, and engineers to advance AI capabilities across the firm with strong governance and security focus.

Qualifications

• 4+ years of relevant experience in tech, professional services, or legal environments, ideal in a law firm or complex org.
• Strong Python development with SQL, APIs, databases, and data handling (structured/unstructured).
• Hands-on experience building LLM/Generative AI apps with OpenAI/Azure OpenAI, Hugging Face, LangChain or similar.
• Designing and implementing RAG pipelines and AI agents, using vector stores and knowledge graphs.
• Experience securing scalable AI solutions within Microsoft Azure.
• Familiarity with Docker, Kubernetes, CI/CD; Azure AI Studio and related Azure services advantageous.
• Experience building LLM evaluation frameworks using Promptfoo, OpenAI Evals, LangSmith.
• Experience creating internal AI apps with Retool/Streamlit/Gradio.

Responsibilities

• Design and build secure internal AI products and workflows using LLMs, vectors, orchestration, and modern UI tech.
• Develop RAG pipelines, AI agents, and applications for document analysis and legal research.
• Architect resilient AI solutions on Microsoft Azure with cross-team collaboration.
• Move AI products from prototyping to production deployment and monitoring.
• Build evaluation frameworks for accuracy, robustness, safety, and ROI.
• Collaborate with attorneys and stakeholders to identify high-value AI opportunities.
• Advise on prompt engineering, model choice, architecture, and evaluation practices.
• Create reusable libraries, docs, and patterns for firm-wide AI development.
• Evaluate new AI technologies, open-source frameworks, and vendor platforms.
• Support pilots, demonstrations, user training, and knowledge sharing.

Skills

Python

SQL

APIs

Databases

LLMs

Generative AI

RAG pipelines

AI agents

Vector databases

Pinecone

Knowledge graphs

Azure

Docker

Kubernetes

CI/CD

Azure AI Studio

Azure Cognitive Search

Azure Cosmos DB

Prompt engineering

LangChain

LangFlow

Retool

Streamlit

Gradio

AI security governance

Stakeholder management

Communication

Bachelor’s degree in CS

Education

Bachelor’s degree in CS

Tools

OpenAI

Azure OpenAI

Hugging Face

LangChain

LangFlow

Pinecone

Azure AI Studio

Azure Cognitive Search

Azure Cosmos DB

Docker

Kubernetes

Job description

Onward Legal Innovation seeks a Senior AI Software Engineer to design and deploy production-grade AI for a top US law firm. You will build secure AI products, leverage LLMs, RAG, and AI agents within Azure, delivering scalable solutions for document analysis, diligence, and legal research.

You will work in a hybrid setup across NY/Chicago, collaborating with attorneys, data scientists, and engineers to advance AI capabilities across the firm with strong governance and security focus.