Senior AI Software Engineer | Python | Azure | Legal Tech

Onward Legal Innovation; New York (NY); Hybrid; USD 176,000 - 264,000; Full time

Benefits offered by this job

Comprehensive benefits

Paid time off

Retirement plan

Learning budget

Job summary

Onward Legal Innovation seeks a Senior AI Software Engineer to design and deploy production-grade AI for a top US law firm. You will build secure AI products, leverage LLMs, RAG, and AI agents within Azure, delivering scalable solutions for document analysis, diligence, and legal research.

You will work in a hybrid setup across NY/Chicago, collaborating with attorneys, data scientists, and engineers to advance AI capabilities across the firm with strong governance and security focus.

Qualifications

• 4+ years of relevant experience in tech, professional services, or legal environments, ideal in a law firm or complex org.
• Strong Python development with SQL, APIs, databases, and data handling (structured/unstructured).
• Hands-on experience building LLM/Generative AI apps with OpenAI/Azure OpenAI, Hugging Face, LangChain or similar.
• Designing and implementing RAG pipelines and AI agents, using vector stores and knowledge graphs.
• Experience securing scalable AI solutions within Microsoft Azure.
• Familiarity with Docker, Kubernetes, CI/CD; Azure AI Studio and related Azure services advantageous.
• Experience building LLM evaluation frameworks using Promptfoo, OpenAI Evals, LangSmith.
• Experience creating internal AI apps with Retool/Streamlit/Gradio.

Responsibilities

• Design and build secure internal AI products and workflows using LLMs, vectors, orchestration, and modern UI tech.
• Develop RAG pipelines, AI agents, and applications for document analysis and legal research.
• Architect resilient AI solutions on Microsoft Azure with cross-team collaboration.
• Move AI products from prototyping to production deployment and monitoring.
• Build evaluation frameworks for accuracy, robustness, safety, and ROI.
• Collaborate with attorneys and stakeholders to identify high-value AI opportunities.
• Advise on prompt engineering, model choice, architecture, and evaluation practices.
• Create reusable libraries, docs, and patterns for firm-wide AI development.
• Evaluate new AI technologies, open-source frameworks, and vendor platforms.
• Support pilots, demonstrations, user training, and knowledge sharing.

Skills

Python

SQL

APIs

Databases

LLMs

Generative AI

RAG pipelines

AI agents

Vector databases

Pinecone

Knowledge graphs

Azure

Docker

Kubernetes

CI/CD

Azure AI Studio

Azure Cognitive Search

Azure Cosmos DB

Prompt engineering

LangChain

LangFlow

Retool

Streamlit

Gradio

AI security governance

Stakeholder management

Communication

Bachelor’s degree in CS

Education

Bachelor’s degree in CS

Tools

OpenAI

Azure OpenAI

Hugging Face

LangChain

LangFlow

Pinecone

Azure AI Studio

Azure Cognitive Search

Azure Cosmos DB

Docker

Kubernetes

Job description

Senior AI Software Engineer is required to design and build secure, production-ready AI solutions for one of the top five law firms in the US and most prestigious legal organisations globally. With a market leading budget invested in AI, technology and innovation, the firm is building serious engineering capability around Generative AI and creating innovative AI solutions that will fundamentally change how its attorneys serve clients and operate at scale.

Location: New York or Chicago - Hybrid 3 days per week. (can't sponsor H1B, muts be eligable to work)

This is an opportunity to work at the forefront of Generative AI, LLMs, RAG, AI agents and cloud engineering, taking sophisticated AI solutions from experimentation and evaluation through to production deployment and monitoring.

You’ll join a growing Innovation Engineering function and work alongside engineers, data scientists, innovation leaders and attorneys to solve complex, high-value problems. You’ll develop AI-powered products supporting areas such as document analysis, summarisation, legal research and diligence, while helping establish reusable AI engineering standards and architecture across a global organisation.

The position offers a base salary of $176,000–$264,000, alongside comprehensive benefits, paid time off, retirement benefits and ongoing learning and development opportunities.

Key skills required for the Innovation AI Developer

• 4+ years of relevant experience across technical, professional services or legal environments, ideally within a law firm or similarly complex or regulated organisation.
• Strong Python development skills together with proficiency in SQL, APIs, databases and structured and unstructured data.
• Hands‑on experience building applications using LLMs and Generative AI, with technologies such as OpenAI, Azure OpenAI, Hugging Face, LangChain, LangFlow or comparable frameworks.
• Strong experience designing and implementing RAG pipelines and AI agents, including vector databases such as Pinecone and/or knowledge graph‑backed architectures.
• Experience designing secure, scalable AI solutions within Microsoft Azure.
• Familiarity with Docker, Kubernetes and CI/CD; experience with Azure AI Studio, Azure Cognitive Search and Azure Cosmos DB would be advantageous.
• Experience developing LLM evaluation frameworks using Promptfoo, OpenAI Evals, LangSmith or similar tooling.
• Experience building accessible internal AI applications using Retool, Streamlit or Gradio.
• Understanding of AI security, privacy and governance requirements when working with highly sensitive information.
• Ability to lead technical discovery, quantify ROI, establish acceptance criteria and communicate complex technical concepts to non-technical stakeholders.
• Bachelor’s degree required, preferably in Computer Science or a related discipline.
• Strong communication and stakeholder management skills, with the ability to influence technical decisions and support the development of other engineers.

What you’ll do as an Innovation AI Developer

• Design and build secure internal AI products and workflows using LLMs, vector databases, orchestration frameworks and modern UI technologies.
• Develop RAG pipelines, AI agents and intelligent applications supporting document analysis, diligence, summarisation and legal research.
• Architect resilient AI solutions on Microsoft Azure, collaborating with infrastructure and security teams across cloud, on‑premise and hybrid environments.
• Take AI products through the complete engineering lifecycle, from experimentation and prototyping through evaluation, production deployment and monitoring.
• Build sophisticated evaluation frameworks measuring accuracy, robustness, safety, business value and use‑case alignment.
• Partner with attorneys, business leaders and technical stakeholders to identify high‑value AI opportunities and turn complex requirements into practical products.
• Advise on prompt engineering, model selection, architecture and evaluation methodologies.
• Build reusable libraries, technical documentation and solution patterns to accelerate AI development across the firm.
• Evaluate emerging AI technologies, open‑source frameworks, vendor platforms and third‑party applications.
• Support AI pilots, demonstrations, user training and feedback programs while contributing to internal knowledge sharing and thought leadership.
• Collaborate across Innovation, engineering and data science teams and help coordinate technical workstreams.

Why this role?

This is a chance to join a top‑five US law firm that is making AI and technology a major strategic priority, this isn’t an environment where AI is confined to small‑scale proofs of concept — the ambition is to build secure, scalable technology that can be deployed across a sophisticated global legal business.

You’ll have the opportunity to build real‑world, production‑grade Generative AI products in an environment where security, accuracy and engineering quality are critical. The combination of LLMs, RAG, AI agents, Azure and complex proprietary data creates significant technical depth and the opportunity to solve problems where AI can have meaningful commercial impact.

For an AI engineer looking to apply emerging technology at serious scale, this offers the combination of substantial investment, sophisticated use cases, high‑calibre technical teams and direct access to senior legal and business stakeholders.