AI Engineer (Applied LLM Systems)

Discernis; New York (NY); On-site; USD 140,000 - 200,000; Full time

Job summary

Discernis is seeking a lead AI architect in New York to design and evolve AI-powered workflows that help legal teams understand massive document sets with speed and reliability.

You will own end-to-end ML features, including self-hosted model serving, evaluation frameworks, and domain adaptation, using a full toolset from prompt engineering to retrieval and post-training.

Qualifications

• Experience building production LLM or ML powered features end to end.
• Hands on experience with agentic or multi step LLM orchestration and tool use.
• Breadth across the modern LLM toolkit: prompt engineering and retrieval, and post-training methods such as fine tuning, DPO, or RLHF.
• Experience designing and running evaluations for LLM systems.
• Hands on experience with self hosted model serving such as vLLM, TGI, or similar.
• Strong Python skills and comfort with concurrent or distributed processing.
• Bonus: embedding and vector search pipelines, or retrieval augmented generation at scale.
• Bonus: background in legal tech, document intelligence, or information retrieval.

Responsibilities

• Design and build new AI powered workflows that solve real problems for legal teams, using agentic and multi-step orchestration, prompt engineering, retrieval, and post-training
• Improve quality, reliability, latency, and cost of existing workflows with the full toolkit
• Own and develop evaluation and benchmarking frameworks to measure and improve quality and detect regressions
• Design agentic patterns and structured outputs robust for complex legal and investigative use cases
• Own self-hosted model serving and inference optimization across customer deployments
• Post-train and adapt models including fine tuning, preference optimization, and domain adaptation

Skills

Production LLM features

Orchestration & multi-step workflows

Prompt engineering

Retrieval

Post-training methods (Fine-tuning, DP

Self-hosted model serving (vLLM)

Python

Vector search / RAG

Legal tech / document intelligence

Tools

vLLM

TGI

Job description

The Role

You will lead the design and evolution of the AI systems at the core of our product. Day to day, that means inventing new AI powered workflows that help legal teams understand massive document sets, then making them faster and more reliable, and building the evaluations that tell us we are getting it right. You will reach for whatever technique fits the problem, prompt engineering, agentic orchestration, retrieval, or post-training, and because our customers often cannot send data to external providers, much of this runs on models we host and improve ourselves.

What You Will Do

• Design and build new AI powered workflows that solve real problems for legal teams, using whatever gets the best result: agentic and multi step orchestration, prompt engineering, retrieval, and post-training
• Improve the quality, reliability, latency, and cost of existing workflows using that same full toolkit
• Own and develop our evaluation and benchmarking frameworks so we can measure and improve quality, catch regressions, and know we are moving in the right direction
• Design agentic patterns and structured outputs robust enough for complex legal and investigative use cases
• Own self hosted model serving and inference optimization across customer deployments
• Post-train and adapt models where that is the right lever, including fine tuning, preference optimization, and domain adaptation

What You Bring

• Experience building production LLM or ML powered features end to end, with the judgment to know which technique fits which problem
• Hands on experience with agentic or multi step LLM orchestration and tool use
• Breadth across the modern LLM toolkit: prompt engineering and retrieval, and ideally post-training methods such as fine tuning, DPO, or RLHF
• Experience designing and running evaluations for LLM systems
• Hands on experience with self hosted model serving such as vLLM, TGI, or similar
• Strong Python skills and comfort with concurrent or distributed processing
• Bonus: embedding and vector search pipelines, or retrieval augmented generation at scale
• Bonus: background in legal tech, document intelligence, or information retrieval

Tech Environment

Python, self hosted inference (vLLM), LLM orchestration and agent frameworks, evaluation tooling

Apply Here: https://app.dover.com/apply/Discernis%20AI/da075608-1260-4521-bdb6-f499b8d5fa26?rs=42706078