LLM Engineer

Licorne Society; Paris; On-site; EUR 75,000 - 110,000; Full time

Job summary

Licorne Society is building a high‑reliability AI coworker for professional services in Paris. You will own and scale production AI systems, focusing on fast, reliable outputs across workflows like emails and document analysis.

The role emphasizes evaluating performance, reducing hallucinations, and iterating rapidly from live data. Collaboration with product and founders shapes what to build next.

Qualifications

• Shipping real LLM systems with measurable performance.
• Evaluation-driven development mindset.
• Ability to debug complex failures in production.
• Fast iteration cycles from production data.
• Strong technical judgment in system design.

Responsibilities

• Design and evolve LLM/agent architecture for production use.
• Own output quality across core use cases (emails, document analysis, workflows).
• Build evaluation systems (datasets, metrics, regression detection).
• Drive fast iteration loops from production data and feedback.
• Improve retrieval, reasoning, and tool usage; ensure reliability and latency targets.
• Collaborate with product and founders on what to build and why.

Skills

Shipping Real LLM Systems

Evaluation-Driven Development

Debugging Complex Failures

Speed of Iteration

Strong Technical Judgment

Tools

LangGraph

LangChain

PostHog

Langfuse

Azure OpenAI

GCP

PostgreSQL

FastAPI

Job description

We are building the AI coworker for legal and professional services - automating emails, document analysis, and complex workflows in high‑stakes industries.

We are not looking for someone to "use GPT APIs." We are looking for someone to own and scale a production AI system that users rely on daily.

What you will own

You will be responsible for one primary outcome: making our AI outputs reliable, fast, and indispensable in real workflows.

Concretely:

• Design and evolve our LLM / agent architecture.
• Own output quality across key use cases (emails, document analysis, etc.).
• Build evaluation systems (datasets, metrics, regression detection).
• Drive fast iteration loops from production data.
• Improve retrieval, reasoning, and tool usage.
• Ensure production reliability (latency, failure modes, fallbacks).
• Work directly with product and founders on what to build and why.

What this role is really about

Most teams fail because they don’t know what "good output" means, lack proper evals, iterate randomly, or overuse agents.

Your job is to fix that. You will turn vague user problems into structured AI systems with measurable performance that improve every week.

What You Need to Be Excellent At

1. Shipping Real LLM Systems
2. Evaluation-Driven Development
3. Debugging Complex Failures
4. Speed of Iteration
5. Strong Technical Judgment

What we don’t care about

• Number of years of experience.
• Whether you’ve used a specific framework.
• Fancy research credentials.

If you can build, debug, and improve real systems, you’re a fit.

What success looks like (First 90 Days)

• A clear evaluation framework established for core use cases.
• Measurable, quantitative improvement in output quality.
• Faster iteration cycles across the team.
• Significantly reduced hallucinations and failure modes.
• Robust, scalable system architecture decisions.

Tech Stack (Context, Not Strict Requirements)

• Language/Backend: Python (FastAPI)
• Database: Postgres
• Cloud: Google Cloud Platform (GCP)
• Orchestration: LangGraph / LangChain (evolving)
• Observability: PostHog (analytics), Langfuse (LLM tracing)
• LLM Infrastructure: Azure OpenAI / Multi-provider APIs