Senior AI Data Engineer - RAG & LLM Infra

Cooley AI; New York (NY); On-site; USD 220,000 - 250,000; Full time

Job summary

Cooley AI, the innovation arm of the leading technology law firm, seeks a Senior AI Data Engineer to build production AI infrastructure powering Northstar Lakehouse-based applications. You will own RAG pipelines, embedding strategies, and LLM integration, ensuring reliability and scale in a high-stakes data environment.

You will collaborate with the Data Engineering & Platform teams, implement observability, and advance AI feature maturity while adhering to governance and security standards

Qualifications

• 5+ years of data engineering or AI engineering experience with hands-on production responsibility for RAG pipelines, LLM integration, or AI-powered data infrastructure

Responsibilities

• Design, build, and maintain production RAG pipelines that ingest, process, chunk, embed, and index firm data assets for retrieval by AI-powered legal intelligence applications.
• Own the vector search implementation including index design, embedding model selection and maintenance, retrieval strategy configuration, and ongoing retrieval quality assessment.
• Build and maintain embedding pipelines that transform structured and unstructured firm data assets into vector representations used by retrieval systems and LLM-powered features.
• Monitor and maintain retrieval quality in production, surface degradation early, and collaborate with governance controls.
• Build and maintain the LLM integration layer for Data Applications squad products with prompt engineering, validation, latency management, and cost-aware API usage.
• Implement AI feature observability tooling for LLM response quality, retrieval relevance, and latency patterns.
• Apply responsible AI practices aligned with firm governance and ongoing AI landscape awareness.
• Collaborate with Senior Data Application Engineer and other squad functions to ensure integration across application and intelligence layers.

Skills

Data engineering

AI engineering

RAG pipelines

LLM integration

Python proficiency

Databricks

Vector search

CI/CD

Agile delivery

Tools

Databricks Delta Lake

Unity Catalog

Databricks SQL

Databricks Appkit

Databricks Vector Search

Pinecone

Job description

Cooley AI, the innovation arm of the leading technology law firm, seeks a Senior AI Data Engineer to build production AI infrastructure powering Northstar Lakehouse-based applications. You will own RAG pipelines, embedding strategies, and LLM integration, ensuring reliability and scale in a high-stakes data environment.

You will collaborate with the Data Engineering & Platform teams, implement observability, and advance AI feature maturity while adhering to governance and security standards