Lead Data Engineer — MLOps & LLMs

Ranwalk; California; On-site; USD 170,000 - 230,000; Full time

Benefits offered by this job

Equity incentives

Free lunches

Job summary

Random Walk is seeking a hands‑on Lead Data Engineer to own our data and ML infrastructure. You will build scalable pipelines for high‑volume email text data, and engineer LLM‑driven analysis with API integrations and fine‑tuning.

You will lead end‑to‑end ML lifecycle, deploy and monitor models in production, and collaborate with backend teams to minimize latency. Southern California is preferred; equity incentives may accompany compensation.

Qualifications

• 5+ years of experience in data engineering or backend engineering with production data‑intensive apps and ML systems.
• Experience with end‑to‑end ML lifecycle, deployment, monitoring, and maintenance of models; personalization experience is a plus.
• Hands‑on experience building applications that use LLMs via APIs or open‑source models; familiarity with LangChain or Hugging Face is desirable.
• Strong backend development with Python or Go.
• Deep experience with cloud platforms (AWS, GCP, or Azure), containers (Docker, Kubernetes), and data orchestration tools (Airflow/Prefect).
• Bachelor’s or Master’s degree in CS/Engineering or related field.
• Leadership, communication, and agile work style; passion for consumer‑centric experiences.

Responsibilities

• Build, scale, and maintain data pipelines to process high‑volume email text data.
• Engineer systems leveraging LLMs for semantic analysis and feature extraction at scale; optimize latency and cost.
• Own end‑to‑end ML lifecycle: train, deploy, monitor, and update models in production.
• Design and manage cloud‑based infrastructure for data processing, model training, and real‑time inference.
• Collaborate with backend engineers to integrate data services and ML models with the core app.
• Implement privacy‑by‑design practices ensuring anonymized, non‑PII data.

Skills

Backend development (Python/Go)

MLOps lifecycle

LLM integration

Cloud platforms

Data pipelines

Leadership

Education

Bachelor’s or Master’s in CS/Engineering

Tools

LangChain

Hugging Face

Airflow

Prefect

Docker

Kubernetes

Python

Go

Job description

Random Walk has delivered best in class alternative data insights focused on consumer email behavior for over a decade.

We are seeking a Lead Data Engineer to serve as the technical owner for the infrastructure and systems powering an exciting new product. This product is focused on improving how Fortune 500 brands market new products and services with customers and leads. This is a hands-on role where you will lead our technical processes and manage a lean development team, directly impacting the systems that help leading institutional asset managers uncover market inflections.

Role & Responsibilities

This role is focused on building and scaling the data and machine learning infrastructure.

• Build Data Pipelines: Design, build, and maintain robust, scalable data pipelines to process high-volume email text data.
• Implement LLM Systems: Engineer systems that leverage LLMs to perform semantic analysis and feature extraction on text data at scale. This includes API integration, fine-tuning, and optimizing for latency and cost.
• MLOps & Deployment: Own the end-to-end lifecycle of machine learning models. You will build, deploy, and maintain our primary recommendation and personalization models (e.g., collaborative filtering, gradient boosting, neural networks) in a production environment.
• Infrastructure Ownership: Design and manage the cloud-based infrastructure required for data processing, model training, and real-time inference, ensuring high availability and scalability.
• System Integration: Collaborate closely with the backend engineering team to integrate data services and ML models into our core application. The models must execute with minimal latency to ensure a seamless user experience.
• Privacy by Design: Implement and enforce data handling and modeling practices that are fully compliant with our strict privacy-first approach, ensuring all systems work exclusively with anonymized, non-PII data.

We are looking for a hands-on builder with a strong engineering background.

Qualifications

Experience: 5+ years of experience in data engineering or backend engineering, with a proven track record of building and deploying data-intensive applications and machine learning systems in production.

ML in Production: Demonstrated experience with the full MLOps lifecycle, including deploying, monitoring, and maintaining machine learning models. Experience with personalization and recommendation systems is a major plus.

LLM Application: Hands-on experience building applications that use Large Language Models (LLMs) via APIs or by deploying open-source models. Familiarity with frameworks like LangChain or Hugging Face is highly desirable.

Backend Proficiency: Strong proficiency in backend development with languages like Python or Go.

Infrastructure & Cloud: Deep experience with cloud platforms (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and data orchestration tools (e.g., Airflow, Prefect).

Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.

Leadership: Strong communication and leadership skills with a passion for building consumer-centric experiences in an agile environment.

Location: Preference for candidates located in Southern California

Position

• Competitive salary and equity incentives
• Free lunches

please submit CV with brief 4-5 sentences note specifically describing how your recent experiences will help lead in this role.