Senior Data Engineer – Real-Time AI Pipelines (Remote)

Linkup; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 230,000; Full time

Benefits offered by this job

Flexible remote-work policy

Job summary

Linkup is seeking a Senior Data Engineer in San Francisco to build and operate large-scale data pipelines powering AI-ready web search. You will design ETL/ELT workflows, optimize data ingestion, and manage scalable storage across SQL/NoSQL and object stores.

You will collaborate with ML and backend teams, contribute to system architecture, and ensure high data quality, observability, and low-latency data delivery for ranking and retrieval.

Qualifications

• 3+ years of data engineering or backend development experience
• Experience with distributed systems and streaming architectures
• Focus on data quality, observability, and reliability
• Ability to debug pipelines and optimize for latency/throughput
• Interest in AI infrastructure, indexing, and retrieval-augmented generation
• Thrives in fast-paced, high-ownership startup environment

Responsibilities

• Build and operate large-scale data pipelines powering AI-ready web search
• Design and maintain ETL/ELT workflows for web indexing, embeddings, and analytics
• Develop and optimize data ingestion and transformation across sources
• Manage and scale data storage layers (SQL/NoSQL/object stores)
• Participate in architecture discussions and system design for data infra
• Collaborate with ML and backend teams to deliver real-time data for ranking/retrieval
• Write clean, maintainable code and participate in code reviews

Skills

Data pipelines

Distributed systems

Back-end development

Observability

Latency optimization

AI infrastructure

Startup environment

Tools

PostgreSQL

OpenSearch

Vespa

Apache Storm

Kubernetes

GitHub Actions

Terraform

Job description

Linkup is seeking a Senior Data Engineer in San Francisco to build and operate large-scale data pipelines powering AI-ready web search. You will design ETL/ELT workflows, optimize data ingestion, and manage scalable storage across SQL/NoSQL and object stores.

You will collaborate with ML and backend teams, contribute to system architecture, and ensure high data quality, observability, and low-latency data delivery for ranking and retrieval.