Senior Data Engineer — Remote, AI-Driven Pipelines

Lighthouse; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Flexible time off

Alan Flex benefits: 160€/month forfood

Flexible retribution: tax-free payroll

ClassPass wellbeing subscription

Comprehensive health insurance

Impactful work: 85,000+ users

Referral bonuses

Job summary

Lighthouse is seeking a Senior Data Engineer to own the end-to-end data ingestion and transformation pipelines at scale. You’ll design, build, and maintain systems handling massive data streams (≈100 TB/day) and drive quality through observability and testing.

You’ll collaborate with product, engineering, support and customers while leveraging Python, dbt, Airflow and cloud platforms to accelerate delivery. Join a fast-growing, global team powering Lighthouse’s intelligence.

Qualifications

• You have experience building and maintaining production data pipelines.
• You take pride in the software you build, from initial design to production monitoring and continuous improvement.
• You have professional proficiency in Python for large-scale data processing and pipeline development.
• You have strong knowledge of data cloud technologies and hands-on experience with a major cloud platform like GCP, AWS, or Azure.
• You are a forward-thinking builder who uses AI-assisted tools to ship high-quality code faster.
• You are fluent in English with excellent communication skills to discuss technical details and explain complex data flows to technical stakeholders.

Responsibilities

• Design, build, and deploy scalable data pipelines that ingest data from various sources like APIs, Webhooks, and Cloud Storage.
• Implement complex domain modeling and transformation logic to unify disparate data sources into high-fidelity data products.
• Contribute to architectural improvements by transitioning legacy processes into modern, scalable, and stable data architectures.
• Own the support and troubleshooting of live integrations, from diagnosing hotel and internal issues to communicating clearly with stakeholders during incidents.
• Drive operational excellence by improving the reliability and quality of our data pipelines through advanced observability, automated testing, and automation.
• Collaborate closely with product, engineering, support, and business teams, as well as customers and partners.
• Investigate and resolve complex technical issues by performing deep-dive analysis into our code and systems to identify root causes.
• Leverage AI and automation to increase engineering velocity, streamline complex debugging, and handle recurring tasks.

Skills

Python programming

Data pipeline design

Cloud data platforms

Distributed systems

Communication

Tools

dbt

Airflow

BigQuery

Kubernetes

GCP

AWS

Job description

Lighthouse is seeking a Senior Data Engineer to own the end-to-end data ingestion and transformation pipelines at scale. You’ll design, build, and maintain systems handling massive data streams (≈100 TB/day) and drive quality through observability and testing.

You’ll collaborate with product, engineering, support and customers while leveraging Python, dbt, Airflow and cloud platforms to accelerate delivery. Join a fast-growing, global team powering Lighthouse’s intelligence.