Real-Time Ingestion Engineer - Remote Ownership

NewsMesh; Northern (KY); Hybrid; USD 165,000 - 220,000; Full time

Benefits offered by this job

Remote across the US

Real ownership: ship work customers依赖

Small, senior team with high standards

Job summary

NewsMesh is seeking a backend engineer to own the ingestion pipeline from fetch to stored records. You will ensure high-volume, reliable data processing, deduplication, and accurate enrichment for real-time access via API.

The role emphasizes Python/SQL proficiency, Celery/RabbitMQ experience, and a bias for correctness. Remote work across the United States is supported, with a senior, low-process team driving quality.

Qualifications

• Strong backend engineering and fluency in Python and SQL.
• Experience with high-volume ETL, queues, or distributed workers.
• Bias for correctness and measuring before trusting results.
• Ability to own a system end to end from fetch to stored record.
• Experience with RSS, crawling, or document parsing at scale.
• Familiarity with Celery, RabbitMQ or similar task queues.
• History of identifying gnarly data-quality bugs.

Responsibilities

• Own and improve the Celery-based ingestion pipeline across thousands of changing sources.
• Keep articles fresh, deduplicated, and correct as source count grows.
• Harden fetching against blocks, rate limits, and feeds that quietly change shape.
• Build the monitoring that catches a broken source in minutes, not days.
• Own reliability: this pipeline runs continuously and customers notice when it stops.

Skills

Backend engineering

Python

SQL

ETL

Distributed workers

Data quality

Crawling

Document parsing

RSS

Bug hunting

Tools

Celery

RabbitMQ

Task queues

Job description

NewsMesh is seeking a backend engineer to own the ingestion pipeline from fetch to stored records. You will ensure high-volume, reliable data processing, deduplication, and accurate enrichment for real-time access via API.

The role emphasizes Python/SQL proficiency, Celery/RabbitMQ experience, and a bias for correctness. Remote work across the United States is supported, with a senior, low-process team driving quality.