Global Data Ingestion Architect – Provenance & Reliability

Kilwa; Chicago (IL); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Kilwa Technologies is hiring a Data Engineer to build and maintain the ingestion backbone across 54 markets and 85 languages. You will own pipelines for macroeconomic, policy, and news data, ensuring provenance and data quality at every step.

The role requires handling messy international sources and maintaining platform reliability with strong encryption. Location is Chicago or remote within the US, full-time in the Engineering team.

Qualifications

• Experience building production-grade data pipelines across diverse sources.
• Strong focus on data quality as a premier feature of the product.
• Ability to handle messy, multilingual data sources and formats.

Responsibilities

• Own ingestion pipelines across 54 markets and 85 languages.
• Maintain end-to-end provenance for inputs and model outputs.
• Ensure data processing meets reliability and security standards (AES-256).
• Operate the platform with robust monitoring and incident response.

Skills

Production data pipelines

Data quality mindset

International data sources

Tools

Airflow

Spark

Kubernetes

Job description

Kilwa Technologies is hiring a Data Engineer to build and maintain the ingestion backbone across 54 markets and 85 languages. You will own pipelines for macroeconomic, policy, and news data, ensuring provenance and data quality at every step.

The role requires handling messy international sources and maintaining platform reliability with strong encryption. Location is Chicago or remote within the US, full-time in the Engineering team.