Senior API & Data Connectors Engineer

Three Pillars Recruiting; San Francisco (CA); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Three Pillars Recruiting is seeking a Senior Software Engineer for our Data Products team in San Francisco. You will own and extend core data ingestion frameworks and SDK abstractions, building high-throughput native connectors and a Flask-based runtime service.

You will design extensible connectors, integrate PyAirbyte, and optimize data transfer while maintaining strong typing and developer ergonomics for teams consuming the data platform.

Qualifications

• 5+ years writing production Python software with strong OOP and typing.
• Experience building microservices/APIs using Flask for high-throughput data.
• Proven track record developing extensible SDKs or framework components.

Responsibilities

• Architect Connector Abstractions: define core interfaces for data ingestion.
• Build native connectors using DuckDB, Arrow, Iceberg with memory efficiency.
• Own the Flask runtime service exposing catalog, check, discover, read.
• Integrate PyAirbyte to unlock 300+ external data sources.
• Optimize data transfer semantics for low latency data passing.
• Enhance developer experience with well-typed, extensible plugins.

Skills

Python

APIs & Microservices

High-Performance Data

Tools

Flask

Docker

Git

pytest

Poetry

Job description

Three Pillars Recruiting is seeking a Senior Software Engineer for our Data Products team in San Francisco. You will own and extend core data ingestion frameworks and SDK abstractions, building high-throughput native connectors and a Flask-based runtime service.

You will design extensible connectors, integrate PyAirbyte, and optimize data transfer while maintaining strong typing and developer ergonomics for teams consuming the data platform.