Software Engineer, Data Pipelines

Reinforced Learning; Town of Tonawanda (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Reinforced Learning in the United States (New York) seeks an expert for building and maintaining data pipelines that validate, transform, and ship expert submissions to customers. You will implement schemas, quality gates, and dataset versioning with lineage from raw submissions to delivered files.

Delivery tooling must be secure and operable by dataset leads. The role emphasizes production experience with batch and streaming data in cloud environments, pragmatic tooling decisions, and a strong

Qualifications

• Proficient in Python and data pipelines.
• Experience in production data systems.
• Working with batch and streaming data in cloud environments.

Responsibilities

• Build pipelines to validate and transform expert submissions into customer formats.
• Define schemas and quality gates to prevent bad records from shipping.
• Implement dataset versioning and lineage from raw input to delivered files.
• Develop delivery tooling for secure handoffs into customer environments.
• Ensure pipelines are observable and maintainable for dataset leads.

Skills

Python

Batch processing

Streaming data

Data quality

Tools

Cloud platforms

Job description

The role Expert work arrives as trajectories, documents, diffs and preferences. Customers receive versioned, validated, documented datasets. Everything in between is yours: ingestion, validation, transformation, packaging and delivery.

What you’ll own

• Pipelines that validate and transform expert submissions into customer formats.
• Schemas and quality gates, so bad records never reach a delivery.
• Dataset versioning and lineage from raw submission to shipped file.
• Delivery tooling for secure handoff into customer environments.
• Pipelines that are observable and easy for dataset leads to operate.

What you bring

• Strong Python and experience with batch and streaming data systems.
• Care for data correctness: schemas, idempotency, backfills, audits.
• Pipelines you’ve operated in production on a major cloud.
• Pragmatism about tooling.

Helpful, not required

• Dataset formats used in model training.
• Data contracts or lineage tooling.