Data Engineer — Real-Time Streaming & Data Platforms

Infinitive Inc; McLean (VA); On-site; USD 104,000 - 140,000; Full time

Job summary

Infinitive Inc in the United States is seeking an experienced Data Engineer to design, build, and scale event-driven data platforms. You will bridge real-time streaming with durable workflows using Kafka and Temporal, focusing on strict data contracts and robust schema design.

You will develop batch and streaming ETL/ELT pipelines with Python, Spark, and cloud data warehouses, ensure data quality, and collaborate across teams to deliver reliable, high-throughput data infrastructure.

Qualifications

• 4+ years of professional experience in data engineering, backend distributed systems, or software engineering.
• Hands-on experience with Temporal (or Cadence): Proven understanding of durable workflows, activities, retries, signals, queries, and long-running distributed task orchestration.
• Deep expertise with Apache Kafka: Practical experience with message partitioning, consumer groups, offset management, and topic design.
• Strong background in Data Schema Design & Validation: proficiency with schema definition frameworks (Avro, Protobuf/gRPC, or JSON Schema).
• Experience enforcing data validation rules, contract testing, and data quality checks (e.g., Great Expectations, Pandera, Pydantic, dbt tests).
• Strong programming proficiency in Python (Go or Java is a plus) with clean code and testing standards.
• Distributed computing experience: hands-on development with Apache Spark (PySpark/Spark SQL).
• Advanced SQL & Data Modeling: Relational databases, dimensional modeling, performance tuning.

Responsibilities

• Stream Processing & Messaging: Architect, deploy, and maintain high-volume distributed data streams using Apache Kafka (producers, consumers, Kafka Connect, Schema Registry).
• Data Schema Design & Validation: Establish and enforce schema standards, versioning, and automated validation across microservices, streaming consumers, and lakehouse storage.
• Resilient Workflow Orchestration: Design durable execution workflows using Temporal to coordinate long-running distributed pipelines and cross-system ETL tasks.
• Pipeline Development: Build end-to-end batch and near-real-time pipelines using Python, SQL, and Apache Spark / PySpark.
• Data Modeling & Warehousing: Design and optimize analytical data models in cloud data warehouses/lakehouses (Snowflake, BigQuery, Databricks, Redshift).
• Reliability & Data Quality: Implement automated testing, continuous schema validation, data drift detection, and observability across workflows.
• Cross-Functional Collaboration: Partner with software engineers, ML engineers, and analysts to define data contracts and CI/CD release patterns.

Skills

Temporal

Apache Kafka

Python

SQL & Data Modeling

Spark / PySpark

Data quality & validation

Tools

Confluent Schema Registry

dbt

Great Expectations

Pydantic

Job description

Infinitive Inc in the United States is seeking an experienced Data Engineer to design, build, and scale event-driven data platforms. You will bridge real-time streaming with durable workflows using Kafka and Temporal, focusing on strict data contracts and robust schema design.

You will develop batch and streaming ETL/ELT pipelines with Python, Spark, and cloud data warehouses, ensure data quality, and collaborate across teams to deliver reliable, high-throughput data infrastructure.