Data Engineer

Infinitive Inc; McLean (VA); On-site; USD 90,000 - 154,000; Full time

Job summary

Infinitive Inc is hiring a Data Engineer to design, build, and scale next-generation, event-driven data platforms in McLean, VA onsite. You will work on high-volume streaming systems with Apache Kafka, Temporal, and schema contracts across microservices and lakehouse storage.

You will develop end-to-end batch and near-real-time pipelines using Python, SQL, and Spark, and model data in Snowflake, BigQuery, Databricks, and Redshift.

Qualifications

• 4+ years of data engineering or backend distributed systems experience.
• Experience with Temporal or Cadence and durable workflows.
• Deep expertise in Apache Kafka and related tooling.
• Schema definition frameworks like Avro, Protobuf, or JSON Schema.
• Experience with schema evolution and registries (Confluent, AWS Glue).
• Ability to perform data validation and quality checks with GE, Pandera, Pydantic, or dbt tests.
• Strong Python and Spark (PySpark) skills are required.

Responsibilities

• Architect, deploy, and maintain high-volume data streams with Kafka and Connect.
• Define schema standards, versioning, and automated validation across services.
• Build durable Temporal workflows coordinating long-running pipelines.
• Create end-to-end batch and near-real-time pipelines with Python, SQL, Spark.
• Model analytics in Snowflake, BigQuery, Databricks, and Redshift.
• Implement testing, data drift detection, and observability for pipelines.
• Collaborate with engineers and analysts on data contracts and CI/CD release patterns.

Skills

Python

SQL

Distributed systems

Go

Java

Tools

Temporal

Cadence

Apache Kafka

Kafka Connect

Schema Registry

Avro

Protobuf

JSON Schema

Confluent Schema Registry

AWS Glue Schema Registry

Great Expectations

Pandera

Pydantic

dbt

Spark

PySpark

Spark SQL

Snowflake

BigQuery

Databricks

Redshift

Python

SQL

gRPC

Job description

Infinitive Inc is hiring a Data Engineer to help design, build, and scale next-generation, event-driven data platforms in McLean, VA (onsite). The role blends high-volume streaming with durable workflow orchestration, with a strong focus on schema discipline and reliable data pipelines spanning batch and near-real-time processing.

In this position, you will work on distributed systems using Apache Kafka for streaming data movement and Temporal for long-running pipeline coordination. You will also design robust data contracts through schema versioning and automated validation, then connect that rigor to analytical models in cloud data warehouses and lakehouses.

What you’ll do

• Architect, deploy, and maintain high-volume distributed data streams using Apache Kafka, including producers, consumers, Kafka Connect, and Schema Registry.
• Define and enforce schema design standards, including versioning strategies and automated schema validation (Avro, Protobuf, JSON Schema) to keep strict data contracts across microservices, streaming consumers, and lakehouse storage.
• Build durable execution workflows with Temporal to coordinate long-running pipelines, handle cross-system ETL tasks, and apply compensation patterns such as the Saga pattern.
• Create end-to-end batch and near-real-time pipelines using Python, SQL, and Apache Spark / PySpark.
• Design and optimize analytical data models (dimensional/star schema) in modern cloud data platforms such as Snowflake, BigQuery, Databricks, and Redshift.
• Implement automated testing, continuous schema validation, data drift detection, and observability for both streaming and batch workflows.
• Collaborate with software engineers, machine learning engineers, and analysts to define standard schema definitions, data contracts, and production-grade CI/CD release patterns.

What you bring

• 4+ years of professional experience in data engineering, backend distributed systems, or software engineering.
• Hands‑on experience with Temporal (or Cadence), including durable workflows concepts such as activities, retries, signals, queries, and long‑running orchestration.
• Deep expertise in Apache Kafka, including message partitioning, consumer groups, offset management, and topic design.
• Practical experience with schema definition frameworks: Apache Avro, Protocol Buffers/gRPC, or JSON Schema.
• Experience managing schema evolution and compatibility modes (backward, forward, full), plus schema registries such as Confluent Schema Registry or AWS Glue Schema Registry.
• Ability to enforce data validation rules, contract testing, and data quality checks using tools such as Great Expectations, Pandera, Pydantic, or dbt tests.
• Strong programming proficiency in Python (Go or Java is a plus), with clean code, design patterns, and unit/integration testing standards.
• Hands‑on development with Apache Spark (PySpark/Spark SQL) for large‑scale data processing.
• Strong relational database experience, dimensional data modeling, and query performance tuning.

Tools you’ll work with

Apache Kafka, Kafka Connect, Schema Registry, Temporal, Cadence, Apache Avro, Protocol Buffers, gRPC, JSON Schema, Confluent Schema Registry, AWS Glue Schema Registry, Great Expectations, Pandera, Pydantic, dbt, Python, SQL, Apache Spark, PySpark, Spark SQL, Snowflake, BigQuery, Databricks, Redshift.

Compensation

Infinitive Inc is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. A reasonable estimate of the current range for this role in the U.S. is $90,000 - $154,00.00 per year.