Remote Lead Data Engineer — Real-Time Data Pipelines

Empower LLC; United States; Remote; USD 132,000 - 191,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) with company match

Tuition reimbursement

Generous paid time off

Parental leave and volunteer time

Inclusive BRGs

Job summary

Empower LLC is seeking a Lead Data Engineer to architect and optimize scalable data pipelines and real-time ingestion frameworks in a remote nationwide capacity. You will mentor engineers, collaborate with architects, and ensure governance, security, and data quality across enterprise platforms.

The role requires 6–8+ years of experience with Kafka, real-time streaming, and cloud data platforms, plus strong leadership and Agile collaboration skills.

Qualifications

• 6–8+ years of data engineering on enterprise-scale platforms.
• Deep experience with Kafka, Confluent Cloud, and AWS Kinesis.
• Proficiency in Python, PySpark, and SQL; production use of Flink/Kafka Streams/Spark Streaming.
• Strong knowledge of CDC tools and schema evolution with Registry.
• Hands-on with Docker, Kubernetes, Terraform, and CI/CD for event-driven microservices.
• Experience with lakehouse formats (Iceberg/Delta Lake) and cloud warehouses (Snowflake/BigQuery).
• Experience leading technical initiatives and mentoring engineers; Agile environment experience.

Responsibilities

• Lead design and implementation of advanced data pipelines and real-time streaming solutions.
• Build and maintain scalable Kafka producers/consumers for sub-second latency.
• Configure CDC and integrate across relational and NoSQL systems into Kafka topics.
• Develop stream-processing solutions using Flink, Spark Streaming, or Kafka Streams.
• Implement schema evolution and data contracts using Schema Registry.
• Guide automation with AI agents (LangGraph, AutoGen, CrewAI) for schema drift and routing.
• Monitor and optimize data platforms for performance, reliability, and governance.
• Mentor engineers and drive architectural decisions with stakeholders.

Skills

Kafka

Confluent Cloud

AWS Kinesis

Apache Pulsar

Python

PySpark

SQL

Java

Scala

Go

Flink

Kafka Streams

Spark Structured Streaming

Schema Registry

Docker

Kubernetes

Terraform

CI/CD

LangGraph

AutoGen

CrewAI

Iceberg

Delta Lake

Education

Bachelor's degree in Computer Science or related field

Tools

Apache Kafka

Confluent Cloud

AWS Kinesis

Apache Pulsar

Python

Java

Scala

Go

Spark

Flink

Delata/Iceberg

Snowflake/BigQuery

Docker

Kubernetes

Terraform

Job description

Empower LLC is seeking a Lead Data Engineer to architect and optimize scalable data pipelines and real-time ingestion frameworks in a remote nationwide capacity. You will mentor engineers, collaborate with architects, and ensure governance, security, and data quality across enterprise platforms.

The role requires 6–8+ years of experience with Kafka, real-time streaming, and cloud data platforms, plus strong leadership and Agile collaboration skills.