Lead Data Engineer – Real-Time Pipelines & Leadership

Talanto; Northern (KY); Hybrid; USD 125,000 - 182,000; Full time

Benefits offered by this job

Medical, dental, vision and life

401(k) with company match

Tuition reimbursement

Generous PTO and holidays

Volunteer time

Inclusive BRGs

Job summary

Talanto seeks a Lead Data Engineer to drive design, development and optimization of scalable data pipelines and real-time ingestion frameworks. The role leads complex data initiatives, mentors engineers, and aligns solutions with enterprise standards.

You will implement CDC, stream-processing, and schema governance across multiple data sources, emphasizing reliability and performance in a cloud-based lakehouse environment.

Qualifications

• Bachelor's degree in computer science or a related field or equivalent experience.
• 6-8+ years of data engineering with enterprise-scale platforms.
• Deep experience with Kafka, Confluent Cloud, AWS Kinesis, or Pulsar.
• Proficiency in Python, PySpark, and SQL.
• Production experience with Flink, Kafka Streams, or Spark Streaming.

Responsibilities

• Lead design and implementation of advanced data pipelines and real-time streaming
• Build and maintain scalable Kafka producers/consumers for sub-second latency
• Configure CDC solutions across relational and NoSQL sources
• Develop stream-processing logic with Flink, Spark, Kafka Streams
• Enforce schema evolution using Confluent Schema Registry
• Integrate AI agents to support schema drift and dynamic routing
• Instrument end-to-end monitoring and handle offsets, rebalances, DLQs
• Mentor engineers and lead architectural decisions
• Ensure governance, security, and data quality standards

Skills

Apache Kafka

Python

SQL

Distributed systems

Cloud platforms

Education

Bachelor's degree in CS or related field

Tools

Debezium

Kafka Connect

AWS DMS

Iceberg/Delta Lake

Job description

Talanto seeks a Lead Data Engineer to drive design, development and optimization of scalable data pipelines and real-time ingestion frameworks. The role leads complex data initiatives, mentors engineers, and aligns solutions with enterprise standards.

You will implement CDC, stream-processing, and schema governance across multiple data sources, emphasizing reliability and performance in a cloud-based lakehouse environment.