Lead Data Engineer - Hadoop & Real-Time Pipelines

Robotic Technologies Automation; Cleveland (OH); On-site; USD 120,000 - 180,000; Full time

Job summary

ROBOTIC PROCESS AUTOMATION LLC seeks an experienced Data Engineering Lead to drive end-to-end data pipelines that power onboarding, transactions, and decisioning workflows. You will architect ingestion, transformation, and storage using Hadoop ecosystems such as HDFS and Hive, with Avro/Parquet serialization and Spark/Flink-based ETL pipelines.

You will mentor a team of data engineers, shape data models and lake architectures, and partner with product, analytics, and risk teams to enable

Qualifications

• 8-10 years of data engineering experience.
• 2-3 years in a technical leadership role.
• Strong Hadoop ecosystem expertise (HDFS, Hive, MapReduce, HBase, Pig).
• Experience with Avro, Parquet, and serialization formats.
• Design and maintenance of ETL/ELT pipelines (Spark, Flink, Airflow, NiFi).
• Strong data modeling, warehousing, and data lake architectures.
• Hands-on SQL with relational and NoSQL stores.
• AWS cloud data platform experience.
• Deep data security, compliance, and governance understanding.
• Excellent problem-solving and leadership abilities.

Responsibilities

• Lead end-to-end design, implementation, and optimization of data pipelines for onboarding, transactions, and decisioning workflows.
• Architect data ingestion, transformation, and storage frameworks using Hadoop, Avro, and distributed processing.
• Collaborate with product, analytics, and tech teams to translate requirements into scalable data solutions.
• Provide technical leadership and mentorship to a data engineering team.
• Design robust data frameworks to support next-gen product launches.
• Develop best practices for data governance, security, and compliance.
• Optimize existing pipelines for efficiency and scalability.
• Partner with analytics and risk teams to enable predictive insights and decision making.
• Evaluate and adopt new data technologies to future-proof the platform.

Skills

Data engineering

Leadership

Hadoop

ETL/ELT

Spark

Flink

Airflow

NiFi

SQL

NoSQL

AWS

Data security

Data governance

Tools

Spark

Flink

Airflow

NiFi

Job description

ROBOTIC PROCESS AUTOMATION LLC seeks an experienced Data Engineering Lead to drive end-to-end data pipelines that power onboarding, transactions, and decisioning workflows. You will architect ingestion, transformation, and storage using Hadoop ecosystems such as HDFS and Hive, with Avro/Parquet serialization and Spark/Flink-based ETL pipelines.

You will mentor a team of data engineers, shape data models and lake architectures, and partner with product, analytics, and risk teams to enable