Data Engineering Tech Lead – Hadoop

Robotic Technologies Automation; Cleveland (OH); On-site; USD 120,000 - 180,000; Full time

Job summary

ROBOTIC PROCESS AUTOMATION LLC seeks an experienced Data Engineering Lead to drive end-to-end data pipelines that power onboarding, transactions, and decisioning workflows. You will architect ingestion, transformation, and storage using Hadoop ecosystems such as HDFS and Hive, with Avro/Parquet serialization and Spark/Flink-based ETL pipelines.

You will mentor a team of data engineers, shape data models and lake architectures, and partner with product, analytics, and risk teams to enable

Qualifications

• 8-10 years of data engineering experience.
• 2-3 years in a technical leadership role.
• Strong Hadoop ecosystem expertise (HDFS, Hive, MapReduce, HBase, Pig).
• Experience with Avro, Parquet, and serialization formats.
• Design and maintenance of ETL/ELT pipelines (Spark, Flink, Airflow, NiFi).
• Strong data modeling, warehousing, and data lake architectures.
• Hands-on SQL with relational and NoSQL stores.
• AWS cloud data platform experience.
• Deep data security, compliance, and governance understanding.
• Excellent problem-solving and leadership abilities.

Responsibilities

• Lead end-to-end design, implementation, and optimization of data pipelines for onboarding, transactions, and decisioning workflows.
• Architect data ingestion, transformation, and storage frameworks using Hadoop, Avro, and distributed processing.
• Collaborate with product, analytics, and tech teams to translate requirements into scalable data solutions.
• Provide technical leadership and mentorship to a data engineering team.
• Design robust data frameworks to support next-gen product launches.
• Develop best practices for data governance, security, and compliance.
• Optimize existing pipelines for efficiency and scalability.
• Partner with analytics and risk teams to enable predictive insights and decision making.
• Evaluate and adopt new data technologies to future-proof the platform.

Skills

Data engineering

Leadership

Hadoop

ETL/ELT

Spark

Flink

Airflow

NiFi

SQL

NoSQL

AWS

Data security

Data governance

Tools

Spark

Flink

Airflow

NiFi

Job description

• Lead and oversee the end-to-end design, implementation, and optimization ofdata pipelinessupporting key customer onboarding, transaction, and decisioning workflows.
• Architect and implementdata ingestion, transformation, and storageframeworks leveragingHadoop,Avro, and distributed data processing technologies.
• Partner with product, analytics, and technology teams to translate business requirements into scalable data engineering solutions that enhance real-time data accessibility and reliability.
• Providetechnical leadership and mentorshipto a team of data engineers, ensuring adherence to coding, performance, and data quality standards.
• Design and implement robust data frameworks to support next-generation customer and business product launches.
• Develop best practices fordata governance,security, andcompliancealigned with enterprise and regulatory requirements.
• Drive optimization of existing data pipelines and workflows for improved efficiency, scalability, and maintainability.
• Collaborate closely with analytics and risk modeling teams to ensure data readiness for predictive insights and strategic decision-making.
• Evaluate and integrate emerging data technologies to future-proof the data platform and enhance performance.

Must-Have Skills

• 8-10 years of experience indata engineering, with at least2-3 years in a technical leadership role.
• Strong expertise in theHadoop ecosystem(HDFS, Hive, MapReduce, HBase, Pig, etc.).
• Experience working withAvro,Parquet, or other serialization formats.
• Proven ability to design and maintainETL / ELT pipelinesusing tools such asSpark,Flink,Airflow, orNiFi.
• Strong understanding ofdata modeling,data warehousing, anddata lakearchitectures.
• Hands-on experience withSQLand bothrelationalandNoSQLdata stores.
• Cloud data platform experience withAWS.
• Deep understanding ofdata security,compliance, andgovernance frameworks.
• Excellent problem-solving, communication, and leadership skills.

ROBOTIC PROCESS AUTOMATION LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. ROBOTIC PROCESS AUTOMATION LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will ROBOTIC PROCESS AUTOMATION LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract