Data Engineering Tech Lead – Hadoop

Cloud Analytics Technologies, LLC; Cleveland (OH); On-site; USD 120,000 - 150,000; Full time

Job summary

A data analytics firm located in Cleveland, Ohio, is looking for a Data Engineering Lead. This role involves leading and optimizing data pipelines, architecting data frameworks, and ensuring technical excellence within the team. The perfect candidate will have over 8 years in data engineering, experience with Hadoop, and proficiency in tools like Spark and AWS. Strong leadership and communication skills are essential. Join a collaborative and inclusive work environment focused on innovation and technical growth.

Qualifications

• 8–10 years of experience in data engineering, with at least 2–3 years in a technical leadership role.
• Strong expertise in the Hadoop ecosystem (HDFS, Hive, MapReduce, HBase, Pig, etc.).
• Proficiency in Python and Scala for large-scale data processing.

Responsibilities

• Lead and oversee the design, implementation, and optimization of data pipelines.
• Architect and implement data ingestion, transformation, and storage frameworks.
• Provide technical leadership and mentorship to a team of data engineers.

Skills

Data engineering

Technical leadership

Hadoop ecosystem

ETL / ELT pipelines

Data modeling

SQL

Tools

Hadoop

Spark

Flink

Airflow

NiFi

AWS

Avro

Parquet

Job description

Responsibilities

• Lead and oversee the end-to-end design, implementation, and optimization of data pipelines supporting key customer onboarding, transaction, and decisioning workflows.
• Architect and implement data ingestion, transformation, and storage frameworks leveraging Hadoop,Avro, and distributed data processing technologies.
• Partner with product, analytics, and technology teams to translate business requirements into scalable data engineering solutions that enhance real‑time data accessibility and reliability.
• Provide technical leadership and mentorship to a team of data engineers, ensuring adherence to coding, performance, and data quality standards.
• Design and implement robust data frameworks to support next‑generation customer and business product launches.
• Develop best practices for data governance,security, and compliance aligned with enterprise and regulatory requirements.
• Drive optimization of existing data pipelines and workflows for improved efficiency, scalability, and maintainability.
• Collaborate closely with analytics and risk modeling teams to ensure data readiness for predictive insights and strategic decision‑making.
• Evaluate and integrate emerging data technologies to future‑proof the data platform and enhance performance.

Must‑Have Skills

• 8–10 years of experience in data engineering, with at least 2–3 years in a technical leadership role.
• Strong expertise in the Hadoop ecosystem (HDFS, Hive, MapReduce, HBase, Pig, etc.).
• Experience working with Avro,Parquet, or other serialization formats.
• Proven ability to design and maintain ETL / ELT pipelines using tools such as Spark,Flink,Airflow, or NiFi.
• Proficiency in Python,Scala for large‑scale data processing.
• Strong understanding of data modeling, data warehousing, and data lake architectures.
• Hands‑on experience with SQL and both relational and NoSQL data stores.
• Cloud data platform experience with AWS.
• Deep understanding of data security, compliance, and governance frameworks.
• Excellent problem‑solving, communication, and leadership skills.

CLOUD ANALYTICS TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract.