Hadoop Data Engineering Lead: Real-Time Pipelines

Ethereum Technologies; Cleveland (OH); Presencial; USD 120.000 - 160.000; Jornada completa

Descripción de la vacante

Ethereum Technologies LLC seeks a senior data engineer lead to own end-to-end design, implementation, and optimization of data pipelines supporting onboarding, transactions, and decisioning workflows.

Team leadership and mentorship of data engineers, hands-on work with Hadoop ecosystem, Spark, Airflow, and cloud platforms to deliver scalable, secure data platforms that meet regulatory requirements.

Formación

• 8-10 years of experience in data engineering with 2-3 years in a leadership role.
• Deep expertise in Hadoop ecosystem components (HDFS, Hive, MapReduce, HBase, Pig).
• Hands-on with data serialization formats (Avro, Parquet).
• Proven ability to design and maintain ETL/ELT pipelines (Spark, Flink, Airflow, NiFi).
• Strong data modeling, warehousing, and data lake architectures.
• Experience with SQL and both relational and NoSQL stores.
• Cloud data platforms experience with AWS.
• Knowledge of data security, compliance, and governance.

Responsabilidades

• Lead end-to-end design, implementation, and optimization of data pipelines for onboarding, transactions, and decisioning.
• Architect data ingestion, transformation, and storage frameworks using Hadoop, Avro, Parquet, and distributed processing.
• Collaborate with product, analytics, and tech teams to deliver scalable data solutions.
• Provide technical leadership and mentorship to a team of data engineers.
• Design robust data frameworks to support new product launches.
• Develop best practices for data governance, security, and regulatory compliance.
• Drive optimization for efficiency, scalability, and maintainability of pipelines.
• Work with analytics and risk teams to enable predictive insights and strategic decisions.
• Evaluate new data technologies to future-proof the platform.

Conocimientos

Data engineering

Technical leadership

Hadoop ecosystem

ETL pipelines

SQL

NoSQL

Spark

Flink

Airflow

NiFi

Parquet

Avro

Data modeling

Data warehousing

AWS

Data security

Regulatory compliance

Communication

Leadership

Herramientas

Spark

Flink

Airflow

NiFi

Hadoop

AWS

Descripción del empleo

Ethereum Technologies LLC seeks a senior data engineer lead to own end-to-end design, implementation, and optimization of data pipelines supporting onboarding, transactions, and decisioning workflows.

Team leadership and mentorship of data engineers, hands-on work with Hadoop ecosystem, Spark, Airflow, and cloud platforms to deliver scalable, secure data platforms that meet regulatory requirements.