Senior Data Engineer - Lakehouse & Streaming Platforms

Ethereum Technologies; Atlanta (GA); Presencial; USD 120.000 - 180.000; Tempo integral

Resumo da vaga

Ethereum Technologies in Atlanta, GA is seeking a senior data engineer to design, build, and operate production-grade data pipelines across SQL Server, cloud apps, and streaming sources.

You will develop ingestion, transformation, and enrichment workflows using Python/Java/Scala, and implement lakehouse layers with Iceberg and Trino. The role includes performance tuning, data quality, and collaboration with global engineering teams.

Qualificações

• 6+ years in data engineering, data warehousing, or data platform development.
• Strong production experience building and supporting end-to-end data pipelines.
• Expert SQL, including complex queries, query-plan analysis, and performance tuning.
• Strong development skills in Python, Java, or Scala.
• Hands-on experience with Iceberg, Delta Lake, or Hudi.
• Experience with Trino, Presto, or Spark SQL.
• Hands-on Snowflake experience, including data modeling, performance optimization, external/Iceberg tables, secure data sharing, or access management.
• Experience with streaming and CDC technologies such as Flink, Kafka, Spark Structured Streaming, or Debezium.
• Strong understanding of dimensional modeling, data quality, schema evolution, and data contracts.
• Experience with Kubernetes, containers, CI/CD, and infrastructure-as-code.
• Experience implementing monitoring, alerting, lineage, reconciliation, and operational support for production data platforms.
• Ability to independently troubleshoot complex production issues and drive them to resolution.

Responsabilidades

• Build and operate production-grade data pipelines across SQL Server, cloud apps, and streams.
• Develop ingestion, transformation, and enrichment pipelines using Flink, Kafka, SQL, Python/Java/Scala.
• Implement lakehouse layers with Iceberg and Trino.
• Create dimensional and analytical models for buildings, tenants, credentials, devices, and people.
• Ensure CDC, late-arriving data, retries, backfills, schema evolution, and data reconciliation.
• Integrate lakehouse with Snowflake, including Iceberg-backed/external tables and secure sharing.
• Develop data-quality checks, monitoring, lineage, and dashboards.
• Troubleshoot production data issues, pipeline failures, and performance bottlenecks.
• Optimize queries, storage layouts, partitioning, clustering, compute, and cost.
• Implement RBAC, row/column security, masking, and data protection.

Conhecimentos

Hands-on experience

Advanced SQL

Python/Java/Scala

Iceberg/Delta/Hudi

Trino/Presto/Spark SQL

Snowflake

Flink/Kafka/CDC

Kubernetes/CI/CD/IaC

Data modeling

Monitoring/Lineage

Ferramentas

Apache Iceberg

Delta Lake

Apache Hudi

Trino/Presto

Spark SQL

Descrição da vaga

Ethereum Technologies in Atlanta, GA is seeking a senior data engineer to design, build, and operate production-grade data pipelines across SQL Server, cloud apps, and streaming sources.

You will develop ingestion, transformation, and enrichment workflows using Python/Java/Scala, and implement lakehouse layers with Iceberg and Trino. The role includes performance tuning, data quality, and collaboration with global engineering teams.