Senior Data Engineer: Lakehouse & Streaming

Robotics Technologies LLC; Atlanta (GA); Vor Ort; USD 120.000 - 180.000; Vollzeit

Zusammenfassung

Robotics Technologies LLC is seeking a senior data engineer to build and operate production-grade batch and streaming data pipelines. You will develop ingestion, transformation, and enrichment pipelines using Flink, Kafka, SQL, and Python/Java/Scala.

You will implement lakehouse layers with Iceberg and Trino, build dimensional models, and ensure data quality with monitoring and data contracts. Work with global teams on CI/CD and IaC, with on-call duties as needed.

Qualifikationen

• 6+ years in data engineering or data platform development.
• Experience building end-to-end data pipelines in production.
• Expert SQL with performance tuning.
• Proficient in Python/Java/Scala.
• Hands-on with Iceberg, Delta Lake, or Hudi.
• Experience with Trino/Presto or Spark SQL.
• Snowflake experience including data sharing and external tables.
• Familiarity with streaming tech (Flink, Kafka, Debezium).
• Strong dimensional modeling, data quality, and contracts.
• Experience with Kubernetes, CI/CD, and IaC.

Aufgaben

• Build and operate production-grade batch and streaming pipelines across SQL Server, cloud apps, device/event streams, and CRM sources.
• Develop ingestion, transformation, and enrichment pipelines using Flink, Kafka, SQL, Python/Java/Scala.
• Implement lakehouse layers with Iceberg and Trino; create analytical models for entities.
• Ensure data quality, monitoring, lineage, and alerting; resolve data issues in production.
• Collaborate with Product, QA, DevOps, Analytics, and distributed teams.
• Drive CI/CD, IaC, tests, and deployment automation; participate in on-call rota.

Kenntnisse

Data engineering

SQL optimization

Python/Java/Scala

Production pipelines

Root Cause Analysis

Kubernetes

CI/CD

Data modeling

Data quality

Streaming data

Tools

Apache Iceberg

Delta Lake

Apache Hudi

Trino

Presto

Spark SQL

Snowflake

Apache Flink

Kafka

Debezium

Airflow

Dagster

dbt

Power BI

Looker

Superset

Jobbeschreibung

Robotics Technologies LLC is seeking a senior data engineer to build and operate production-grade batch and streaming data pipelines. You will develop ingestion, transformation, and enrichment pipelines using Flink, Kafka, SQL, and Python/Java/Scala.

You will implement lakehouse layers with Iceberg and Trino, build dimensional models, and ensure data quality with monitoring and data contracts. Work with global teams on CI/CD and IaC, with on-call duties as needed.