Remote Lakehouse Data Engineer (Spark/NiFi)

TeamStation Artificial Intelligence, LLC.; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Fully remote (LATAM/US)

Career growth & mentorship

Open-source collaboration

Flexible work environment

Job summary

TeamStation AI is hiring a Data Engineer to design and implement a scalable lakehouse platform. You will build high-performance pipelines, manage schema evolution, and support analytics workloads with Spark, NiFi, and Airflow.

You will collaborate with analytics teams to optimize Trino queries and enable dashboards in Superset while working with Iceberg/Delta Lake table formats in a cloud data lake.

Qualifications

• 5+ years in data engineering with robust ETL pipelines.
• Strong experience with Spark (SQL/DataFrame), NiFi, Airflow or similar.
• Proficient in Python and/or Scala, and SQL with optimization knowledge.
• Familiarity with lakehouse table formats (Iceberg, Delta Lake, Hudi).
• Experience with cloud storage (ADLS Gen2, S3, GCS) and security controls.

Responsibilities

• Architect scalable Data Lakehouse pipelines and time-travel aware schemas.
• Design Bronze/Silver/Gold layers and manage Iceberg/Parquet metadata.
• Develop Bronze-to-Silver and Silver-to-Gold ETL using NiFi/Airflow/Spark.
• Collaborate with analytics to optimize Trino queries and dashboards.

Skills

Python

Scala

SQL

Apache Spark

Apache NiFi

Apache Airflow

Trino/Presto/Athena

Git

Tools

Apache Iceberg

Delta Lake

Hive external tables

Azure Data Lake Storage Gen2

S3 / GCS

Superset

Job description

TeamStation AI is hiring a Data Engineer to design and implement a scalable lakehouse platform. You will build high-performance pipelines, manage schema evolution, and support analytics workloads with Spark, NiFi, and Airflow.

You will collaborate with analytics teams to optimize Trino queries and enable dashboards in Superset while working with Iceberg/Delta Lake table formats in a cloud data lake.