Remote Data Engineer: Scalable Pipelines & Spark

BairesDev; United States; Remote; USD 110,000 - 170,000; Full time

Benefits offered by this job

Remote work

USD payments

Home setup provided

Flexible hours

Paid parental leave

Mentorship & growth

Job summary

BairesDev is seeking a Software Engineer (Data) to design and maintain large-scale data pipelines powering data-driven insights across cloud environments. You will collaborate with engineering teams to optimize Spark-based workloads, implement Delta Lake or Iceberg storage, and orchestrate complex workflows with Airflow, while ensuring data quality and availability at scale.

This role requires 4+ years of data engineering experience and strong English skills.

Qualifications

• 4+ years of software engineering experience focusing on data engineering.
• Proven expertise building large-scale data pipelines with Python and SQL.
• Hands-on experience with Apache Spark and PySpark; knowledge of Spark internals.
• Experience with data lakehouse technologies like Delta Lake or Iceberg.
• Experience orchestrating data workflows with Airflow.
• Strong English communication skills.

Responsibilities

• Design and develop scalable data pipelines using Python and PySpark for large-scale processing.
• Optimize Spark jobs to improve performance and cost-efficiency.
• Implement robust data storage with Delta Lake or Iceberg for reliable data lakes.
• Orchestrate data workflows and dependencies using Airflow or similar tools.
• Design and execute SQL queries for data transformation and analysis across cloud environments.
• Collaborate with engineering teams to ensure data quality, integrity, and availability at scale.

Skills

Python

SQL

Apache Spark

PySpark

Spark internals

Delta Lake

Apache Iceberg

Airflow

English proficiency

Tools

Delta Lake

Apache Iceberg

Airflow

Spark internals

Job description

BairesDev is seeking a Software Engineer (Data) to design and maintain large-scale data pipelines powering data-driven insights across cloud environments. You will collaborate with engineering teams to optimize Spark-based workloads, implement Delta Lake or Iceberg storage, and orchestrate complex workflows with Airflow, while ensuring data quality and availability at scale.

This role requires 4+ years of data engineering experience and strong English skills.