Remote Senior Data Engineer — ETL, Spark & Kafka

MBO Partners; United States; A distancia; USD 90.000 - 130.000; A tiempo parcial

Ventajas ofrecidas por este puesto de trabajo

Eligibility for Paid Sick Leave (PSL)

Flexible working hours

Descripción de la vacante

A leading independent jobs platform is seeking an experienced Data Engineer to work remotely. You will design and implement data pipelines while optimizing data flows using technologies like Spark and Kafka. Candidates should have a minimum of 5 years in data engineering, preferably with experience in production environments. This role requires a strong understanding of data formats, streaming, and query optimization, with a focus on driving innovation in the civil market.

Formación

• 5+ years of experience designing, building, and maintaining data pipelines in production environments.
• 3+ years of experience with Spark (Databricks preferred) and distributed systems.
• Ability to obtain and maintain a Public Trust clearance.

Responsabilidades

• Work with large, complex datasets and build robust ETL/ELT pipelines.
• Optimize data flows using technologies like Databricks, Spark, and Kafka.

Conocimientos

Designing data pipelines

Building with Spark

Processing data with Kafka

Proficiency in Python

Implementing data quality best practices

Managing structured/unstructured data

Herramientas

Databricks

Spark

Kubernetes

AWS

Descripción del empleo

A leading independent jobs platform is seeking an experienced Data Engineer to work remotely. You will design and implement data pipelines while optimizing data flows using technologies like Spark and Kafka. Candidates should have a minimum of 5 years in data engineering, preferably with experience in production environments. This role requires a strong understanding of data formats, streaming, and query optimization, with a focus on driving innovation in the civil market.