Remote Senior Data Engineer: Scalable ETL & Streaming

Talentify; Oregon; Hybrid; USD 120,000 - 150,000; Full time

Job summary

Diverse Lynx LLC is seeking a Senior Data Engineer in Sunnyvale, CA (Remote) to design, build, and optimize data pipelines for analytics, AI/ML, and reporting. You will implement data quality controls, governance, and reliable data movement across platforms, leveraging Kafka, PySpark, Airflow, and Hive.

The role emphasizes scalable ETL/ELT development, ingestion automation, and collaboration with data teams to support advanced analytics initiatives. Remote-friendly for the US market.

Qualifications

• Experience with big data processing landscapes and building reliable data pipelines.
• Proficiency with Python-based data tooling and Spark-based processing.
• Strong knowledge of data quality, governance, and monitoring practices.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines.
• Automate data ingestion from multiple data sources.
• Develop and optimize PySpark-based data processing frameworks.
• Build and manage Airflow workflows and orchestration pipelines.
• Work with Hive and Kafka-based batch and streaming data solutions.

Skills

Kafka

Python

PySpark

Airflow

Hive

ETL testing

Tools

Google Cloud Platform

Job description

Diverse Lynx LLC is seeking a Senior Data Engineer in Sunnyvale, CA (Remote) to design, build, and optimize data pipelines for analytics, AI/ML, and reporting. You will implement data quality controls, governance, and reliable data movement across platforms, leveraging Kafka, PySpark, Airflow, and Hive.

The role emphasizes scalable ETL/ELT development, ingestion automation, and collaboration with data teams to support advanced analytics initiatives. Remote-friendly for the US market.