Senior Data Engineer

Diverse Lynx, LLC

Oregon

Hybrid

USD 120,000 - 150,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Diverse Lynx LLC is seeking a Senior Data Engineer in Sunnyvale, CA (Remote) to design, build, and optimize data pipelines for analytics, AI/ML, and reporting. You will implement data quality controls, governance, and reliable data movement across platforms, leveraging Kafka, PySpark, Airflow, and Hive.

The role emphasizes scalable ETL/ELT development, ingestion automation, and collaboration with data teams to support advanced analytics initiatives. Remote-friendly for the US market.

Qualifications

  • Experience with big data processing landscapes and building reliable data pipelines.
  • Proficiency with Python-based data tooling and Spark-based processing.
  • Strong knowledge of data quality, governance, and monitoring practices.

Responsibilities

  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources.
  • Develop and optimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.

Skills

Kafka
Python
PySpark
Airflow
Hive
ETL testing

Tools

Google Cloud Platform

Job description

About this job

Job Title: Senior Data Engineer

Location: Sunnyvale, CA (Remote)

Duration: Fulltime

Skills: Kafka | Python | ETL Testing | Hive

Salary: 120K-150K/Year

Required Skills & Qualifications
  • Google Cloud Platform (Eligible to workP)
  • PySpark
  • Python
  • Airflow
  • Hive
  • Kafka
  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources.
  • Ensure reliable, high-performance, and scalable data movement.
  • Develop and Eligible to workimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.
  • Implement data quality, monitoring, governance, and operational controls.Support production incidents, performance tuning, and continuous improvement initiatives
  • The Data Engineer will be responsible for designing, building, governing, and Eligible to workimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.
Preferred Experience
  • Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
  • Strong experience with large-scale Eligible to workP Data Engineering solutions.
  • Hands-on experience with Airflow DAG development and support
  • Experience building and Eligible to workimizing Spark/PySpark data pipelines.
  • Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
  • Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 110,000 - 150,000
Discretionary Annual Incentive
Comprehensive Medical Coverage: Health
Dental & Vision
+6
Engineer Software Design
Engineer Software Design

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Leave
+3
DATA ENGINEER
DATA ENGINEER

Diverse Lynx, LLC • Las Vegas (NV)

On-site
USD 76,000 - 90,000
Engineer Software Design
Engineer Software Design

Tata Consultancy Services Limited • Herndon (VA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive.
Comprehensive Medical Coverage:Medical
Dental & Vision
+7
Senior Data Engineer
Senior Data Engineer

Compunnel Inc. • Sunnyvale (CA)

On-site
USD 100,000 - 140,000
Data Engineer
Data Engineer

Diverse Lynx, LLC • Seattle (WA)

On-site
USD 110,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 110,000 - 150,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Support: Parental Leaves
+4
Senior Data Engineer
Senior Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000
Remote Senior Data Engineer: Scalable ETL & Streaming
Remote Senior Data Engineer: Scalable ETL & Streaming

Diverse Lynx, LLC • Oregon

Hybrid
USD 120,000 - 150,000
Senior Data Engineer (Remote)
Senior Data Engineer (Remote)

Dahl Consulting • Brooklyn Park (MN)

Hybrid
USD 92,000 - 156,000