Data Engineer: Scalable Pipelines, Spark & AI Tools

Matrix Global

United States

Remote

USD 110,000 - 170,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Matrix seeks an experienced Data Pipeline Engineer to develop, implement, and maintain production-grade data pipelines. You will build scalable data flows, support ML and analytics, and optimize large-scale processing environments.

T he role requires Python, PySpark, Spark, SQL, Airflow, and Hadoop ecosystem expertise. Collaborate with customers and internal teams to deliver reliable data solutions with a business focus.

Qualifications

  • Hands-on experience with Python and Pandas.
  • Experience with Spark and PySpark for distributed processing.
  • Strong SQL skills and large dataset analysis.
  • Practical Airflow and workflow orchestration.
  • Familiarity with Hadoop ecosystem (Hive, Impala, HDFS, Sqoop).
  • Experience with Git and production data pipelines.
  • Understanding of structured and semi-structured data formats.
  • Exposure to Kubernetes workloads and LLM-related tooling.

Responsibilities

  • Implement, maintain, and support production data pipelines.
  • Design use-case data flows and ML pipelines for analytics teams.
  • Build and optimize data processing with Spark, PySpark, Python, SQL, and Airflow.
  • Work with Hadoop ecosystem (Hive/Impala/HDFS/Sqoop).
  • Analyze data characteristics to inform pipeline design.
  • Deploy workloads in Kubernetes environments.
  • Collaborate with customers and cross-functional teams.
  • Apply business-oriented approaches to data workflows.
  • Leverage LLMs and AI agents to accelerate delivery.

Skills

Pandas
PySpark
SQL
Airflow
Hadoop
Kubernetes
Git
LLMs / AI agents

Tools

Spark
Hive
Impala
HDFS
Sqoop
Airflow

Job description

Matrix seeks an experienced Data Pipeline Engineer to develop, implement, and maintain production-grade data pipelines. You will build scalable data flows, support ML and analytics, and optimize large-scale processing environments.

T he role requires Python, PySpark, Spark, SQL, Airflow, and Hadoop ecosystem expertise. Collaborate with customers and internal teams to deliver reliable data solutions with a business focus.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer: Scalable Pipelines & AI Data Platforms
Data Engineer: Scalable Pipelines & AI Data Platforms

Resolve Tech Solutions • Irving (TX)

On-site
USD 110,000 - 150,000
Senior Data Engineer - Scalable Pipelines & Spark Expert
Senior Data Engineer - Scalable Pipelines & Spark Expert

Ethereum Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 180,000
Data Engineer: Build Scalable Pipelines & Insights
Data Engineer: Build Scalable Pipelines & Insights

Compunnel Inc. • Orlando (FL)

On-site
USD 85,000 - 110,000
Data Engineer - Scalable Pipelines & AI-Driven Analytics
Data Engineer - Scalable Pipelines & AI-Driven Analytics

Optomi • Austin (TX)

On-site
USD 100,000 - 130,000
Data Engineer: Build Scalable Pipelines for Analytics
Data Engineer: Build Scalable Pipelines for Analytics

Thecorporate • Tulsa (OK)

On-site
USD 110,000 - 150,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • United States

On-site
USD 120,000 - 180,000
Data Engineer: Build Scalable Pipelines & Reliable Data
Data Engineer: Build Scalable Pipelines & Reliable Data

Externo • United States

Remote
USD 100,000 - 140,000
Data engineer
Data engineer

Matrix Global • United States

Remote
USD 110,000 - 170,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • Dallas (TX)

On-site
USD 100,000 - 160,000
Senior Data Engineer - Scalable Pipelines & Cloud Analytics
Senior Data Engineer - Scalable Pipelines & Cloud Analytics

Initvalue • Fremont (CA)

Hybrid
USD 130,000 - 190,000