Data Pipeline Engineer — Python, Spark & AWS

TechDigital Group

Los Angeles (CA)

On-site

USD 110,000 - 150,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

TechDigital Group in Los Angeles, CA is seeking a hands-on Data Engineer to design and build scalable data pipelines using Spark, PySpark, and Kafka, with AWS services like DynamoDB, Lambda, Redshift, and Elasticsearch.

You will collaborate across teams, ensuring robust data integration for streaming and batch workloads, and contribute to CI/CD using GitHub, Jenkins, and Terraform in a fast-paced environment.

Qualifications

  • Hands-on experience with AWS services including DynamoDB, Lambda, Redshift, and Elasticsearch.
  • Proficient in Python, PySpark, Apache Spark, and Kafka for data pipelines.
  • Design and develop data pipelines for ingestion and transformation using Spark.
  • Experience with GitHub, Jenkins, and Terraform.
  • Strong knowledge of data integration (streaming, batch, error, replay) and data analysis.

Responsibilities

  • Design and implement data pipelines for ingestion and transformation using Spark.
  • Maintain and optimize data workflows for streaming and batch processing.
  • Collaborate with teams using GitHub, Jenkins, and Terraform for CI/CD and deployment.

Skills

AWS
DynamoDB
Lambda
Redshift
Elasticsearch
Python
PySpark
Apache Spark
Kafka
GitHub
Jenkins
Terraform
Data integration
Data analysis

Tools

Teradata
SQL

Job description

TechDigital Group in Los Angeles, CA is seeking a hands-on Data Engineer to design and build scalable data pipelines using Spark, PySpark, and Kafka, with AWS services like DynamoDB, Lambda, Redshift, and Elasticsearch.

You will collaborate across teams, ensuring robust data integration for streaming and batch workloads, and contribute to CI/CD using GitHub, Jenkins, and Terraform in a fast-paced environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Cloud Data Engineer: AWS, Databricks & PySpark Pipelines
Cloud Data Engineer: AWS, Databricks & PySpark Pipelines

Advanced Tech Placement • Atlanta (GA)

On-site
USD 120,000 - 150,000
Data Engineer
Data Engineer

TMV Global Inc • Virginia (MN)

On-site
USD 95,000 - 120,000
Data Engineer
Data Engineer

Advanced Tech Placement • Atlanta (GA)

On-site
USD 120,000 - 150,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • United States

On-site
USD 120,000 - 180,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • Dallas (TX)

On-site
USD 100,000 - 160,000
Data Engineer
Data Engineer

Smart IT Frame LLC • Reston (VA)

On-site
USD 110,000 - 180,000
Real-Time Spark Engineer – Streaming & Pipelines
Real-Time Spark Engineer – Streaming & Pipelines

TechDigital Group • Jersey City (NJ)

On-site
USD 120,000 - 190,000
Data Engineer
Data Engineer

Unisys • McLean (VA)

On-site
USD 130,000 - 170,000
Data Pipeline Engineer (ETL/Databricks, Python)
Data Pipeline Engineer (ETL/Databricks, Python)

Attain Talent • United States

Remote
USD 120,000 - 180,000
Health benefits
PTO & 401(k) match
Tuition reimbursement
Python Developer: Spark, Airflow & AWS Data Pipelines
Python Developer: Spark, Airflow & AWS Data Pipelines

Ethereum Technologies • Wilmington (DE)

On-site
USD 120,000 - 170,000