Senior Data Engineer | Python & PySpark Cloud Pipelines

MACHINE LEARNING TECHNOLOGIES LLC

Atlanta (GA)

Hybrid

USD 93,000 - 133,000

Full time

9 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking a Senior Data Engineer in Atlanta (Hybrid) to design, develop and optimize scalable data pipelines using Python, PySpark, and SQL. You will build ETL/ELT workflows, work with cloud platforms, and ensure data quality across large datasets.

The role requires 7+ years in data engineering, experience with Spark, Airflow, and modern data warehousing concepts. Collaboration with analysts and architects is essential for delivering robust data solutions.

Qualifications

  • 7+ years of experience in data engineering.
  • Strong hands-on experience with Python and PySpark.
  • Expertise in SQL and database development.
  • Experience with Apache Spark and distributed data processing.
  • Knowledge of ETL/ELT development.
  • Experience with AWS, Azure, or GCP.
  • Knowledge of data warehousing concepts and data modeling.
  • Experience with Apache Airflow or similar orchestration tools.
  • Familiarity with Git and CI/CD pipelines.
  • Strong analytical and problem-solving skills.

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Python and PySpark.
  • Build and optimize ETL/ELT workflows for structured and unstructured data.
  • Develop SQL queries and optimize database performance.
  • Process large datasets using Apache Spark.
  • Integrate data from multiple sources into centralized data platforms.
  • Develop data transformation and validation processes.
  • Implement data quality checks and monitoring solutions.
  • Work with cloud-based data storage and processing services.
  • Optimize data processing workflows for performance and scalability.
  • Collaborate with data analysts, architects, and application development teams.
  • Troubleshoot data pipeline failures and resolve production issues.
  • Maintain technical documentation and follow data engineering best practices.

Skills

Python
PySpark
SQL
Apache Spark
Airflow
CI/CD
Git
Data warehousing
ETL/ELT
Cloud platforms

Tools

AWS
Azure
GCP

Job description

MACHINE LEARNING TECHNOLOGIES LLC is seeking a Senior Data Engineer in Atlanta (Hybrid) to design, develop and optimize scalable data pipelines using Python, PySpark, and SQL. You will build ETL/ELT workflows, work with cloud platforms, and ensure data quality across large datasets.

The role requires 7+ years in data engineering, experience with Spark, Airflow, and modern data warehousing concepts. Collaboration with analysts and architects is essential for delivering robust data solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer - Python / PySpark | Hybrid (Atlanta)
Senior Data Engineer - Python / PySpark | Hybrid (Atlanta)

Quantum Technologies LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000
Senior Data Engineer
Senior Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000
Senior Data Engineer - Scalable Pipelines & Spark Expert
Senior Data Engineer - Scalable Pipelines & Spark Expert

Ethereum Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 180,000
Senior Data Engineer: Scalable Pipelines & Spark
Senior Data Engineer: Scalable Pipelines & Spark

Robotics Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 170,000
Senior PySpark Data Engineer - Cloud & ETL Specialist
Senior PySpark Data Engineer - Cloud & ETL Specialist

Highbrow Technology Inc • Dallas (TX)

On-site
USD 140,000 - 200,000
Senior Data Engineer - Scalable Pipelines & Cloud Analytics
Senior Data Engineer - Scalable Pipelines & Cloud Analytics

Initvalue • Fremont (CA)

Hybrid
USD 130,000 - 190,000
Senior Data Integration Engineer, AI-Driven Cloud Pipelines
Senior Data Integration Engineer, AI-Driven Cloud Pipelines

EPAM Systems Inc • United States

Remote
USD 120,000 - 180,000
Senior PySpark Data Engineer: Build Robust ETL Pipelines
Senior PySpark Data Engineer: Build Robust ETL Pipelines

MTK Technologies • Irving (TX)

On-site
USD 90,000 - 120,000
Senior Data Engineer: Python, SQL, Databricks & ML Pipelines
Senior Data Engineer: Python, SQL, Databricks & ML Pipelines

Insight Global (United States) • Westbrook (ME)

On-site
USD 140,000 - 190,000
Senior Data Engineer: Cloud Data Pipelines & Warehousing
Senior Data Engineer: Cloud Data Pipelines & Warehousing

Ethereum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 180,000