Lead Data Engineer - AWS, Databricks & Automation (Remote)

MACHINE LEARNING TECHNOLOGIES LLC

Chicago (IL)

Remote

USD 84,000 - 120,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Machine Learning Technologies LLC in Chicago, IL seeks a Data Engineer to design and optimize data pipelines, storage, and transformations for analytics and revenue growth. Work spans production AWS infrastructure and data products.

Ideal candidates have 5+ years in data engineering, strong SQL/PySpark, AWS experience with Redshift/S3/Athena, Terraform, and Databricks. Excellent communication and leadership skills are required for cross‑functional collaboration.

Qualifications

  • Bachelor's degree in a quantitative field or equivalent experience.
  • 5+ years of data engineering or ETL development experience.
  • Experience performing root cause analysis on data processes.
  • Strong analytics with structured, semi-structured, and unstructured data.
  • Experience with BigQuery, SQL Server, and AWS data services.

Responsibilities

  • Support large scale data pipelines in a distributed and scalable environment.
  • Enable and optimize production AWS environment for data infrastructure and frameworks.
  • Create Terraform modules to automate deployments.
  • Knowledge of Databricks and Datalake technologies.
  • Partner with development teams to provide technical solutions enabling business capabilities.
  • Lead design and development of batch and streaming data applications using AWS technologies.
  • Provide technical direction and resolve team queries/issues.
  • AWS Certification.

Skills

Python
Bash scripting
PySpark
AWS services
Terraform
Databricks

Education

Bachelor's degree in quantitative field
Master's degree

Tools

BigQuery
SQL Server
SSMS
Oracle SQL Developer
Databricks

Job description

Machine Learning Technologies LLC in Chicago, IL seeks a Data Engineer to design and optimize data pipelines, storage, and transformations for analytics and revenue growth. Work spans production AWS infrastructure and data products.

Ideal candidates have 5+ years in data engineering, strong SQL/PySpark, AWS experience with Redshift/S3/Athena, Terraform, and Databricks. Excellent communication and leadership skills are required for cross‑functional collaboration.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer: Databricks Lakehouse Architect (Remote)
Lead Data Engineer: Databricks Lakehouse Architect (Remote)

Celebal Technologies • United States

Remote
USD 140,000 - 190,000
Remote Databricks Data Engineer - AI Lakehouse Architect
Remote Databricks Data Engineer - AI Lakehouse Architect

Livefront • United States

Remote
USD 150,000 - 190,000
Remote Databricks Data Engineer — AI & Lakehouse Architect
Remote Databricks Data Engineer — AI & Lakehouse Architect

Livefront, Inc. • Peru (IL)

Remote
USD 90,000 - 140,000
Lead Data Engineer
Lead Data Engineer

KDR Talent Solutions USA • United States

Remote
USD 198,000 - 242,000
Competitive compensation
Full benefits package
Flexible remote or hybrid options
+1
Lead Data Engineer — AI/ML Pipelines (Remote, US)
Lead Data Engineer — AI/ML Pipelines (Remote, US)

3M • United States

Remote
USD 188,000 - 230,000
Medical, Dental & Vision
Health Savings Accounts
Disability Benefits
+3
Remote Databricks Data Engineer — Pipelines & Data Lakes
Remote Databricks Data Engineer — Pipelines & Data Lakes

Eliassen Group • Olympia (WA)

Remote
USD 150,000 - 154,000
Medical insurance
Dental insurance
Vision insurance
+4
Lead Data Engineer - Remote 75/hr
Lead Data Engineer - Remote 75/hr

ContractStaffingRecruiters.com • Arapahoe (NC)

Remote
USD 120,000 - 170,000
Lead Data Engineer, Databricks & Multi-Cloud
Lead Data Engineer, Databricks & Multi-Cloud

U.S. Bank • Chicago (IL)

On-site
USD 133,000 - 157,000
Healthcare
Life insurance
Disability
+6
Lead Data Engineer
Lead Data Engineer

TechDigital Group • Torrance (CA)

On-site
USD 100,000 - 120,000
Senior Data Engineer - Databricks/PySpark/SQL | Remote
Senior Data Engineer - Databricks/PySpark/SQL | Remote

HumanIT Digital Consulting • United States

Remote
USD 28,000 - 37,000
15th month salary
Health insurance for family
Birthday off
+3