Remote | Data Engineer — $140,000–$180,000/year

24-MAG

United States

Remote

USD 140,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

24-MAG LLC is seeking an experienced Data Engineer to design and operate scalable data architectures and AI-ready pipelines in a fully remote, full-time role. You will build large-scale distributed pipelines with Python and SQL, optimize Spark workloads in AWS, and collaborate with AI researchers and data scientists to enable analytics, experimentation, and model development.

The role emphasizes reliability, monitoring, data quality, and performance tuning across cloud-native infrastructures,

Qualifications

  • Strong professional experience in data engineering or distributed data systems.
  • Hands-on experience with Python and SQL.
  • Experience with large-scale data processing and distributed frameworks.
  • Familiarity with cloud-based data services (AWS).
  • Experience with data quality, monitoring, and automation workflows.

Responsibilities

  • Design, build, and maintain large-scale pipelines for structured and unstructured data.
  • Develop distributed processing workflows using Spark or comparable frameworks.
  • Optimise transformations, partitioning, and workloads; resolve performance bottlenecks.
  • Support analytics, experimentation, and model-development requirements.
  • Design scalable AWS-based data architectures; ingestion, transformation, storage, and distribution.

Skills

Python
SQL
Apache Spark
AWS
Distributed data processing
Data architecture

Tools

Apache Spark
AWS

Job description

We are sharing a full-time opportunity for an experienced Data Engineer with strong expertise in Python, SQL, Apache Spark, AWS, distributed data processing, and scalable data architecture to build and operate infrastructure supporting AI-driven products and research initiatives. The role will focus on designing and scaling distributed data pipelines, managing large datasets across cloud environments, and building reliable systems for analytics, experimentation, and model development.

Key Responsibilities
Data Pipelines & Distributed Processing
  • Design, build, and maintain large-scale pipelines for structured and unstructured data
  • Develop distributed processing workflows using Apache Spark or comparable frameworks
  • Optimise transformations, partitioning strategies, and computational workloads
  • Identify and resolve performance bottlenecks across high-volume data systems
  • Support downstream analytics, experimentation, and model-development requirements
Cloud Architecture & Data Engineering
  • Design scalable AWS-based data architectures across SQL and NoSQL systems
  • Build reliable ingestion, transformation, storage, and distribution workflows
  • Write efficient Python and SQL for production data processing
  • Evaluate storage and database technologies against workload requirements
  • Improve scalability, maintainability, accessibility, and operational efficiency
Data Quality, Reliability & AI
  • Support system reliability
  • Implement monitoring, validation, and automation across data workflows
  • Identify failures, anomalies, and data-quality issues
  • Maintain integrity and reliability throughout pipelines and storage layers
  • Collaborate with AI researchers, data scientists, and engineering teams
  • Support data infrastructure for AI/ML training, evaluation, and experimentation
Ideal Profile
  • Strong professional experience in data engineering or distributed data systems
  • Advanced proficiency in Python and SQL
  • Hands‑on experience with Apache Spark or comparable distributed-processing frameworks
  • Strong experience with AWS data services and cloud‑native architecture
  • Experience with SQL and NoSQL databases
  • Demonstrated experience processing large‑scale datasets
  • Strong understanding of partitioning, performance optimisation, and scalable architecture
  • Familiarity with orchestration, automation, monitoring, and data‑quality workflows
  • Exposure to AI/ML or research environments is advantageous
  • Familiarity with LLM training, evaluation, or experimentation datasets is beneficial
  • Experience with data‑visualisation tools such as Matplotlib, Seaborn, or Plotly is a plus
Engagement Details
  • Full‑time engagement
  • Fully remote
  • Base compensation: $140,000–$180,000/year
  • Work will involve Python, SQL, Apache Spark, AWS, distributed data processing, and scalable data architecture
  • Responsibilities will span data ingestion, transformation, storage, monitoring, and operational reliability
  • The role may support AI/ML experimentation, model‑development workflows, and LLM‑related data infrastructure
  • Data volumes, infrastructure requirements, and technical priorities may evolve as products and research initiatives scale
About the Platform

This opportunity is available through 24‑MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project‑based workstreams. By submitting this application, you acknowledge that your information may be processed by 24‑MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote | Big Data Engineer — $30–$80/hour
Remote | Big Data Engineer — $30–$80/hour

24-Mag Llc • New York (NY)

Remote
USD 41,000 - 110,000
Data Engineer, Cloud Pipelines & AI Infrastructure
Data Engineer, Cloud Pipelines & AI Infrastructure

24-MAG • United States

Remote
USD 140,000 - 180,000
Remote | Strategic Project Lead, Coding/SWE — $300,000–$500,000/year
Remote | Strategic Project Lead, Coding/SWE — $300,000–$500,000/year

24-Mag Llc • New York (NY)

Remote
USD 300,000 - 500,000
AI/ML Data Engineer - Scalable Data Solutions
AI/ML Data Engineer - Scalable Data Solutions

Placements24 • Riverton (UT)

Remote
USD 90,000 - 150,000
Sr. Software Engineer, Data Analytics Engineering
Sr. Software Engineer, Data Analytics Engineering

Pinterest, Inc. • United States

Hybrid
USD 183,000 - 320,000
Equity
Remote-eligible
Remote | Strategic Project Lead, Coding/SWE — $300,000–$500,000/year
Remote | Strategic Project Lead, Coding/SWE — $300,000–$500,000/year

24-MAG • United States

Remote
USD 300,000 - 500,000
Data Architect (Remote)
Data Architect (Remote)

Innowhyte Inc • Irvine (CA)

Remote
USD 120,000 - 160,000
Flexible work from home
Direct interaction with CXO team
Career growth opportunities
Sr. Data Engineer (Remote)
Sr. Data Engineer (Remote)

modemobile • United States

Remote
USD 160,000 - 260,000
Company equity
Performance bonus
Open PTO
+3
Senior Data Engineer - Fully Remote
Senior Data Engineer - Fully Remote

Sowelo Consulting sp. z o.o. sp. k. • United States

Remote
USD 120,000 - 180,000
100% remote
Flexible hours
B2B contract
+1
Remote Data Engineer: Build Scalable Cloud Pipelines
Remote Data Engineer: Build Scalable Cloud Pipelines

SpaceCoast AV Consultants • Florida

Remote
USD 110,000 - 140,000
Health, dental, and vision insurance
401(k) with company matching
Generous paid time off and parental leave
+1