Data Engineer (Salesforce Focus)

In All Media

United States

Remote

USD 130,000 - 180,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

In All Media is seeking a Data Engineer focused on Salesforce data flows in a fully remote role. You will design and migrate data pipelines, modernize lakehouse architectures with Iceberg on AWS, and optimize Snowflake models in a multi-region AWS environment.

You will work on end-to-end data migration, streaming pipelines, and ensuring data quality with robust testing across large-scale datasets. The role requires aligning with US Central Time for collaboration and daily standups.

Qualifications

  • 6+ years of hands-on data engineering experience designing, deploying, and maintaining distributed, production-grade data pipelines.
  • Python and advanced SQL proficiency for complex data transformations and modeling.
  • Production experience with Apache Spark / PySpark for batch and streaming.
  • Experience authoring and maintaining Apache Airflow DAGs and Kafka pipelines.
  • Snowflake data warehousing, Iceberg on S3, and Parquet formats knowledge.
  • Ability to troubleshoot distributed systems and ensure data quality across migrations.

Responsibilities

  • Dataset & Pipeline Migration: Migrate mission-critical datasets and DAGs to modern cloud infrastructure.
  • Lakehouse Modernization: Convert Hive to Iceberg on AWS S3 for better partitioning and performance.
  • Distributed Streaming: Configure Kafka connections across AWS regions for continuous data streaming.
  • Snowflake Modeling & Query: Model and tune pipelines in Snowflake using advanced SQL.
  • Airflow DAGs: Develop, schedule, and debug Airflow workflows to prevent bottlenecks.
  • Data Quality: Define validation strategies to guarantee data integrity across migrations.
  • Root-Cause Remediation: Diagnose and resolve performance issues in Spark/PySpark and storage layers.

Skills

Data engineering
Python
Advanced SQL
Apache Spark
Airflow DAGs
Kafka
Snowflake
AWS / S3
Iceberg
Data quality

Tools

Apache Iceberg
Hive
Parquet
Snowflake
Airflow
Kafka
PySpark
Python

Job description

Position: Data Engineer (Salesforce Focus) Location: Remote (Aligned with US Central Time) Contract Type: Full-time vendor (Contract directly with In All Media) Time Zone Alignment: US Central Time (CT)

About In All Media

In All Media is a global technology and design firm focused on building impactful digital solutions through remote, distributed teams globally. We partner with international clients across industries, providing long-term technical expertise, product innovation, and team augmentation.

Contract Details: This is a 100% remote contractor position. Your contract and engagement will be directly managed with In All Media.

Project Overview

In this role, you will drive critical data platform migration, modernization, and integration initiatives across large-scale enterprise environments with a core focus on Salesforce data flows. You will architect, refactor, and operate robust data pipelines—transitioning legacy Hive table structures to modern Apache Iceberg formats, migrating and scaling Apache Airflow DAGs, and establishing cross-regional Kafka event streams. Operating within an AWS-native cloud environment and Snowflake data warehouse, you will ensure high-throughput processing, optimal query performance, and strict data quality standards throughout the migration lifecycle.

Key Responsibilities
  • Dataset & Pipeline Migration: Execute the end-to-end migration of mission-critical datasets, legacy data workflows, and orchestrated DAGs to modernized cloud infrastructure.
  • Lakehouse Modernization (Hive to Iceberg): Convert legacy Hive table architectures to Apache Iceberg format on AWS S3, improving partition evolution, query performance, and metadata management.
  • Distributed Streaming Infrastructure: Reconfigure, validate, and manage Apache Kafka connections, brokers, and topics across multiple AWS regions to support continuous data streaming.
  • Snowflake Modeling & Query Optimization: Adapt, model, and tune data pipelines directly in Snowflake, leveraging advanced SQL techniques (CTEs, window functions, complex joins) to optimize compute utilization and latency.
  • Workflow Orchestration & Maintenance: Author, schedule, and actively debug Apache Airflow DAGs, resolving upstream failures, optimizing sensor dependencies, and preventing pipeline bottlenecks.
  • Data Quality & Testing Strategy: Define and execute structured data reconciliation and validation strategies to guarantee zero data loss, schema consistency, and end-to-end data integrity across migrations.
  • Independent Root-Cause Remediation: Autonomously diagnose, profile, and resolve complex performance issues across distributed Spark/PySpark execution jobs, Kafka consumer groups, and AWS storage layers.
Must-Have Skills
  • Senior Data Engineering Background: 6+ years of hands-on data engineering experience designing, deploying, and maintaining distributed, production-grade data pipelines on petabyte-scale datasets.
  • Core Programming & Advanced Querying: High proficiency in Python (pipeline scripting, data manipulation) and advanced SQL (complex CTEs, window functions, schema design, and query optimization).
  • Distributed Processing & Big Data: Proven production expertise with Apache Spark / PySpark for batch and streaming transformations.
  • Orchestration & Streaming: Strong hands-on experience authoring and maintaining complex DAGs in Apache Airflow and configuring cross-regional event pipelines with Apache Kafka.
  • Modern Lakehouse & Cloud Storage: Direct experience managing AWS S3 data lakes utilizing Apache Iceberg and Parquet formats.
  • Cloud Data Warehousing: Hands-on experience with Snowflake schema modeling, warehouse configuration, and query tuning.
  • Autonomous Debugging: Demonstrated capability to independently troubleshoot distributed pipeline execution failures, network latencies, and serialization bottlenecks.
  • Enterprise Environment Compliance: Must be able to operate within an enterprise AWS WorkSpace (VDI) environment and complete a mandatory AWS network health-check verification (with passing screenshot required during application).
Nice-to-Have Skills
  • Enterprise Snowflake Migrations: Hands-on experience migrating legacy on-prem or cloud data warehouses to Snowflake.
  • Airflow Platform Upgrades: Practical experience handling platform-level Airflow migrations (e.g., major version upgrades, executor switches to Celery/Kubernetes).
  • AI-Assisted Development: Experience utilizing AI-assisted software development tools (e.g., GitHub Copilot, Cursor) to optimize data modeling, pipeline scripting, and debugging.
  • Salesforce Domain Experience: Exposure to Salesforce data architectures, object modeling, and CDC/API ingestion mechanisms.
Time Zone & Collaboration

This role requires full-time business-hour alignment with US Central Time (CT) to ensure real-time collaboration with US engineering leadership, active participation in daily standups, and live architectural syncs.

Language

All interviews, technical documentation, architectural reviews, daily team syncs, and operational communications will be conducted in English.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer (Salesforce / Snowflake Focus) 2153/
Data Engineer (Salesforce / Snowflake Focus) 2153/

In All Media • United States

Remote
USD 120,000 - 180,000
Data Engineer (Salesforce Focus) (Fully Remote)
Data Engineer (Salesforce Focus) (Fully Remote)

In All Media • United States

Remote
USD 140,000 - 190,000
Remote work
Contractor role
2128 Data Engineer
2128 Data Engineer

In All Media Inc • United States

Remote
USD 120,000 - 170,000
Data Platform Engineer
Data Platform Engineer

In All Media Inc • United States

Remote
USD 90,000 - 140,000
Remote Data Engineer - Salesforce & Cloud Pipelines
Remote Data Engineer - Salesforce & Cloud Pipelines

In All Media • United States

Remote
USD 140,000 - 190,000
Remote work
Contractor role
Remote Senior Data Engineer - Salesforce Focus
Remote Senior Data Engineer - Salesforce Focus

In All Media • United States

Remote
USD 130,000 - 180,000
Senior Salesforce Developer
Senior Salesforce Developer

In All Media Inc • United States

Remote
USD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

AgileEngine, LLC. • United States

Hybrid
USD 130,000 - 190,000
Professional growth
Flexible schedule
Remote and office options
Senior Salesforce Developer (Security & Data Cloud Governance)
Senior Salesforce Developer (Security & Data Cloud Governance)

In All Media Inc • United States

Remote
USD 140,000 - 210,000
Salesforce Developer
Salesforce Developer

In All Media Inc • United States

Remote
USD 120,000 - 180,000