Senior Data Engineer — AI-Driven Data Pipelines

Jobtailor; New Jersey; On-site; USD 140,000 - 190,000; Full time

Job summary

BetMGM seeks a senior data platform engineer to own end-to-end data pipelines from raw data to clean, trusted datasets powering analytics and ML systems. You will design and operate batch, micro-batch, and streaming pipelines feeding Snowflake, using Prefect-orchestrated flows on ECS Fargate and dbt for transformations.

The role requires deep Snowflake administration, strong AWS experience, and IaC with Terraform.

Qualifications

• 5+ years building production data pipelines on a modern stack (Python + SQL + dbt + cloud).
• Deep Snowflake administration knowledge: warehouse sizing, RBAC, monitors, Streams/Tasks, Dynamic Tables, secure data sharing, cost tuning via ACCOUNT_USAGE.
• Strong AWS experience: S3, ECS/Fargate, Lambda, IAM, Secrets Manager, VPC, plus EMR Serverless/Glue/MWAA experience.
• Terraform for cloud and Snowflake: demonstrated IaC ownership.

Responsibilities

• Own the path from raw transactional and event data to trustworthy datasets powering analytics, ML, and operational systems.
• Design, build, and operate batch, micro-batch, and streaming pipelines feeding Snowflake.
• Own the full dbt lifecycle with contracts, freshness SLAs, tests, and docs.
• Stand up Snowflake objects via Terraform—no ClickOps in production.
• Build AWS-native data infrastructure (S3, ECS Fargate, Lambda, EMR Serverless, Glue) in Terraform.
• Maintain CI/CD pipelines with linting, dbt build, tests, and contract checks.
• Tune warehouses and query patterns for cost and latency; monitor usage.
• Design RBAC and masking policies to meet regulatory needs.
• Own data freshness SLAs and triage incidents end-to-end.
• Direct AI coding agents to scale workflows and review AI-generated PRs.
• Collaborate with analytics engineers, data scientists, and ML platform engineers on standards.

Skills

Python

SQL

dbt

Snowflake

AWS

Terraform

Prefect

Airflow

CI/CD

SLA management

AI in workflow

Education

BS/MS in CS/Math

Tools

Snowflake Admin

Kinesis

EMR Serverless

MWAA

S3

Job description

BetMGM seeks a senior data platform engineer to own end-to-end data pipelines from raw data to clean, trusted datasets powering analytics and ML systems. You will design and operate batch, micro-batch, and streaming pipelines feeding Snowflake, using Prefect-orchestrated flows on ECS Fargate and dbt for transformations.

The role requires deep Snowflake administration, strong AWS experience, and IaC with Terraform.