Senior Data Ingestion Engineer - Streaming & Lakehouse

Momentum

San Francisco (CA)

On-site

USD 180,000 - 240,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Slack is seeking a Senior Software Engineer for the Data Ingestion Team within the Data Engineering org. You will design and operate the platform that ingests data from diverse sources into our Lakehouse using streaming and batch approaches, handling 180PB+ storage and 1T records/day across 400+ pipelines and 1500+ Airflow DAGs.

You’ll lead initiatives to scale streaming and lakehouse strategies, build scalable services, mentor engineers, and collaborate with cross-functional partners to deliver

Qualifications

  • 7+ years of software/data engineering experience in streaming and lakehouse tech.
  • Experience with building apps using Kafka, Flink or Spark Streaming, Iceberg/Hudi/Delta.
  • Hands-on with Spark, Airflow, EMR, and AWS S3 for batch pipelines.
  • Proficient in SQL, Python, Java/Scala, Go.
  • Experience crafting distributed microservices with Kubernetes, Docker, AWS ECS/EKS, Terraform, and observability tools.

Responsibilities

  • Design and develop highly scalable streaming and batch pipelines for ingesting and processing datasets.
  • Design and implement Slack’s Lakehouse solutions and enable production use cases for customers.
  • Improve the reliability and performance of services/pipelines with AI-assisted/Agentic solutions.
  • Automate dataset lifecycle: schema evolution, metadata management, backfill, deprecation, migration.
  • Lead technical initiatives end to end and collaborate with cross-functional partners.
  • Mentor and coach engineers across the team and org.

Skills

Kafka
Spark Streaming
Airflow
S3
Kubernetes
Docker
Terraform
Iceberg
Delta

Tools

Kafka Connect
Flink
Spark

Job description

Slack is seeking a Senior Software Engineer for the Data Ingestion Team within the Data Engineering org. You will design and operate the platform that ingests data from diverse sources into our Lakehouse using streaming and batch approaches, handling 180PB+ storage and 1T records/day across 400+ pipelines and 1500+ Airflow DAGs.

You’ll lead initiatives to scale streaming and lakehouse strategies, build scalable services, mentor engineers, and collaborate with cross-functional partners to deliver

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Ingestion Engineer - Lakehouse & Streaming
Senior Data Ingestion Engineer - Lakehouse & Streaming

Slack • San Francisco (CA)

On-site
USD 173,000 - 286,000
Senior Data Ingestion Engineer – Streaming & Lakehouse
Senior Data Ingestion Engineer – Streaming & Lakehouse

Slack • Bellevue (WA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer – Streaming & Lakehouse
Senior Data Ingestion Engineer – Streaming & Lakehouse

Showcify, Inc. • Atlanta (GA)

On-site
USD 173,000 - 260,000
Medical
Dental
Vision
+2
Senior Data Ingestion & Lakehouse Engineer
Senior Data Ingestion & Lakehouse Engineer

Slack • Atlanta (GA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer: Streaming & Lakehouse
Senior Data Ingestion Engineer: Streaming & Lakehouse

Salesforce, Inc. • Atlanta (GA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer - Streaming & Lakehouse
Senior Data Ingestion Engineer - Streaming & Lakehouse

Slack • Seattle (WA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer: Streaming & Lakehouse
Senior Data Ingestion Engineer: Streaming & Lakehouse

Salesforce, Inc. • Bellevue (WA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer – Streaming & Lakehouse
Senior Data Ingestion Engineer – Streaming & Lakehouse

Salesforce, Inc. • San Francisco (CA)

On-site
USD 173,000 - 286,000
Senior Data Ingestion Engineer - Streaming & Lakehouse Lead
Senior Data Ingestion Engineer - Streaming & Lakehouse Lead

Slack Enterprise • Atlanta (GA)

On-site
USD 173,000 - 260,000
Senior Data Ingestion Engineer, Streaming & Lakehouse
Senior Data Ingestion Engineer, Streaming & Lakehouse

Salesforce, Inc. • Seattle (WA)

On-site
USD 173,000 - 260,000