Senior Data Engineer: Lead Large-Scale Pipelines & AI

Visa

Foster City (CA)

On-site

USD 157,000 - 251,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Visa is seeking a Staff Data Engineer to lead and build large-scale data systems. You will code, debug, and optimize data workloads with Spark, Scala, Hadoop, Hive, and Kafka while partnering with product managers, data scientists, and other teams.

You will own data products end-to-end, ensure security and governance, and mentor engineers while adapting to AI-assisted development tools. This role requires 3 days in office in Foster City, CA.

Qualifications

  • 5+ years of relevant work experience with a Bachelor's Degree or 8+ years with an advanced degree.
  • Strong hands-on Apache Spark and Scala expertise with Hadoop, Hive and Kafka experience.
  • Proven experience with complex batch and streaming pipelines, SQL, data modeling, orchestration, and automated data quality controls.

Responsibilities

  • Translate business requirements into scalable data architectures and designs.
  • Develop and optimize distributed data workloads using Spark, Scala, Hadoop, Hive, and Kafka.
  • Deliver curated datasets, reusable data products, and secure API-based data services for enterprise consumption.
  • Own pipelines end-to-end: deployment, monitoring, troubleshooting, documentation, and debt reduction.
  • Mentor engineers and enforce secure coding standards and reviews.

Skills

Apache Spark
Scala
Hadoop
Hive
Kafka
SQL
Data modeling
CI/CD
Observability
Security

Education

Bachelor's degree or higher

Tools

Databricks
Delta Lake
Unity Catalog
Apache Iceberg

Job description

Visa is seeking a Staff Data Engineer to lead and build large-scale data systems. You will code, debug, and optimize data workloads with Spark, Scala, Hadoop, Hive, and Kafka while partnering with product managers, data scientists, and other teams.

You will own data products end-to-end, ensure security and governance, and mentor engineers while adapting to AI-assisted development tools. This role requires 3 days in office in Foster City, CA.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer: Spark, AI Tools | Hybrid + Equity
Senior Data Engineer: Spark, AI Tools | Hybrid + Equity

VISA • Foster City (CA)

On-site
USD 132,000 - 204,000
Data Engineer - Build Robust Data Pipelines
Data Engineer - Build Robust Data Pipelines

VISA • Foster City (CA)

On-site
USD 102,000 - 158,000
Medical
Dental
Vision
+5
Senior Data Engineering Leader | GenAI & Big Data
Senior Data Engineering Leader | GenAI & Big Data

Visa • Foster City (CA)

On-site
USD 192,000 - 308,000
Comprehensive benefits
Bonus eligibility
Equity potential
Onsite Associate Data Engineer - Pipelines & Growth
Onsite Associate Data Engineer - Pipelines & Growth

Visa Technology and Operations LLC • Foster City (CA)

On-site
USD 102,000 - 158,000
Medical, Dental, Vision
401(k)
FSA/HSA
+4
Senior AI Data Engineer: Databricks, Spark & ML Pipelines
Senior AI Data Engineer: Databricks, Spark & ML Pipelines

HCL Global Systems Inc • Princeton (NJ)

On-site
USD 150,000 - 190,000
Senior Data Scientist: AI Solutions Leader
Senior Data Scientist: AI Solutions Leader

Visa • Austin (TX)

On-site
USD 133,000 - 213,000
Medical insurance
Dental insurance
Vision insurance
+3
Senior Data Analytics Engineer — Scalable Pipelines & AI
Senior Data Analytics Engineer — Scalable Pipelines & AI

Cloud Destinations LLC. • Santa Clara (CA)

On-site
USD 83,000 - 87,000
Senior Data Engineer: Scalable Pipelines & AI Workflows
Senior Data Engineer: Scalable Pipelines & AI Workflows

Code and Theory, LLC • San Francisco (CA)

On-site
USD 120,000 - 160,000
Staff Data Engineering Lead - Scalable Pipelines
Staff Data Engineering Lead - Scalable Pipelines

EngineersOfAI • Seattle (WA)

On-site
USD 130,000 - 170,000
Junior Data Engineer: Lead Scalable Pipelines & Data Solutions
Junior Data Engineer: Lead Scalable Pipelines & Data Solutions

Jobright.ai • Cupertino (CA)

On-site
USD 120,000 - 180,000