Senior Data Platform Engineer - Real-Time & ML Pipelines

Visa Technology and Operations LLC; Austin (TX); Hybrid; USD 137,000 - 220,000; Full time

Benefits offered by this job

Medical

Dental

Vision

401(k)

FSA/HSA

Life Insurance

Paid Time Off

Wellness Program

Bonus and equity

Job summary

Visa Technology and Operations LLC is seeking a Staff Data Engineer in Austin, TX to design and advance the company’s data token platform. This hybrid role focuses on building large-scale data pipelines and data platforms for analytics, reporting, and ML, collaborating with product, analytics, and business teams to deliver scalable data solutions.

You will utilize Hadoop, Spark, Python, SQL, and modern data technologies, while mentoring engineers and driving data governance and performance

Qualifications

• 5+ years of relevant work experience with a Bachelor’s Degree, or equivalent combination
• Demonstrated leadership delivering high-quality, large-scale, enterprise-class applications.
• Solid big data engineering experience with Hadoop, Apache Spark, Python, and SQL.
• Experience with containerization using Docker and Kubernetes.
• Ability to build and manage large-scale data pipelines and ML models.
• ETL processes, Spark pipelines, and productizing AI/ML models.
• Proficiency with Kafka, Redis, Flink, TensorFlow, Triton, and AWS services.
• Unix/Shell scripting and scheduling tools like Airflow and Control-M.
• Familiarity with Agile, TDD, CI/CD, and various databases.
• Strong communication, collaboration, and mentoring skills.

Responsibilities

• Design, build, and optimize large-scale data pipelines for batch and real-time processing.
• Develop and maintain distributed data systems that are highly available, scalable, and resilient.
• Lead data integration across platforms and govern data quality and performance.
• Drive development of data platforms and data products for analytics, reporting, and ML.
• Partner with product, analytics, and business teams to translate requirements into scalable data solutions.
• Architect solutions using Hadoop, Spark, and distributed processing frameworks.
• Build and optimize ETL/ELT pipelines with strong data governance.
• Implement automation, monitoring, and observability across data pipelines.
• Lead PoCs to evaluate new data engineering technologies.
• Mentor engineers and promote best practices in data modeling and governance.

Skills

Hadoop

Apache Spark

Python

SQL

Docker

Kubernetes

Kafka

Redis

Flink

TensorFlow

Triton

AWS services

Unix/Shell

Airflow

Control-M

Agile

TDD

CI/CD

Databases

Education

Bachelor’s Degree

Tools

Docker

Kubernetes

Airflow

Control-M

AWS services

Job description

Visa Technology and Operations LLC is seeking a Staff Data Engineer in Austin, TX to design and advance the company’s data token platform. This hybrid role focuses on building large-scale data pipelines and data platforms for analytics, reporting, and ML, collaborating with product, analytics, and business teams to deliver scalable data solutions.

You will utilize Hadoop, Spark, Python, SQL, and modern data technologies, while mentoring engineers and driving data governance and performance