Senior Consultant - Data Engineer (Data Platforms)

DataJobs; Foster City (CA); On-site; USD 181,000 - 289,000; Full time

Benefits offered by this job

Medical

Dental

Vision

401(k)

FSA/HSA

Life Insurance

Paid Time Off

Wellness Program

Job summary

Visa’s Data Platforms group seeks a Senior Data Engineer to design and build petabyte-scale data processing and delivery platforms supporting core payment domains. The role blends hands-on engineering with leadership, reliability improvements, and cross-functional collaboration to advance data discovery and governance.

As a senior contributor, you will lead end-to-end data pipelines, mentor engineers, and drive architecture reviews, while partnering with data scientists and product managers to

Qualifications

• Strong knowledge of SQL, NoSQL, graph databases, and RESTful APIs.
• Extensive experience with Spark, Hive, and Big Data tools.
• Proficient in Java, Scala, or Python for production-grade components.
• Experience leading large-scale data platform projects.

Responsibilities

• Collaborate with product managers and stakeholders to define requirements for scalable data platform solutions.
• Design, develop, and enhance distributed data processing systems for payment and analytics workloads.
• Build and maintain high-performance data pipelines delivering large data volumes.
• Contribute to data platform architecture, delivery capabilities, and Master Data Management.
• Lead technical design discussions and architecture reviews.
• Develop clean, scalable code aligned with engineering best practices and standards.
• Drive reliability, observability, automation, and operational excellence.
• Mentor engineers and lead code reviews to promote technical excellence.

Education

Bachelor's degree

Tools

SQL

NoSQL

Graph databases

RESTful APIs

Spark

Hive

Java

Scala

Python

Apache Airflow

Git

CI/CD

Generative AI

Large Language Models

Copilot

Job description

As a Senior Data Engineer within Visa’s Data Platforms organization, you will design and build petabyte-scale data processing and delivery platforms that support core payment domains. The role combines hands-on engineering with technical leadership, reliability improvements, and cross-functional collaboration to advance data discovery and governance.

Role Overview

Visa’s Data Platforms organization delivers the foundational data ecosystem powering critical business capabilities. Teams build and operate large-scale platforms for authorization, clearing, settlement, and related payment data domains, while enabling next-generation data products, analytics, and insights across Visa. This position operates in a cross-functional Agile team focused on scalable data solutions, modernization of data delivery capabilities, and advancement of Master Data Management and platform initiatives.

Key Responsibilities

• Collaborate with product managers, architects, and business stakeholders to define requirements and translate business needs into scalable data platform solutions.
• Design, develop, and enhance distributed data processing systems for Visa’s critical payment, settlement, authorization, and analytics workloads.
• Build and maintain high-performance, reliable data pipelines that process and deliver large volumes of data across platforms, products, and business domains.
• Contribute to the evolution of Visa’s data platform architecture, data delivery capabilities, and Master Data Management solutions.
• Lead technical design discussions, architecture reviews, and engineering decision-making for complex initiatives.
• Develop clean, scalable, maintainable, and reusable code aligned with engineering best practices and platform standards.
• Drive improvements in reliability, observability, automation, performance, and operational excellence.
• Lead code reviews and mentor fellow engineers to support a culture of technical excellence and continuous learning.
• Troubleshoot and resolve complex production issues while identifying opportunities to automate manual work and reduce operational risk.
• Develop testing strategies and quality assurance practices to ensure robust and dependable data solutions.
• Work closely with engineers, data scientists, architects, product managers, and platform teams to deliver business-critical capabilities.
• Identify, analyze, and address data quality, performance, scalability, governance, and security challenges proactively.
• Support project planning, estimation, and delivery while balancing business priorities, technical debt, and long-term platform health.

Required Qualifications

• Strong working knowledge of SQL, NoSQL, graph databases, and RESTful APIs.
• Expert-level experience with the Spark framework, Hive-based technologies, and broader Big Data development tools and platforms.
• Extensive hands-on programming experience in Java, Scala, or Python, with the ability to produce clean, production-quality prototypes and core application components.
• Proven ability to lead large-scale, end-to-end projects that deliver measurable business impact from concept through production delivery.
• Strong understanding of algorithms, data structures, and core computer science fundamentals.
• Practical experience with data modeling for both relational and non-relational database systems.
• Demonstrated leadership behaviors, including positive mindset, rapid learning, collaboration, courage in leadership, and strong customer obsession.

Technology Stack

• SQL, NoSQL, graph databases, RESTful APIs
• Spark, Hive, Big Data
• Java, Scala, Python
• Apache Airflow, Git, CI/CD
• Generative AI, Large Language Models, Microsoft Copilot, ChatGPT, and LLM-powered engineering solutions

Preferred Qualifications

• Practical experience applying Generative AI and Large Language Models in data engineering, including leading initiatives that improve productivity and operational efficiency.
• Experience collaborating with data scientists and machine learning teams, including discussion of major ML paradigms, algorithms, and common software tools.
• Hands-on experience designing, building, and managing reliable data pipelines using tools such as Apache Airflow.
• Proficiency with Git and experience implementing CI/CD pipelines for automated testing, deployment, and release management.
• Experience mentoring junior engineers and providing technical leadership.
• Experience working with cross-functional teams to deliver solutions, including participation in security assessment processes and sharing architectural learnings and code plans with the team.
• Ability to manage multiple competing priorities in a fast-paced, dynamic environment.
• Experience supporting data quality, data lineage, data governance, compliance, and security requirements within data engineering environments.

Compensation

Estimated salary range: USD 180,600 to 289,300 per year.

For US applicants, the estimated salary range may include potential sales incentive payments (if applicable). Salary may vary based on job-related factors, including knowledge, skills, experience, and location. The position may also be eligible for bonus and equity.

Benefits

• Medical
• Dental
• Vision
• 401(k)
• FSA/HSA
• Life Insurance
• Paid Time Off
• Wellness Program

Work Arrangement and Logistics

• Location: Foster City, CA (onsite)
• Travel Requirements: 5-10% of the time
• Work Hours: Varies based on department needs
• Mental/Physical Requirements: Office setting; sitting and standing at a desk; communication in person and by telephone; frequent use of standard office equipment such as telephones and computers

Equal Employment Opportunity

Visa is an EEO employer. Qualified applicants will receive consideration without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Visa will also consider qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Additional Information for the Role

• Minimum Experience: 8 years
• Education: Bachelor’s degree