Senior Data Lake Engineer - Pipelines & Databricks

VISA; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Visa is seeking a Data Engineer at Analyst level to join our Data Lake team. You will build and maintain data ingestion and transformation pipelines using Spark, Databricks, and Airflow, contributing to the reliability and quality of the corporate data lake.

This is an intermediate role with moderate supervision while deepening expertise in lakehouse architecture. We focus on centralizing data into a single platform that supports decision-making across the company and for external clients,

Qualifications

• 3+ years of relevant work experience with a Bachelor's or Associate's Degree OR 5+ years of relevant work experience.
• Experience with Python for automation and data processing.
• Knowledge of data pipeline concepts (batch, ETL/ELT).

Responsibilities

• Develop, test, and maintain data pipelines (ingestion, transformation, quality checks) using PySpark/SparkSQL on Databricks.
• Build and modify Airflow DAGs (MWAA) for pipeline orchestration.
• Write and optimize SQL queries for data transformation and validation.
• Support data quality by implementing and monitoring quality checks (Great Expectations or equivalent).
• Participate in code reviews and provide/receive feedback.
• Investigate pipeline failures and data quality issues with guidance from senior engineers.
• Write and maintain documentation for datasets and pipelines you build.
• Participate in sprint planning, estimation, and retrospectives.

Skills

Python

Spark

Databricks

SQL

Airflow

Git/GitHub

Education

Bachelor's degree or equivalent

Tools

AWS (S3)

Delta Lake

Airflow MWAA

Job description

Visa is seeking a Data Engineer at Analyst level to join our Data Lake team. You will build and maintain data ingestion and transformation pipelines using Spark, Databricks, and Airflow, contributing to the reliability and quality of the corporate data lake.

This is an intermediate role with moderate supervision while deepening expertise in lakehouse architecture. We focus on centralizing data into a single platform that supports decision-making across the company and for external clients,