Remote Senior Data Lake Engineer — Spark & Airflow

VISA; United States; À distance; USD 27 000 - 44 000; Plein temps

Résumé du poste

Visa is seeking a Data Engineer at the Analyst level to join our Data Lake team. You'll build and maintain data ingestion and transformation pipelines using PySpark, SparkSQL, Databricks, and Airflow, contributing to the reliability and quality of the corporate data lake in a remote role.

This role expects you to work under moderate supervision on defined tasks while developing deeper expertise in lakehouse architecture.

Qualifications

• 3+ years of relevant work experience with a Bachelor's or Associate’s Degree OR 5+ years of relevant work experience.
• Python for automation and data processing.
• Data pipeline concepts (batch, ETL/ELT patterns).

Responsabilités

• Develop, test, and maintain data pipelines (ingestion, transformation, quality checks) using PySpark/SparkSQL on Databricks.
• Build and modify Airflow DAGs (MWAA) for pipeline orchestration.
• Write and optimize SQL queries for data transformation and validation.
• Support data quality by implementing and monitoring quality checks.
• Participate in code reviews — both giving and receiving feedback.
• Investigate pipeline failures and data quality issues with guidance from senior engineers.
• Write and maintain documentation for datasets and pipelines you build.
• Participate in sprint planning, estimation, and retrospectives.
• Follow CI/CD, testing, and governance standards defined by the team.

Description du poste

Visa is seeking a Data Engineer at the Analyst level to join our Data Lake team. You'll build and maintain data ingestion and transformation pipelines using PySpark, SparkSQL, Databricks, and Airflow, contributing to the reliability and quality of the corporate data lake in a remote role.

This role expects you to work under moderate supervision on defined tasks while developing deeper expertise in lakehouse architecture.