Senior Staff Software Engineer - Data Engineering Staff

D8 LooP; Santa Clara (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

NVIDIA is seeking a data engineer to optimize distributed data processing, build robust ingestion pipelines, and ensure reliable analytics at scale. You will collaborate with teams to model data, implement incremental loads, and manage schema evolution using Databricks and Spark.

You will implement data freshness checks, monitoring, and automated recovery, while writing maintainable Python and SQL code. This role emphasizes production reliability and scalable architectures across lakehouse

Qualifications

• Experience with data modeling, incremental processing, partitioning, schema evolution and analytical query performance.
• Strong Python and SQL skills, with experience writing maintainable, tested code and debugging complex data transformations.
• Experience with automated tests, data quality checks, monitoring, incident diagnosis and recovery.
• Build reusable ingestion frameworks and data contracts for APIs and file-based sources, handling schema changes, late-arriving data and safe backfills.
• Experience with Databricks, Spark and lakehouse technologies, or comparable distributed data platforms.
• Experience with data catalogs, workflow orchestration, observability platforms or AI-assisted anomaly detection and root-cause analysis.

Responsibilities

• Own ingestion pipelines and data contracts for APIs and file-based sources, handling schema changes and late-arriving data.
• Implement data freshness, completeness and accuracy checks, lineage and monitoring.
• Diagnose incidents and automate retries, replay and recovery.

Skills

Cost & performance

Data modeling

Incremental processing

Partitioning

Schema evolution

Analytical query performance

Python

SQL

Ingestion frameworks

Databricks

Apache Spark

Airflow

Data governance

Backfills

Tools

Databricks

Apache Spark

Job description

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing.

Skills, with evidence

• Cost & performance Optimize distributed processing, storage and serving-layer queries for performance, scalability and infrastructure cost. must have
• Experience with data modeling, incremental processing, partitioning, schema evolution and analytical query performance. must have
• A track record of owning production reliability through automated tests, data quality checks, monitoring, incident diagnosis and recovery. must have
• Diagnose incidents and automate retries, replay and recovery. must have
• Incremental loads Experience with data modeling, incremental processing, partitioning, schema evolution and analytical query performance. must have
• Python Strong Python and SQL skills, with experience writing maintainable, tested code and debugging complex data transformations. must have
• SQL Strong Python and SQL skills, with experience writing maintainable, tested code and debugging complex data transformations. must have
• Build reusable ingestion frameworks and data contracts for APIs and file-based sources, handling schema changes, late-arriving data and safe backfills. must have
• Spark Hands-on experience with Databricks, Apache Spark and lakehouse technologies, or comparable distributed data platforms. must have
• Idempotency & backfills Build reusable ingestion frameworks and data contracts for APIs and file-based sources, handling schema changes, late-arriving data and safe backfills.
• Governance & security Implement data freshness, completeness and accuracy checks, lineage and actionable monitoring. not practised here
• Airflow / orchestration Experience with data catalogs, workflow orchestration, observability platforms or AI-assisted anomaly detection and root-cause analysis.
• Warehousing Hands-on experience with Databricks, Apache Spark and lakehouse technologies, or comparable distributed data platforms.