Data Engineer

Neoveda Technologies; Bengaluru; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Neoveda Technologies is seeking a Senior Data Engineer to design, build, and own end-to-end production analytics pipelines with strong data architecture and system design. The role emphasizes Python + SQL, transaction-level fact tables, and rigorous data quality and auditability within finance/risk/compliance domains. Location: Bengaluru (on-site).

6+ years of experience required. Familiarity with Uber's data platform stack—Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder,

Qualifications

• 6+ years in data engineering.
• Strong data architecture and system design skills.
• Expert-level Python + SQL proficiency.
• Experience with finance, risk, or compliance data.
• End-to-end ownership of production data pipelines.
• Proven ability designing and building transaction/fact tables.
• Advanced SQL skills (window functions, cross-dialect MySQL/PostgreSQL).

Responsibilities

• Own data architecture and system design decisions for pipelines and data models.
• Take end-to-end ownership of pipelines in production: design, build, deploy, monitor, and operate.
• Design and build transaction tables (append-only, immutable) and medallion architecture.
• Build and support pipelines for finance, risk, or compliance use cases.
• Implement data quality controls, validation, reconciliation, anomaly detection, audit trails.
• Write complex cross-dialect SQL and templated (Jinja) SQL for late-arriving data.
• Orchestrate DAGs with secure credential handling via uSecret.
• Build Python ETL tooling with pandas and SQLAlchemy.
• Integrate with external APIs and YAML-driven configs; maintain Git-based CI/CD.

Skills

Python

SQL

Data architecture

End-to-end pipelines

Airflow

Data quality

Finance domain

Tools

Piper/uWorc

Databook

uSecret

DSW

SourceGraph

Query Builder

OneETL

Job description

About the Role:

We're looking for a Senior Data Engineer with strong data architecture and system design

skills to design, build, and own end-to-end production pipelines powering analytics and

decision-making across the org.

The role centers on deep Python + SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain. Internal candidates familiar with

Uber's data platform stack (Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder, OneETL) will be prioritized.

Location: Bangalore

Experience: 6+ Years

What You'll Do:

• Own data architecture and system design decisions for pipelines and data models -

grain, partitioning, schema evolution, and scalability tradeoffs.

• Take end-to-end ownership of pipelines in production: design, build, deploy, monitor,

and operate.

• Design and build transaction tables (append-only, immutable event/transaction-grain fact

tables) alongside standard fact/dimension models using medallion (bronze/silver/gold)

architecture.

• Build and support pipelines for finance, risk, or compliance use cases where

accuracy, auditability, and data lineage are critical.

• Implement data quality and auditability controls: validation checks, reconciliation

logic, anomaly detection, and audit trails for every pipeline you own.

• Write complex, cross-dialect SQL (MySQL + PostgreSQL): window functions,

multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and

date/timestamp handling.

• Build idempotent reload patterns (DELETE+INSERT), UNION ALL/set operations, and

templated (Jinja-style) SQL including handling for late-arriving/corrected transaction

records without double-counting.

• Orchestrate Airflow-style DAGs (Pipeline/BaseTask, ExternalTaskSensor for

cross-pipeline deps) with secure credential handling via uSecret.

• Build Python ETL tooling: pandas (CSV DB), SQLAlchemy + raw drivers (MySQLdb,

psycopg2), type hints/dataclasses, class-based pipeline design.

• Integrate with Google Drive/Sheets APIs; build YAML-driven pipeline configs; handle

Piper staging/user_staging quirks.

• Own CI/CD for pipelines via standard Git workflows.

• Use Uber's internal stack day-to-day: OneETL for pipeline authoring, Piper/uWorc for

scheduling, Databook for lineage, uSecret for credentials, DSW notebooks for CSV/table

work, SourceGraph for code search, Query Builder for ad hoc SQL.

• Document data models and pipeline logic for maintainability, keeping compliance/audit

needs in mind.

What We're Looking For:

Must-Have:

• 6+ years in data engineering.
• Strong data architecture and system design skills to make sound tradeoffs on modeling,

partitioning, and scalability, not just write queries.

• Demonstrated end-to-end production ownership of data pipelines from design through

deployment, monitoring, and operational support.

• Expert-level Python + SQL as the primary toolset. Hive/Hadoop is a nice-to-have only.
• Robust data quality and auditability practices - validation, reconciliation, anomaly

detection, audit trails.

• Experience working with finance, risk, or compliance data - comfort with the heightened

accuracy, auditability, and data-lineage requirements typical of these domains.

• Proven experience designing and building transaction/fact tables.
• Expert SQL (window functions, CTEs, CASE logic, cross-dialect MySQL/PostgreSQL).
• Strong Python (pandas, SQLAlchemy, type hints/dataclasses, class-based pipelines).
• Airflow-like orchestration experience (Piper/uWorc a plus).
• Solid data modeling fundamentals (fact/dim, normalization, medallion architecture).
• YAML-based config experience; standard CI/CD and Git workflows.

Strongly Preferred (Uber Internal Tooling):

Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder, OneETL.

Nice-to-Have:

Hive/Hadoop or Spark experience