Data Engineer

VIDA Digital Identity; Bengaluru; On-site; INR 800,000 - 1,500,000; Full time

Job summary

VIDA Digital Identity is seeking a skilled Data Engineer to help modernize data pipelines and support real-time fraud detection efforts. The ideal candidate will have 5+ years of relevant experience, particularly in building data pipelines with technologies such as Scala Spark and Terraform.

Your role will involve defining data lakes, enhancing existing ETLs, and collaborating with engineering and data science teams to deliver reliable data solutions. A familiarity with AWS services and a strong engineering background are essential.

Qualifications

• 5+ years of relevant experience in data engineering and/or distributed systems.
• Hands-on experience with data pipelines using distributed frameworks.
• Familiarity with AWS data services including S3 and Redshift.

Responsibilities

• Build and expose the data lake/lakehouse for better decision-making.
• Design and evolve a data platform for scientists and analysts.
• Develop scalable batch and streaming data pipelines.

Skills

Data engineering

Distributed systems

SQL

Streaming pipelines

Batch pipelines

Reliability

Collaboration

Tools

Scala Spark

Terraform

AWS services

dbt

Airflow

Job description

At VIDA we’re building the future of digital identity. As a Data Engineer you’ll work across the stack—from platform and infrastructure to data pipelines and end-user tooling. You’ll help us modernize our batch ETLs and lead our push into streaming and real-time fraud detection. We run on AWS and make extensive use of Scala Spark, dbt, and Terraform.

Responsibilities

• Build and expose the data lake/lakehouse so teams can understand business and product performance and make better decisions.
• Design and evolve a data platform that enables scientists and analysts to query, model, and ship features and product metrics.
• Develop scalable and reliable batch and streaming pipelines that ingest from varied sources and land trusted datasets.
• Maintain and fix existing ETLs, improving reliability, observability, and cost while planning safe backfills and replays.
• Model curated layers and maintain transformation logic (e.g., dbt), including testing, documentation, and data quality.
• Build with Infrastructure as Code (e.g., Terraform) and contribute to CI/CD and deployment practices.
• Partner with Engineering, and Data Science to deliver low-latency signals and features for real-time decisioning.
• Contribute to internal tools, reusable components, and simple dashboards that unblock self-service.

Requirements

• Solid engineering foundation with 5+ years of relevant experience in data engineering and/or distributed systems.
• Hands‑on work building data pipelines (batch and/or streaming) using at least one distributed framework (e.g., Spark, Kafka, Flink).
• Comfortable with SQL and a transformation framework (e.g., dbt or similar).
• Familiarity with AWS services for data (any mix of S3, EMR/Glue, Kinesis/MSK, Lambda, Redshift/Athena) or equivalent cloud experience.
• Exposure to workflow orchestration (e.g., Airflow).
• Experience using Terraform (or similar IaC) to manage data/platform resources.
• Ownership mindset, focus on reliability, and enjoyment of collaborating in a fast-moving environment.