Senior Data Engineer – AWS & Databricks

Talpro; Bengaluru; Hybrid; INR 1,530,000 - 1,870,000; Full time

Job summary

Talpro is seeking a Senior Data Engineer to design, implement, and optimize scalable data pipelines on AWS and Databricks. You will work with Spark, ETL patterns, and cloud-native architectures to deliver robust analytics solutions.

The role is hybrid in Bengaluru, with 6–10 years of Data Engineering experience, a 12-month contract, and an immediate joining timeline. Collaboration with data science and business teams is essential to drive strategic insights.

Qualifications

• 6–10 years of experience in Data Engineering.
• Strong expertise in Databricks Platform and Apache Spark.
• Hands-on experience with AWS cloud services.
• Strong programming skills in Python or Scala.
• Experience building robust ETL processes and data pipelines.
• Experience with Databricks Unity Catalog.
• Understanding of distributed computing and big data architecture.
• Familiarity with Git and version control practices.

Responsibilities

• Design, develop, and optimize ETL pipelines using Databricks and Apache Spark.
• Build scalable data models and schemas.
• Collaborate with data scientists to support ML model deployment.
• Ensure data quality, consistency, reliability, and governance.
• Optimize SQL queries and troubleshoot performance bottlenecks.
• Implement data security, access controls, and governance policies.
• Work with Databricks Unity Catalog for centralized governance.
• Develop and maintain technical documentation.
• Participate in code reviews and mentor junior engineers.
• Translate business requirements into scalable technical solutions.
• Support real-time and streaming data processing use cases.

Skills

AWS

Databricks

Apache Spark

Python

Scala

ETL pipelines

Data governance

Git

Streaming data

Tools

Databricks Unity Catalog

Git

Job description

Talpro is leading the way in transforming the talent acquisition landscape. We deliver innovative, sustainable, and cost-effective recruitment solutions tailored to today’s business needs. Our mission is to offer comprehensive hiring strategies that address immediate recruitment demands while laying the groundwork for long-term success.

Job Description

Experience: 6–10 Years

Work Mode: Hybrid (2–3 days/week from office mandatory)

Mandatory Skills: AWS + Databricks

Budget: 17 LPA

Joining: Immediate / Early joiners preferred

Contract Duration: 12 months

Role Overview

We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in AWS and Databricks to design, implement, and optimize scalable data pipelines and analytics solutions. The ideal candidate should have deep expertise in Apache Spark, ETL design, cloud-native data engineering , and modern data governance practices.

You will collaborate closely with data scientists, analysts, and business stakeholders to deliver robust, high-performance data solutions that drive strategic insights.

Key Responsibilities

• Design, develop, and optimize ETL pipelines using Databricks and Apache Spark .
• Build scalable and efficient data models and schemas .
• Collaborate with data scientists to support ML model deployment.
• Ensure data quality, consistency, reliability, and governance .
• Optimize SQL queries and troubleshoot performance bottlenecks.
• Implement data security, access controls, and governance policies .
• Work with Databricks Unity Catalog for centralized governance.
• Develop and maintain technical documentation.
• Participate in code reviews and mentor junior engineers.
• Translate business requirements into scalable technical solutions.
• Support real-time and streaming data processing use cases.

Required Skills & Qualifications

Technical Skills

• 6–10 years of experience in Data Engineering.
• Strong expertise in Databricks Platform and Apache Spark .
• Hands-on experience with AWS cloud services .
• Strong programming skills in Python or Scala .
• Experience building robust ETL processes and data pipelines .
• Experience with Databricks Unity Catalog .
• Understanding of distributed computing and big data architecture.
• Familiarity with Git and version control practices.
• Experience with real-time/streaming data processing.
• Strong understanding of data warehousing and analytics concepts.

Nice to Have

• Databricks certifications.
• Experience with data governance and compliance standards.
• Exposure to Azure or GCP (secondary cloud platforms).
• Experience in fast-paced enterprise environments.

Soft Skills

• Strong analytical and problem-solving ability.
• Excellent communication and stakeholder collaboration skills.
• Ability to manage multiple priorities in a dynamic environment.
• Mentoring and team collaboration mindset.