Sr. Data Engineer

Numerator; Vadodara; On-site; INR 2,000,000 - 3,500,000; Full time

Job summary

Numerator is seeking a Senior Data Engineer to design, build, and optimize high-volume data pipelines using Python, Databricks on Azure, and PySpark. You will work with Agile teams to deliver robust data solutions and contribute to AI/ML initiatives while aligning with Numerator standards.

The role requires 7–10 years in data engineering, strong Azure and Spark experience, and a solid DevOps background. Join a team driving market intelligence for top brands and retailers.

Qualifications

• 7–10 years of experience in data engineering roles.
• Good Python skills.
• Experience working with Microsoft Azure Cloud.
• Experience in Apache Spark, PySpark, Databricks.
• Experience working with DevOps pipeline, preferably Azure DevOps.

Responsibilities

• Adhere to coding and Numerator technology standards
• Build suitable automation test suites within Azure DevOps
• Maintain and update automation test suites as required
• Carry out manual testing, load testing, exploratory testing as required
• Work closely with Business Analysts and Senior Data Developers to consistently achieve sprint goals
• Assist in estimation of sprint-by-sprint stories and tasks
• Pro-actively take a responsible approach to product delivery

Skills

Python

Databricks

Azure

Apache Spark

PySpark

CI/CD

DevOps

Agile

Education

Bachelor's or Master's degree in CS/IT/Data Science

Tools

Azure DevOps

Job description

We’re reinventing the market research industry. Let’s reinvent it together.

At Numerator, we believe tomorrow’s success starts with today’s market intelligence. We empower the world’s leading brands and retailers with unmatched insights into consumer behavior and the influencers that drive it.

We are seeking a highly skilled Senior Data Engineer with extensive experience in designing, building, and optimizing high-volume data pipelines. The ideal candidate will have strong expertise in Python, Databricks on Azure Cloud services, DevOps, and CI/CD tools, along with a solid understanding of AI/ML techniques and big data processing frameworks like Apache Spark and PySpark.

Responsibilities

• Adhere to coding and Numerator technology standards


• Build suitable automation test suites within Azure DevOps


• Maintain and update automation test suites as required


• Carry out manual testing, load testing, exploratory testing as required


• Work closely with Business Analysts and Senior Data Developers to consistently achieve sprint goals


• Assist in estimation of sprint-by-sprint stories and tasks


• Pro-actively take a responsible approach to product delivery

What You’ll Bring to Numerator

Requirements

• 7-10 years of experience in data engineering roles


• Good Python skills


• Experience working with Microsoft Azure Cloud


• Experience in Agile methodologies (Scrum/Kanban)


• Experience with Apache Spark, PySpark, Databricks


• Experience working with Devops pipeline, preferably Azure DevOps

Preferred Qualifications

• Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field.


• Experience working in a support focused role


• Knowledge/experience in AI ML techniques


• Certification in relevant Data Engineering discipline or related fields.