Senior Product Data Engineer

emagine; Maharashtra; Hybrid; INR 3,500,000 - 5,500,000; Full time

Job summary

emagine is seeking a Senior Product Software Data Engineer to design, develop, and troubleshoot data solutions. You will work with cross-functional teams to translate business needs into scalable pipelines, real-time and batch processing, and data models.

You will use PySpark, Spark, Azure Databricks, Azure Data Factory, and Azure Synapse to deliver high-quality data products, ensuring governance and data quality while mentoring junior engineers.

Qualifications

• 7+ years of experience in data engineering or related fields.
• 5+ years building data pipelines for structured and unstructured data.
• 2+ years with PySpark, Hadoop, Kafka, EventHub, and Stream Analytics.
• 3+ years in relational and dimensional data modeling using SQL Server.
• 3+ years with T-SQL, stored procedures, and performance tuning.
• 3+ years with Azure Data Factory, Azure Data Lake, Azure SQL DB, and Azure Synapse Analytics.
• Nice to have 2+ years with Azure ML, Python and Scala.
• Nice to have 2+ years with Power BI and Azure Analysis Services.

Responsibilities

• Collaborate with cross-functional teams to translate business requirements into scalable data solutions.
• Design and develop enterprise-level data pipelines for real-time and batch processing.
• Analyze business processes, identify gaps, and recommend data-driven improvements.
• Build real-time data solutions using Spark, Azure Databricks.
• Partner with data scientists to operationalize ML models and ensure governance.
• Champion data governance, metadata management, and data quality.
• Adhere to coding standards, review code, and contribute to documentation.
• Mentor junior engineers and share knowledge across the team.

Skills

Data engineering

Data pipelines

SQL

Python

Scala

Azure

Power BI

Azure Databricks

Spark

Hadoop

Kafka

EventHub

Azure Synapse

Machine Learning

Git

Education

Bachelor’s degree in Computer Science or related field

Tools

SQL Server

Azure Data Factory

Azure Data Lake

Azure SQL DB

Azure Analysis Services

Azure ML

Azure

PySpark

Job description

About the Role

Take your software engineering career to the next level as a Senior Product Software Data Engineer. You will play a key role in designing, developing, and troubleshooting software programs. Your technical skills and innovative mindset will contribute to the continuous improvement of our products, making a significant impact on user experience and satisfaction.

Responsibilities

• Collaborate with cross-functional teams to review use cases and translate business requirements into scalable data solutions.
• Design and develop highly optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing.
• Analyze existing business processes, identify gaps, and recommend data-driven improvements.
• Build modern real-time data solutions using Scala, Apache Spark, and Azure Databricks.
• Partner with data scientists and business stakeholders to operationalize machine learning models.
• Champion data governance, metadata management, and operational excellence across data platforms.
• Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms.
• Adhere to coding standards, participate in peer code reviews, and contribute to technical documentation.
• Mentor junior engineers and contribute to knowledge sharing across the team.

Key Requirements

• 7+ years of overall experience in data engineering or related fields.
• 5+ years of experience building data pipelines for structured and unstructured data.
• 2+ years of experience with big data technologies including PySpark, Hadoop, Kafka, EventHub, and Stream Analytics.
• 3+ years of experience in relational and dimensional data modeling using SQL Server and data warehousing concepts.
• 3+ years of experience with T-SQL, stored procedures, and performance tuning.
• 3+ years of experience with Azure Data Factory, Azure Data Lake, Azure SQL DB, and Azure Synapse Analytics.
• Good to have 2+ years of experience with Azure ML, Python, and Scala.
• Good to have 2+ years of experience with Power BI and Azure Analysis Services.
• Experience working with business stakeholders for requirements gathering and use case analysis.
• Strong communication, collaboration, and creative problem-solving skills.

Nice to Have

• Bachelor’s degree in computer science, Engineering, or equivalent practical experience.
• Experience with Agile/Scrum methodologies and DevOps practices.
• Familiarity with CI/CD pipelines and version control systems (e.g., Git).
• Azure Data Engineer Certification or equivalent is a strong advantage.

Other Details

This role is focused on delivering impactful data solutions that enhance product performance and requires a collaborative approach within a team environment. Flexibility in remote work options is available, as well as opportunities for professional growth.