Senior Data Engineer

Affine Analytics; Bengaluru, Hyderabad; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Affine Analytics is looking for a seasoned data engineering professional based in Bengaluru with 9+ years of experience in Data & Analytics. This role will involve leading a data engineering team, designing scalable solutions, and collaborating with various stakeholders to enhance AI-driven decision-making.

The ideal candidate must have strong hands-on experience with Python, SQL, PySpark, and Azure technologies, as well as a proven track record of delivering effective data solutions.

Qualifications

• 9+ years of experience in Data & Analytics, including hands-on experience with Azure data engineering technologies.
• Strong understanding of data governance and security.
• Proven ability to gather requirements and translate them into high-impact data solutions.

Responsibilities

• Lead and mentor the data engineering team in driving delivery excellence.
• Collaborate with Data Architects, Data Scientists, and Product Owners.
• Design scalable data ingestion frameworks for batch and streaming.

Skills

Python

SQL

PySpark

Azure services/tools

Databricks Unity Catalog

Job description

Responsibilities

• Partner closely with business to understand problems, shape requirements, and define data solutions with clear outcomes.
• Lead and mentor the data engineering team, driving engineering best practices and delivery excellence.
• Provide technical leadership across architecture, development, troubleshooting, and delivery to drive execution speed and quality.
• Collaborate with Data Architects, Data Scientists, and Product Owners to enable business adoption of AI and advanced analytics‑driven decision making.
• Drive innovation and continuous improvement by challenging existing processes and proposing new solutions.
• Maintain and evolve the data platform for business reporting and AI workloads.
• Partner with Data Product Owners to define clear technical designs and execution plans aligned with business outcomes.
• Design and maintain scalable data ingestion frameworks supporting both batch and streaming use cases.

Experience & Skills

• 9+ years of experience in Data & Analytics, including hands‑on experience with Azure data engineering technologies.
• Strong hands‑on experience with Python, SQL, and PySpark.
• Strong understanding of Databricks Unity Catalog for data governance and security, along with CI/CD using DevOps.
• Experience in designing scalable, secure data architectures and building AI‑ready data platforms.
• Proven ability to work with business and technical stakeholders to gather requirements and translate them into high‑impact data solutions.
• Strong sense of ownership, accountability, and execution discipline across multiple initiatives.
• Excellent communication skills, with the ability to handle high‑impact business stakeholder interactions effectively.

SKILLS AND EXPERTISE

Mandatory Skillset

• Recent 4+ years of experience in Azure services/tools
• Python
• SQL
• PySpark
• Databricks Unity Catalog for data governance and security
• Strong sense of ownership, accountability, and execution discipline across multiple initiatives.