Data Engineer

Veraxion; Bengaluru; Presencial; INR 4.200.000 - 6.400.000; Jornada completa

Descripción de la vacante

Veraxion AG in Bengaluru seeks a Senior Data Engineer (Data Engineer III) passionate about building modern data platforms and solving complex engineering challenges at scale. You will own data products end-to-end, from understanding business problems to production-grade pipelines for analytics, AI, and operations.

You will design scalable architectures, build reusable ETL/ELT frameworks, optimise pipelines for performance and cost, and collaborate with stakeholders to translate requirements into

Formación

• 10+ years of experience in Data Engineering for enterprise-scale platforms.
• Expertise in Databricks, Apache Spark (PySpark), Python, and SQL.
• Hands-on experience building cloud-native data platforms on AWS.
• Strong understanding of Lakehouse architecture, ETL/ELT and modern data engineering practices.
• Experience designing scalable data models and large-scale pipelines.
• Strong software engineering fundamentals: Git, CI/CD, testing, and code quality.
• Excellent communication and stakeholder leadership.

Responsabilidades

• Own data products from design through production.
• Design, build, and maintain scalable data pipelines and data products using Databricks, Spark, Python, SQL, and AWS.
• Develop robust batch and streaming pipelines aligned with Lakehouse principles.
• Build reusable ETL/ELT frameworks and self-service data products for analytics, AI/ML, and reporting.
• Optimise pipelines for performance, scalability, reliability, and cost efficiency.
• Implement incremental processing, CDC, and metadata-driven frameworks.
• Partner with business stakeholders to translate requirements into scalable technical solutions.
• Drive end-to-end solution design from concept to production deployment.
• Build production-grade solutions with emphasis on quality, observability, security, and reliability.

Conocimientos

Data engineering expertise

Data pipeline development

Cloud platforms (AWS)

Communication with stakeholders

Herramientas

Databricks

Apache Spark (PySpark)

Python

SQL

AWS

Git

CI/CD

DataOps

Descripción del empleo

We are seeking a Senior Data Engineer (Data Engineer III) who is passionate about building modern data platforms and solving complex engineering challenges at scale. In this role, you will own data products end-to-end, from understanding business problems and designing scalable architectures to building production-grade pipelines that support analytics, AI, and operational excellence.

The candidate will have responsibilities across the following functions:

Data Product Development:

• Own data products from design through production.
• Design, build, and maintain scalable data pipelines and data products using Databricks, Spark, Python, SQL, and AWS.
• Develop robust batch and streaming pipelines aligned with modern Lakehouse architecture principles.
• Build reusable ETL/ELT frameworks, curated datasets, and self-service data products for analytics, AI/ML, and operational reporting.
• Optimise pipelines for performance, scalability, reliability, and cost efficiency.
• Implement incremental processing, CDC, and metadata-driven engineering frameworks.

Solution Design and Architecture:

• Partner with business stakeholders to understand requirements and translate them into scalable technical solutions.
• Design end-to-end data architectures, reusable frameworks, and high-performance data models.
• Evaluate architectural trade-offs and influence technical direction across the data platform.
• Drive solution design from concept through production deployment.

Engineering Excellence:

• Build production-grade solutions with a strong focus on quality, testing, observability, security, and reliability.
• Implement automated data validation, monitoring, lineage, and operational best practices.
• Write clean, maintainable code and contribute to reusable frameworks, CI/CD, DataOps practices, and code reviews.
• Troubleshoot production issues and continuously improve platform performance and developer experience.

Business Partnership:

• Collaborate with business stakeholders, product owners, analysts, architects, and engineers to solve high-impact business problems.
• Translate business requirements into scalable data products and trusted datasets.
• Communicate technical concepts effectively to both technical and non-technical audiences.
• Take end-to-end ownership of solutions from discovery and design to production support.

Requirements:

• 10+ years of experience in Data Engineering, with experience delivering enterprise-scale data platforms.
• Strong expertise in Databricks, Apache Spark (PySpark), Python, and SQL.
• Hands-on experience building cloud-native data platforms on AWS.
• Strong understanding of distributed data processing, Lakehouse architecture, ETL/ELT, and modern data engineering practices.
• Experience designing scalable data models and optimising large-scale data pipelines.
• Strong software engineering fundamentals, including Git, CI/CD, testing, and code quality.
• Excellent communication skills with the ability to influence stakeholders and lead technical discussions.