Deputy Manager-Data Engineer, Big Data and Machine Learning Pipelines

VOIS; Maharashtra; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

VOIS (Vodafone Intelligent Solutions) in Pune is seeking a Data Engineer to design, automate and optimise end-to-end data pipelines for analytics platforms. You will transform large datasets into reliable inputs for Data Scientists, working with stakeholders across business and tech teams.

The role focuses on data ingestion, storage, transformation and ML deployment, using Spark, Airflow, Kafka and NoSQL databases to deliver scalable data solutions.

Qualifications

• Five to seven years of experience with large datasets and distributed computing techniques.
• Experience using Apache Spark to transform data for data science activities.
• Experience with distributed storage technologies (AWS, Google Cloud or HDFS).
• Experience building and scheduling data pipelines with Apache Airflow.
• Ingesting data with technologies like Apache Kafka, Sqoop, Flume or Apache NiFi.
• Real-time analytics with Spark, Flink and Kafka.
• Programming in Scala, Python, Java or R.
• Experience with NoSQL databases such as Cassandra, MongoDB, HBase or Redis.
• Strong communication with stakeholders.

Responsibilities

• Understand data science challenges and design, build and schedule end-to-end data pipelines.
• Select appropriate big data technologies to solve technical and business problems efficiently.
• Automate data science pipelines, deploy machine learning algorithms and monitor their performance.
• Build Customer 360 data solutions and feature stores for ML use cases.
• Develop data models for feature stores using high-velocity databases with flexible schemas.
• Identify data sources and integrate them into a big data lake.
• Translate business challenges into analytics use cases, data structures and models.
• Collaborate with stakeholders to define scope, deliverables, processes and outcomes.
• Provide technical guidance on data architecture, models and metadata management.
• Partner with database teams to ensure data quality and usability.
• Create and modify programs to extract information from databases.
• Work with large datasets and distributed computing, simulation and optimisation tools.
• Diagnose and resolve issues across big data platforms and frameworks.
• Communicate technical concepts clearly to business leads and stakeholders.

Skills

Python

Scala

Java

R

Large datasets

Distributed computing

Spark

Airflow

Kafka

Flink

NoSQL

Tools

Apache Spark

Apache Airflow

Apache Kafka

Apache NiFi

Apache Flink

HDFS

MongoDB

Job description

Who We Are

VOIS (Vodafone Intelligent Solutions) is a strategic arm of Vodafone Group Plc, creating value for customers by delivering intelligent solutions through Talent, Technology & Transformation.

As the largest shared services organisation in the global telco industry with 30,000 FTE, our portfolio of next-generation solutions and services are designed in partnership with customers across Vodafone Group, local markets, and partner markets to simplify and drive growth. With our strategic partner Accenture, we work alongside our Vodafone customers, other Telco and tech companies to drive transformation, meet the challenges of our industry and ensure we stay relevant and resilient. This partnership is a unique, industry-first model which brings together the best of in-house and 3rd party capability.

We work with customers across 28 countries from 10 VOIS locations: Albania, Egypt, Hungary, India, Romania, Spain, Turkey, UK, Germany, Ireland, and with a network of teams in Czech Republic, Italy, Greece, and Portugal.

#VOIS #BeUnrivalled #CreateTheFuture

About This Role

We are seeking a Data Engineer to design, automate and optimise end-to-end data science and big data pipelines. Based in Pune and reporting to a Senior Manager, you will define data lifecycles, models and sources for analytics platforms, transforming large and complex datasets into reliable, ready-to-use inputs for Data Scientists.

You will work with business stakeholders, Data Scientists, senior technologists and database teams to understand business challenges, agree project scope and outcomes, and develop scalable data solutions. The role includes data ingestion, storage, transformation and optimisation, as well as the deployment and performance monitoring of machine learning algorithms.

What You’ll Do

• Understand data science challenges and design, build and schedule end-to-end data pipelines.


• Select appropriate big data technologies to solve technical and business problems efficiently.


• Automate data science pipelines, deploy machine learning algorithms and monitor their performance.


• Build Customer 360 solutions and feature stores for a range of machine learning use cases.


• Develop data models for machine learning feature stores using high-velocity databases with flexible schemas.


• Identify data sources across the organisation and integrate them into a big data lake.


• Translate business challenges into end-to-end analytics use cases, data structures and data model requirements.


• Collaborate with business stakeholders to agree scope, deliverables, processes and expected outcomes.


• Provide technical guidance to senior technologists and business leaders on data architecture, data models and metadata management.


• Partner with database teams to address data quality, accuracy, integrity and usability requirements.


• Create and modify programs that extract information from organisational databases.


• Work with large datasets and distributed computing, simulation and optimisation tools.


• Diagnose and resolve issues affecting big data platforms and frameworks.


• Communicate technical concepts and recommendations clearly to business and project stakeholders.

Who You Are

• You have five to seven years of experience working with large datasets, distributed computing tools, simulation or optimisation techniques.


• You have experience using Apache Spark to transform data for data science activities.


• You understand distributed storage technologies, including AWS, Google Cloud Platform or HDFS.


• You have experience building and scheduling data pipelines with Apache Airflow.


• You have worked with data ingestion technologies such as Apache Kafka, Sqoop, Flume or Apache NiFi.


• You can build real-time analytics solutions using technologies such as Apache Spark, Apache Flink and Apache Kafka.


• You have programming experience in one or more of the following languages: Scala, Python, Java or R.


• You have worked with NoSQL databases such as Cassandra, MongoDB, HBase or Redis.


• You can investigate and resolve issues across big data platforms and frameworks.


• You communicate clearly and can present technical information effectively to business project leads and other stakeholders.


• You take a collaborative approach to working with Data Scientists, database specialists, senior technologists and business stakeholders.

Not a perfect fit?

Concerned you may not meet every requirement? Vodafone is committed to creating an inclusive workplace where everyone can thrive. If you are excited about this role but your experience does not align exactly with every aspect of the job description, you are encouraged to apply. You may be the right candidate for this or another opportunity, and the recruitment team will support you in exploring where your skills fit best.

What's In It For You

• The opportunity to develop scalable data products that support analytics and machine learning use cases.


• Exposure to large organisational datasets, distributed computing environments and real-time analytics systems.


• Collaboration with Data Scientists, database teams, senior technologists, business leaders and project stakeholders.


• The opportunity to contribute across the complete data lifecycle, from sourcing and ingestion to transformation, optimisation and operational monitoring.


• Experience addressing varied technical challenges across cloud storage, big data platforms, feature stores and machine learning pipelines.

What Skills You Will Learn

• How to design and optimise end-to-end data and machine learning pipelines at scale.


• How to develop Customer 360 data solutions and reusable feature stores for machine learning.


• How to integrate data from diverse organisational sources into a big data lake.


• How to strengthen real-time data processing capabilities with Spark, Flink and Kafka.


• How to translate business needs into analytics use cases, data architectures and data models.


• How to monitor deployed machine learning algorithms and improve data pipeline performance.


• How to provide clear technical guidance on data architecture, data modelling and metadata management.

VOIS Equal Opportunity Employer Commitment

Vodafone recognises and celebrates the value of diversity in building a workforce that reflects the customers and communities it serves. No form of discrimination is tolerated. This includes, but is not limited to, discrimination based on race, colour, age, veteran status, gender identity, gender expression, sexual orientation, pregnancy, maternity or parental status, ethnicity, disability, religion or belief, political affiliation, trade union membership, nationality, citizenship, indigenous status, medical condition, HIV status, neurodiversity, social origin, cultural background, marital or civil partnership status, or socio-economic background.

Join Us

At Vodafone, we’re working hard to build a better future. A more connected, inclusive and sustainable world. As a dynamic global community, it's our human spirit, together with technology, that empowers us to achieve this.

We challenge and innovate in order to connect people, businesses, and communities across the world. Delighting our customers and earning their loyalty drive us, and we experiment, learn fast and get it done, together.

With us, you can truly be yourself and belong, share inspiration, embrace new opportunities, thrive, and make a real difference.

#JDEnhancedByTARA