Data Engineer

ZF Group; Thiruvallur District; On-site; INR 1,800,000 - 2,800,000; Full time

Job summary

ZF Group is seeking a Data Engineer to join the central R&D data platform in India. The role focuses on building and maintaining scalable data pipelines and Spark-based processing, with governance for data assets and metadata catalogs.

You will work with AWS/Azure cloud, Hadoop ecosystem, and containerized environments to deliver reliable data solutions for vehicle intelligence initiatives across OEMs.

Qualifications

• Bachelor’s or Master’s degree in computer science or a related field with at least 6+ years of hands-on embedded software development.
• Proficiency: Spark, Scala, Airflow, Hadoop, Impala, Sqoop, AWS (EC2, EMR), Terraform, SQL and HiveQL.
• Tools fluency: JIRA, Confluence, GitLab, IntelliJ, DB Visualizer, Docker, Slack, Opsgenie.
• Experience with Beamer/GIT and Agile/Lean practices.
• Cloud familiarity with AWS/Azure and virtualization/docker/containerization.

Responsibilities

• Build and maintain optimized, highly available data pipelines and Spark (Scala) jobs.
• Contribute to design, deployment, and maintenance of the business data platform.
• Extend data pipelines for large datasets; collect, store, transform and catalog data.
• Develop ad hoc queries and ongoing reporting; build a metadata system for data cataloging.
• Retrieve and analyze data using Spark SQL, Athena, Excel and other data tools; create data loading services from multiple sources.

Skills

Spark Scala

Airflow

Hadoop

Impala

Sqoop

SQL

HiveQL

Beamer/GIT

Agile/Lean

AWS

Azure

Docker

Education

Bachelor’s or Master’s degree in CS or related field

Tools

JIRA

Confluence

GitLab

IntelliJ

DB Visualizer

Slack

Opsgenie

Job description

Job Description:

Job Description

About the team

We are a motivated team in central R&D at CVS helping to change the game through product digitalization and vehicle intelligence. Our focus is on building solutions for truck, bus and trailer OEMs considering both onboard and offboard needs and requirements.

What you can look forward to as Data Engineer:

• Responsible for building and maintaining optimized and highly available data pipelines, Cloud infrastructure optimization and Spark (Scala) jobs.
• Provide technical contributions to a team that is responsible for the design, deployment and maintenance of the business data platform.
• Extend the data pipeline through the collection, storage, processing and transformation of large datasets.
• Additionally, queries for ad hoc business projects developed, as well as ongoing reporting. In this capacity, build a metadata system where all available data is maintained and cataloged.
• Involved in the retrieval and analysis of data using Spark SQL, Athena, Excel, among other data management systems. Built data loading services for the purpose of importing data from numerous data sources, inclusive of logs, relational and non-relational databases.

Your Profile as Data Engineer:

• Bachelor’s or master’s degree in computer science or a related field with at least 6+ yearsof hands-on embedded software development.
• Apache Spark - Scala, Airflow, Hadoop, Impala, Sqoop, AWS (EC2, EMR), Terraform, SQL and Hive QL
• Tools - JIRA, Confluence, GitLab, IntelliJ, DB Visualizer, Docker, Slack, Opsgenie
• Experience in using Beamer/GIT and following Agile methodologies and Lean principles.
• Knowledge of cloud systems like AWS/AZURE and virtualization techniques like docker, containerization.

Why should you choose ZF Group in India?
Innovative Environment: ZF is at the forefront of technological advancements, offering a dynamic and innovative work environment that encourages creativity and growth.
Career Development: ZF is committed to the professional growth of its employees, offering extensive training programs, career development opportunities, and a clear path for advancement.
Global Presence:As a part of a global leader in driveline and chassis technology, ZF provides opportunities to work on international projects and collaborate with teams worldwide.
Sustainability Focus:ZF is dedicated to sustainability and environmental responsibility, actively working towards creating eco-friendly solutions and reducing its carbon footprint.
Employee Well-being:ZF prioritizes the well-being of its employees, providing comprehensive health and wellness programs, flexible work arrangements, and a supportive work-life balance.