Data Engineer

Mahindra and Mahindra Limited [Automotive and Farm Equipment Business]; Chengalpattu; Presencial; INR 2.500.000 - 4.000.000; Jornada completa

Descripción de la vacante

Mahindra and Mahindra Limited (Automotive and Farm Equipment) seeks an experienced Staff / Data Engineer in Tamil Nadu to design, build and validate end-to-end data solutions. You will work with business users and engineering teams to translate requirements into scalable pipelines using PySpark, BigQuery, Airflow, and modern data platforms.

This on-site role emphasizes data governance, quality, and collaboration, with ownership of data platforms and processes.

Formación

• Bachelor's or master's degree in computer science, engineering, data science, or related field.
• 5+ years of professional experience as a Data Engineer with complex data solutions.
• Expert proficiency in Advanced SQL and Big Data built-in functions.
• Strong Python skills for data engineering tasks (PySpark) and DAG development.
• Cloud experience with GCP, AWS, or Azure; Agile project experience with JIRA.

Responsabilidades

• Design, develop, and implement scalable data pipelines for batch and real-time processing using PySpark and Big SQL.
• Orchestrate data workflows with Apache Airflow and manage batch jobs with Control-M.
• Model data and build dimensional models for automotive domain requirements.
• Create CI/CD pipelines using Tekton, GitHub Actions, and Terraform.
• Enforce data governance, quality, security, and compliance across platforms.
• Collaborate with product managers and plant engineering teams to translate requirements into solutions.
• Automate ETL/ELT processes with reusable components and ensure observability of batch processes.

Conocimientos

PySpark

SQL

BigQuery

Databricks

ETL/ELT

Airflow

GCP

Terraform

Jira

Data Modeling

Data Warehousing

GitHub

Educación

Bachelor's or Master's in Computer Science or related field

5+ years Data Engineer experience

Herramientas

GitHub

Jira

Tekton

Databricks

Apache Airflow

Descripción del empleo

Category: Software Development / Engineering

Employment Type: Full Time

Educational Qualification: Bachelor's degree in Computer Science or a related field, or higher, with a minimum of 6 years of relevant experience.

Position Description

We are seeking an experienced and skilled Staff / Data Engineer to join our growing data team. In this pivotal role, you will be instrumental in designing, building, and validating end-to-end data solutions while driving data democratization across the organization. You will work closely with business users and engineering teams to understand requirements and translate them into scalable technical solutions.

Knowledge of data modeling and data standards is desirable as a secondary skill. The ideal candidate should be able to understand data structures, business definitions, and standardization requirements to support consistent, reliable, and high-quality data processing. Additionally, you will be expected to take ownership of data platforms and processes, ensuring that data is accessible, trustworthy, and effectively utilized by end users.

Key Responsibilities

• Design & Development: Independently design, develop, and implement scalable, high-performance data pipelines for batch and real-time processing using Big SQL and PySpark. Orchestrate and manage data workflows, tasks, and batch jobs using Apache Airflow and Control-M
• Data Modeling: Should be able to visualize, build, and stitch physical dimensional models based on functional requirements within the automotive industry.
• Infrastructure Management: Should have advanced knowledge of creating CI/CD pipelines using Tekton,Github Actions and Terraform.
• Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and queries by using query optimization techniques, splitting large processes into smaller chunks without impacting functionality.
• Data Governance & Quality: Implement and enforce data governance best practices, ensuring data quality, integrity, security, and compliance.
• Collaboration: Work closely with product managers and plant engineering teams to understand data requirements and translate them into technical solutions.
• Automation: Automate data extraction, transformation, and loading (ETL/ELT) processes by building reusable components.
• Observability : Should have experience in production batch process observability and the ability to articulate incidents and work on their permanent resolution.

Required Qualifications

• Bachelor's or master's degree in computer science, Engineering, Data Science, or a related field.
• 5+ years of professional experience as a Data Engineer, with a strong track record of designing and implementing complex data solutions.
• Expert proficiency in Advanced SQL and Big Data built-in functions.
• Strong programming skills in Python for data engineering tasks, including PySpark and DAG development.
• Extensive experience with cloud-based data platforms such as:
• GCP: BigQuery, Dataflow, Dataproc, Dataform, Cloud Storage
• Must have Agile project experience and hands-on experience with the JIRA tool.
• Hands-on experience in building data warehouses using DBT, BigQuery, and Databricks.
• Solid understanding of ETL/ELT processes and tools.
• Experience with version control systems (Github).
• Excellent problem-solving skills and the ability to troubleshoot complex data issues.
• Strong communication and interpersonal skills to collaborate effectively with cross-functional teams.

Key Skills

• Python | PySpark | SQL | BigQuery | Databricks | DBT | AWS | Azure | GCP | Data Modeling | ETL/ELT | Data Warehousing | Terraform | Jenkins | Tekton | GitHub Actions | Data Governance | CI/CD | Real-Time Data Processing | Cloud Data Engineering | Airflow | Astro