Data Engineering Lead Bangalore, India

Metyis; Bengaluru; On-site; INR 4,200,000 - 6,000,000; Full time

Job summary

Metyis in Bengaluru seeks a skilled Data Engineer with 7-10 years of experience to lead projects and drive impactful business change. You will independently manage data engineering tasks, engineering solutions for digital marketing and analytics. The role demands proficiency in Python, SQL, and distributed programming, with exposure to cloud environments being advantageous.

Join a diverse and fast-growing team where your expertise will shape big data strategies for multiple clients, ensuring effective collaboration and mentorship while promoting a culture of innovation.

Qualifications

• 7-10 years of professional hands-on experience.
• Broad practice in multiple software engineering fields.
• Experience managing and leading data engineering projects.
• Experience setting up data lakes in the Big Data ecosystem.
• Good communication skills in English.
• Experience setting up data lakes using the full Big Data and DWH ecosystem.
• Experience with ETL and data workflows; Apache Airflow is a plus.
• Comfortable with Unix/Linux OS, Bash and Linux.
• Basic knowledge of web services and APIs.
• Basic knowledge of containers and Docker as a platform.
• Experience with version control, testing and development cycles; Git (GitLab or GitHub) preferred.
• Knowledge of security practices.
• Strong focus on data processing.
• Object-oriented programming skills.
• Understanding of data workflows, monitoring, architecture and security.
• Knowledge of databases: theory and practice.
• Experience in distributed data processing.
• Familiarity with functional programming concepts.
• Experience in enterprise data warehousing, business intelligence and ETL principles.
• Good written and oral communication skills in English.

Responsibilities

• Independently lead and manage data engineering projects.
• Engineer technical solutions for business challenges.
• Support clients in executing Big Data strategies.
• Troubleshoot and quality-check work by team members.
• Guide and mentor team members.
• Collaborate with partners, strategy consultants and data scientists in a flat, agile org.
• Share data engineering knowledge via trainings.
• Communicate with clients at executive level.
• Guide and mentor team members.

Skills

Python

SQL

Distributed programming (especially Spark)

Data workflows and ETL

Unix/Linux OS

Bash

Web services and APIs

Containers and Docker

Version control (preferably Git)

Security practices

Object-oriented programming skills

Data architectures

Enterprise data warehousing principles

English

Tools

GitHub

GitLab

Docker

Apache Airflow

Job description

What we offer

• Interact with C‑level at our clients on a regular basis to drive their business towards impactful change.
• Lead your team in creating new business solutions.
• Seize opportunities at the client and at Metyis in our entrepreneurial environment.
• Become part of a fast‑growing international and diverse team.

What you will do

• Independently lead and manage the execution of data engineering projects.
• Engineer complete technical solutions to solve concrete business challenges in the areas of digital marketing, eCommerce, Business Intelligence and self‑service analytics.
• Collect functional and non‑functional requirements, considering technical environments, business constraints and enterprise organizations.
• Support our clients in executing their Big Data strategies by designing and building operational data platforms: ETL pipelines, data anonymization pipelines, data lakes, near real‑time streaming data hubs, web services, and training and scoring machine learning models.
• Troubleshoot and quality‑check work done by team members.
• Collaborate closely with partners, strategy consultants and data scientists in a flat and agile organization where personal initiative is highly valued.
• Share data engineering knowledge by giving technical trainings.
• Communicate and interact with clients at the executive level.
• Guide and mentor team members.

What you will bring

• 7‑10 years of professional hands‑on experience.
• A broad practice in multiple software engineering fields.
• Experience managing and leading data engineering/warehousing projects.
• Proficiency in Python, SQL and distributed programming languages, preferably Spark.
• Experience working with Cloud, especially Azure, is a plus.
• Experience setting up data lakes using the full Big Data and DWH ecosystem.
• Experience with data workflows and ETL; Apache Airflow is a plus.
• Comfortable with Unix/Linux OS, Bash and Linux.
• Basic knowledge of web services and APIs.
• Basic knowledge of containers and Docker as a platform.
• Experience with version control, testing and development cycles; preferably Git (Gitlab or GitHub).
• Knowledge of security practices.
• Strong focus on data processing.
• Object‑oriented programming skills.
• Understanding of data workflows, monitoring, architecture and security.
• Knowledge of databases: theory and practice.
• Experience in distributed data processing.
• Familiarity with functional programming concepts.
• Experience in enterprise data warehousing, business intelligence and ETL principles.
• Good written and oral communication skills in English.

Metyis is an equal‑opportunity employer. We value diversity, equity, inclusion, and belonging, and we encourage all qualified candidates to apply.