Data Engineering Lead

Metyis; Gurugram District; On-site; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Interact with C-level at clients

Lead your team in creating new data- &

Fast-growing international team

Job summary

Metyis is seeking a Senior Data Engineer to lead end-to-end data engineering initiatives from Gurgaon, India. You will design and implement data pipelines, data lakes and near real-time hubs, aligning with business goals and data governance.

You will collaborate with analytics, marketing and strategy teams, mentor juniors and drive best practices in SQL, Python, Spark and cloud-native data architectures.

Qualifications

• 7–10 years of professional hands-on experience.
• Experience managing data engineering / warehousing projects.
• Must have experience in Python, SQL and distributed programming languages, preferably Spark.
• Experience with Cloud, Azure is a plus.
• Experience building data lakes and Big Data/DWH ecosystems.
• Experience with data workflows and ETL; Apache Airflow is a plus.
• Comfortable with Unix/Linux and Bash.
• Knowledge of web services and APIs.
• Version control and testing within development cycles; Git preferred.
• Strong written and spoken English communication.

Responsibilities

• Independently lead & manage execution of data engineering projects.
• Engineer complete technical solutions for digital marketing, eCommerce and BI.
• Collect requirements considering environments and constraints.
• Design and build operational data platforms: ETL pipelines, data lakes, streaming hubs, APIs.
• Troubleshoot and ensure quality across team deliverables.
• Collaborate with partners, strategy consultants and data scientists in an agile setup.
• Share data engineering knowledge via trainings.
• Communicate with clients at executive level and mentor team members.

Skills

Python

SQL

Distributed programming

Spark

Unix/Linux

APIs

Data processing

Git

Data workflows

Big Data

Tools

Apache Airflow

Docker

Azure Cloud

GitLab/GitHub

Job description

The next step of your career starts here, where you can bring your own unique mix of skills and perspectives to a fast-growing team.

Metyis is a global and forward-thinking firm operating across a wide range of industries, developing and delivering AI & Data, Digital Commerce, Marketing & Design solutions and Advisory services. At Metyis, our long-term partnership model brings long-lasting impact and growth to our business partners and clients through extensive execution capabilities.

With our team, you can experience a collaborative environment with highly skilled multidisciplinary experts, where everyone has room to build bigger and bolder ideas. Being part of Metyis means you can speak your mind and be creative with your knowledge. Imagine the things you can achieve with a team that encourages you to be the best version of yourself.

We are Metyis. Partners for Impact.

What We Offer

• Interact with C-level at our clients on regular basis to drive their business towards impactful change.
• Lead your team in creating new business solutions.
• Seize opportunities at the client and at Metyis in our entrepreneurial environment.
• Become part of a fast-growing international and diverse team.

What You Will Do

• Independently lead & manage execution of data engineering projects
• Engineer complete technical solutions to solve concrete business challenges in the areas of digital marketing, eCommerce, Business Intelligence and self-service analytics
• Collect functional and non-functional requirements, consider technical environments, business constraints and enterprise organizations
• Support our clients in executing their Big Data strategies by designing and building operational data platforms: ETL pipelines, data anonymization pipelines, data lakes, near real-time streaming data hubs, web services, training and scoring machine learning models.
• Troubleshoot and quality check work done by team members
• Collaborate closely with partners, strategy consultants and data scientists in a flat and agile organization where personal initiative is highly valued
• Share data engineering knowledge by giving technical trainings
• Communicate and interact with clients at the executive level
• Guide and mentor team members

What You Will Bring

• 7-10 years of professional hands-on experience
• A broad practice in multiple software engineering fields
• Experience on managing & leading data engineering / warehousing projects
• Must have experience on Python, SQL and Distributed programming languages, preferable Spark
• Experience working on Cloud, Azure is a plus
• Experience working setting up data lakes using entire Big Data and DWH ecosystem
• Experience on data workflows and ETL, Apache Airflow is a plus
• Comfortable with Unix OS type systems, Bash and Linux
• Basic knowledge of webservices and APIs
• Basic knowledge of Containers and Docker as platform
• Experience using Version Control, Testing and development cycle, preferable git (Gitlab or GitHub)
• Knowledge of security practices
• A Strong Focus On Data Processing
• Object oriented programming
• Data workflows, monitoring, architecture and security
• Databases: theory and practice
• Distributed data processing
• Concepts of functional programming
• Enterprise data warehousing, business intelligence and ETL principles
• Good written and oral communication skills in English language

Metyis, we are driven by curiosity and collaboration. We value diversity, equity, inclusion, and belonging (DEIB) in all its forms as it makes us stronger as an organisation and promotes creativity and innovation. We welcome all talents and are committed to creating a workplace where every employee can make a meaningful impact and grow.