Process Manager

eClerx; Maharashtra; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

eClerx in Maharashtra, India, seeks a senior Data Engineer to build and optimize data pipelines on Google Cloud Platform, focusing on BigQuery, Dataflow, Airflow and Cloud Composer. You will ingest new data sources, design data models, and drive ETL across the enterprise data platform.

You will write advanced SQL, optimize ETL resources, collaborate with vendors, and explore new big-data technologies. The role requires 7+ years in data engineering, strong Python and SQL skills, and experience

Qualifications

• Bachelor’s degree in data science, computer science or similar majors with GPA 3.0+ or equivalent experience.
• 7+ years of experience in Data Engineering.
• 7+ years of experience in at least one OOP language, preferably Python.
• Deep SQL knowledge / experience.
• Experience in complex pipeline task management (Airflow and Beam).
• Experience building Data Models.
• Experience with GitHub.

Responsibilities

• Build, maintain and optimize data pipelines for our Enterprise Data Platform using Google Cloud Platform technologies.
• Ingest new data sources from discovery through ETL authoring, operationalizing using Dataflow and Airflow (Cloud Composer) DAGs.
• Research and test new big-data technologies and tools.
• Engage vendors to meet business needs and optimize ETL resources to reduce cost and increase performance.

Skills

Python programming

Advanced SQL

Data modeling

Airflow/Beam

Vendor engagement

GitHub

Education

Bachelor’s degree in data science, computer science or similar majors

Tools

Google Cloud Platform (GCP)

BigQuery

Dataflow

Cloud Composer

Job description

• Build, maintain and optimize data pipelines for our Enterprise Data Platform using Google Cloud Platform technologies.
• Ingest new data sources from initial discovery & data architecture, through ETL authoring, operationalizing using Dataflow, Airflow (Cloud Composer) DAGs, and post launch lifecycle.
• Research and test new big-data technologies and tools.
• Advanced SQL queries, and modeling.
• Engage vendors with required features to meet business needs.
• Leverage the full value of our vendor integrations and APIs.
• Optimize BigQuery and ETL resources to decrease spend & increase performance.

Job Description

• Build, maintain and optimize data pipelines for our Enterprise Data Platform using Google Cloud Platform technologies.
• Ingest new data sources from initial discovery & data architecture, through ETL authoring, operationalizing using Dataflow, Airflow (Cloud Composer) DAGs, and post launch lifecycle.
• Research and test new big-data technologies and tools.
• Advanced SQL queries, and modeling.
• Engage vendors with required features to meet business needs.
• Leverage the full value of our vendor integrations and APIs.
• Optimize BigQuery and ETL resources to decrease spend & increase performance.

Minimum Qualifications

• Bachelor’s degree in data science, computer science or similar majors with a GPA of at least 3.0 or equivalent experience
• 7+ years of experience in Data Engineering
• 7+ years of experience in at least one OOP language, preferably Python
• Deep SQL knowledge / experience
• Experience in complex pipeline task management (i.e., Airflow and Beam)
• Experience building Data Models
• Experience with Github

Desired Qualifications

• Google Cloud Platform tools or equivalent platform experience using tools s/a BigQuery, Dataflow / Apache Beam, Cloud Composer / Airflow, Pub/Sub, Apache Spark
• Experience using DBT
• Experience with Scalr/Terraform
• Experience with CI/CD
• Experience with marketing technologies like a CDP (Customer Data Platform) and SFMC (Saleforce Marketing Cloud)

About The Team

eClerx is a global leader in productized services, bringing together people, technology and domain expertise to amplify business results. Our mission is to set the benchmark for client service and success in our industry. Our vision is to be the innovation partner of choice for technology, data analytics and process management services. Since our inception in 2000, we've partnered with top companies across various industries, including financial services, telecommunications, retail, and high-tech. Our innovative solutions and domain expertise help businesses optimize operations, improve efficiency, and drive growth. With over 18,000 employees worldwide, eClerx is dedicated to delivering excellence through smart automation and data-driven insights. At eClerx, we believe in nurturing talent and providing hands-on experience.

eClerx is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability or protected veteran status, or any other legally protected basis, per applicable law.