Lead Data Engineer

Darwinbox Digital Solutions Pvt. Ltd.; Pune District; On-site; INR 2,400,000 - 4,200,000; Full time

Job summary

Darwinbox Digital Solutions Pvt. Ltd. in Pune is seeking lead data engineers skilled in SQL, Python, ETL, DWH and Google Cloud BigQuery to drive data migration and modernization from on-premise to BigQuery.

You will design data-ingestion and real-time processing frameworks, implement data governance, and collaborate across teams to deliver scalable data platforms.

Qualifications

• Bachelor’s degree in computer science, CIS, or related field.
• 7 to 10 years of IT experience in software engineering or related field.
• Experience on project(s) involving the implementation of software development life cycles (SDLC).
• Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.
• Strong in programming with SQL and able to write complex queries.
• Hands-on over any one (or multiple) ETL tools among Ab-Initio / Informatica / Teradata / DataStage / etc.
• Experience in GCP Big Query, Cloud Composer, DataProc.
• Operationalize open-source data-analytic tools for enterprise use.
• Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.
• Understand and follow the company development lifecycle to develop, deploy and deliver.
• Good to know BigData tools/skills like Hadoop, Hive, Python, PySpark, etc.
• Good to have experience over other cloud services like Azure or AWS

Skills

SQL

Python

ETL

DWH

Google Cloud

Education

Bachelor's degree in computer science, CIS, or related field

Tools

Ab-Initio

Informatica

Teradata

DataStage

Job description

We are looking for lead data engineers who are skilled inSQL, Python, ETL (Any tool), DWHandGoogle Cloud (BigQuery), to join our team for a project involving the data migration and modernization of data from on-premise to Google BigQuery.

Required Skills and Qualifications

• Bachelor’s degree in computer science, CIS, or related field
• 7 to 10 years of IT experience in software engineering or related field
• Experience on project(s) involving the implementation of software development life cycles (SDLC)
• Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.
• Strong in programming with SQL and able to write complex queries
• Hands-on over any one (or multiple) ETL tools amongAb-Initio / Informatica / Teradata / DataStage / etc.
• Experience in GCP Big Query,Cloud Composer,DataProc
• Offer system support as part of a support rotation with other team members.
• Operationalize open-source data-analytic tools for enterprise use.
• Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.
• Understand and follow the company development lifecycle to develop, deploy and deliver.
• Good to know BigData tools/skills like Hadoop, Hive, Python, PySpark, etc.
• Good to have experience over other cloud services like Azure or AWS