ETL Engineer

PURVIEW

Pune City

Hybrid

INR 1,200,000 - 1,800,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Competitive salary

Comprehensive benefits package

Professional development opportunities

Collaborative environment

Job summary

PURVIEW is looking for a Senior Data Engineer / ETL Engineer in Pune to design robust ETL pipelines and optimize data processing systems. The ideal candidate will have extensive experience with Data Stage and GCP, ensuring effective data management and governance. This role involves collaboration with cross-functional teams, requiring strong analytical skills and a proactive approach to data challenges. Join a dynamic team that values innovation and offers opportunities for professional growth.

Qualifications

Minimum of 5 years in data engineering or ETL development with GCP focus.
Experience in designing ETL solutions including data modeling and warehousing.
Proficiency in Data Stage and data integration tools.

Responsibilities

Design and implement ETL processes using Data Stage in GCP.
Optimize data pipelines for performance and troubleshoot ETL processes.
Integrate and manage data from various sources to ensure quality.

Skills

Data Stage

Bigdata

SQL

Google Cloud Platform (GCP)

Apache Beam

Hadoop

Python

Java

Terraform

Kafka

Education

Bachelor’s degree in Computer Science, Information Systems, or a related field.

Job description

2 days ago Be among the first 25 applicants

Senior Data Engineer / ETL Engineer

Job Description:-

Location:Pune (Hybrid)

Role: ETL Engineer

Brief JD below:

Principal responsibilities-

Role: Senior Data Engineer / ETL Engineer

Job Summary:

As a Senior Data Engineer / ETL Engineer, you will be instrumental in designing, developing, and optimizing data processing systems that support our organization's data initiatives. Your expertise in Data Stage, Data Flow, SQL, Bigdata and Google Cloud Platform (GCP) will be essential in building robust ETL pipelines that transform raw data into actionable insights. You will collaborate with cross-functional teams to ensure that data is accurate
accessible, and valuable for decision-making.

Responsibilities:

Design and Develop ETL Processes:

Lead the design and implementation of ETL processes using Data Stage and other relevant tools to extract, transform, and load data from various sources into GCP.
Collaborate with stakeholders to gather requirements and ensure that ETL solutions meet business needs.

Data Pipeline Optimization:

Optimize data pipelines for performance, scalability, and reliability, ensuring efficient data processing workflows.
Monitor and troubleshoot ETL processes, proactively addressing issues and bottlenecks.

Data Integration and Management:

Integrate data from diverse sources, including databases, APIs, and flat files, ensuring data quality and consistency.
Manage and maintain data storage solutions in GCP (e.g., BigQuery, Cloud Storage) to support analytics and reporting.

SQL Development:

Write complex SQL queries for data extraction, transformation, and analysis, ensuring optimal performance and accuracy.
Collaborate with data analysts and data scientists to prepare data for analysis and reporting

GCP Dataflow Development:

Write Apache Beam based Dataflow Job for data extraction, transformation, and analysis, ensuring optimal performance and accuracy.
Collaborate with data analysts and data scientists to prepare data for analysis and reporting.

Automation and Monitoring:

Implement automation for ETL workflows using tools like Apache Airflow or Cloud Composer, enhancing efficiency and reducing manual intervention.
Set up monitoring and alerting mechanisms to ensure the health of data pipelines and compliance with SLAs.

Data Governance and Security:

Apply best practices for data governance, ensuring compliance with industry regulations (e.g., GDPR, HIPAA) and internal policies.
Collaborate with security teams to implement data protection measures and address vulnerabilities.

Documentation and Knowledge Sharing:

Document ETL processes, data models, and architecture to facilitate knowledge sharing and onboarding of new team members.
Conduct training sessions and workshops to share expertise and promote best practices within the team.

Qualifications:

Education:
Bachelor’s degree in Computer Science, Information Systems, or a related field.

Experience:

Minimum of 5 years of industry experience in data engineering or ETL development, with a strong focus on Data Stage and GCP.
Proven experience in designing and managing ETL solutions, including data modeling, data warehousing, and SQL development.

Technical Skills:

Proficiency in Data Stage and experience with data integration tools.
Strong knowledge of GCP services (e.g., BigQuery, Dataflow, Cloud Storage, Pub/Sub) and their application in data engineering.
Experience of cloud-based solutions, especially in GCP, cloud certified candidate is preferred.
Experience and knowledge of Bigdata data processing in batch mode and streaming mode, proficient in Bigdata eco systems, e.g. Hadoop, HBase, Hive, MapReduce, Kafka, Flink, Spark, etc.
Familiarity with Java & Python for data manipulation on Cloud/Bigdata platform.
Familiarity with Python for scripting and data manipulation, as well as Terraform for infrastructure as code (IaC).

Analytical Skills:

Strong problem-solving skills with a keen attention to detail.
Ability to analyze complex data sets and derive meaningful insights.

Benefits:

Competitive salary and comprehensive benefits package.
Opportunity to work in a dynamic and collaborative environment on cutting-edge data projects.Professional development opportunities to enhance your skills and advance your career.

If you are a passionate data engineer with expertise in ETL processes and a desire to make a significant impact within our organization, we encourage you to apply for this exciting opportunity!

Seniority level