Principal Data Engineer - ETL Development

Forward Eye Technologies; Pune District; On-site; INR 1,800,000 - 2,800,000; Full time

Job summary

Forward Eye Technologies in India seeks an experienced BI/ETL Engineer to design, develop, and optimize ETL pipelines for data warehousing and analytics. The role involves Python/PLSQL scripting, SQL data workflows, and CI/CD management with GitLab.

You will collaborate with business stakeholders, ensure data quality, governance, and performance while reporting insights through dashboards and reports. Hyderabad, Ahmedabad, Pune, Chennai, and Kolkata are listed locations.

Qualifications

• 8+ years of experience in IT with focus on BI, Data Warehousing, and Analytics systems.
• Proven experience with DevOps methodologies, CI/CD pipelines, and GitLab workflows.
• Proficient in ETL development and creating data workflows in SQL, PL/SQL, Python, and Linux shell scripting.

Responsibilities

• Design, develop, and implement ETL processes to extract, transform, and load data into data warehouses or analytics systems.
• Implement and support CI/CD pipelines for EDW code development and deployment using GitLab.
• Develop dashboards, reports, and visualizations to communicate insights to stakeholders.
• Develop and maintain Linux shell scripting, Python, and PL/SQL code for data tasks.
• Perform QA checks on installations, configurations, and code to ensure performance and scalability.
• Collaborate with business stakeholders to translate requirements into technical data solutions.
• Implement data governance best practices to ensure security, integrity, and privacy of data.
• Present data findings to stakeholders across organizational levels.

Skills

ETL Development

GitLab CI/CD

Python/PLSQL

Data Analytics

Tools

SQL

Linux

Node.js

Job description

Job Responsibilities :

1. ETL Development and Data Processing :

• - Design, develop, and implement ETL processes to extract, transform, and load data from various sources into data warehouses or analytics systems.
• - Maintain and enhance ETL pipelines to ensure data quality, integrity, and consistency.

2. DevOps and CI/CD Pipeline Management :

• - Implement and support CI/CD pipelines for Enterprise Data Warehouse (EDW) code development and deployment using GitLab.
• - Manage and configure workflows for code repositories, ensuring seamless code integration and promotion across environments.

3. Data Analysis and Reporting :

• - Apply statistical analysis and data mining techniques to uncover trends, patterns, and insights within large datasets.
• - Develop and maintain dashboards, reports, and visualizations to communicate key insights to stakeholders.

4. Programming and Scripting :

• - Develop and maintain Linux shell scripting, Python, and PL/SQL code for data extraction, transformation, and automation tasks.
• - Create efficient scripts for data replication, migration, and data integration between systems.

5. Quality Assurance and System Optimization :

• - Perform quality assurance checks on system installations, configurations, and code to ensure performance and scalability.
• - Recommend improvements to optimize performance and reliability of data processes and systems.

6. Business Understanding and Translation :

• - Work closely with business stakeholders to understand their goals, processes, and requirements, translating them into technical data solutions.
• - Ensure alignment between technical solutions and business objectives to drive strategic decision-making.

7. Data Governance and Security :

• - Implement data governance best practices to ensure the security, integrity, and privacy of enterprise data.
• - Adhere to data security regulations and policies, ensuring that data is handled with confidentiality and compliance.

8. Collaboration and Stakeholder Communication :

• - Collaborate with cross-functional teams, including data analysts, engineers, and business teams, to deliver impactful data solutions.
• - Present data findings and actionable insights to stakeholders at various organizational levels, both written and verbally.

Required Qualifications :

Experience :

• - 8+ years of experience in IT with a focus on Business Intelligence, Data Warehousing, and Analytics systems.
• - Proven experience with DevOps methodologies, CI/CD pipelines, and GitLab workflows.
• - Proficient in ETL development and creating data workflows in SQL, PL/SQL, Python, and Linux shell scripting.

Technical Skills :

• - Strong knowledge of open-source technologies, Linux, and GitLab for codebase management.
• - Experience in data replication and data migration processes.
• - Hands-on experience with Python and Node.js for programming and automation tasks.

Soft Skills :

• - Excellent problem-solving and analytical abilities.
• - Strong communication skills with the ability to convey complex technical concepts to non-technical stakeholders.
• - Ability to manage multiple projects in a dynamic and fast-paced environment.

Preferred Skills :

• - Experience with data governance principles and ensuring data security compliance.
• - Familiarity with data quality assurance and optimization techniques.
• - Understanding of modern data platforms such as cloud-based data warehouses and data lakes.

Location - Hyderabad,Ahmedabad,pune,chennai,kolkata.