Data Engineer

hackajob; Ahmedabad District, Jaipur, Mumbai; Hybrid; INR 1,500,000 - 2,500,000; Full time

Job summary

hackajob is seeking a Data Engineer to design, develop, and maintain data solutions for generation, collection, and processing. You will build robust data pipelines to ensure data quality and enable seamless data movement across systems.

Collaboration with cross-functional teams to translate data requirements into scalable technical solutions is essential, while maintaining governance and best practices for data integrity in cloud environments.

Qualifications

• Must have 3+ years of experience with Databricks Unified Data Analytics Platform.
• Experience in building and managing scalable data pipelines and workflows.
• Strong knowledge of data integration techniques and ETL processes.
• Familiarity with cloud-based data storage and processing solutions.
• Ability to troubleshoot and resolve data-related issues efficiently.
• Understanding of data governance and data quality best practices.

Responsibilities

• Designed, developed and maintained data solutions for data generation, collection and processing.
• Created data pipelines ensuring data quality and seamless data migration across systems.
• Collaborated with cross-functional teams to gather and analyze data requirements and translate them into technical solutions.
• Monitored and optimized data pipelines for performance and reliability.
• Documented data processes and workflows to aid transparency and knowledge sharing.
• Mentored junior team members and provided guidance in data engineering tasks.

Skills

Databricks

Data pipelines

ETL processes

Cloud storage

Data governance

Job description

Project Role Description :

Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills :

Databricks Unified Data Analytics Platform

Summary:

As a Data Engineer, your typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data. You will be responsible for creating efficient data pipelines that facilitate smooth data flow and ensure the integrity and quality of data throughout its lifecycle. Your role includes implementing processes to extract, transform, and load data, enabling seamless migration and deployment across various systems.

Collaboration with different teams to understand data requirements and optimize data architecture is an integral part of your daily activities, ensuring that data solutions align with organizational goals and support business intelligence needs.

Roles & Responsibilities:

• Expected to perform independently and become an SME.
• Required active participation/contribution in team discussions.
• Contribute in providing solutions to work related problems.
• Collaborate with cross-functional teams to gather and analyze data requirements and translate them into technical solutions.
• Monitor and optimize data pipelines to improve performance and reliability.
• Document data processes and workflows to maintain transparency and facilitate knowledge sharing.
• Assist junior team members by providing guidance and support in their tasks.

Professional & Technical Skills:

• Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform.
• Experience in building and managing scalable data pipelines and workflows.
• Strong knowledge of data integration techniques and ETL processes.
• Familiarity with cloud-based data storage and processing solutions.
• Ability to troubleshoot and resolve data-related issues efficiently.
• Understanding of data governance and data quality best practices.

Additional Information:

• The candidate should have minimum 3 years of experience in Databricks Unified Data Analytics Platform.