Data Engineer

Hackajob; Bhopal, Indore District, Nagpur District; Hybrid; INR 1,200,000 - 2,200,000; Full time

Job summary

Hackajob is seeking a Data Engineer to design, develop, and maintain robust data pipelines for data generation, collection, and processing. The role involves collaborating with cross-functional teams to understand data needs and deliver innovative solutions that improve data accessibility and usability for data-driven decision making.

The position requires at least 5 years of PySpark experience, strong data modeling knowledge, and familiarity with AWS or Azure cloud platforms.

Qualifications

• Must have 5+ years of PySpark experience.
• Strong understanding of data modeling and database design principles.
• Familiarity with cloud platforms such as AWS or Azure for data storage and processing.
• Experience with data warehousing solutions and big data technologies.

Responsibilities

• Develop and optimize data pipelines to ensure efficient data flow and processing.
• Monitor and troubleshoot data quality issues, implementing corrective actions as necessary.
• Collaborate with cross-functional teams and SME to understand data needs and provide solutions.
• Engage with multiple teams and contribute on key decisions.
• Provide solutions to problems for their immediate team and across multiple teams.

Skills

PySpark

Data Engineering

Data Modeling

AWS/Azure

Data Warehousing

Education

15 years full time education

Job description

Summary:

As a Data Engineer, you will engage in the design, development, and maintenance of data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating robust data pipelines, ensuring the integrity and quality of data, and implementing efficient ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide innovative solutions that enhance data accessibility and usability, ultimately contributing to the organization's data-driven decision-making processes.

Roles & Responsibilities:

• Expected to be an SME.
• Collaborate and manage the team to perform.
• Responsible for team decisions.
• Engage with multiple teams and contribute on key decisions.
• Provide solutions to problems for their immediate team and across multiple teams.
• Develop and optimize data pipelines to ensure efficient data flow and processing.
• Monitor and troubleshoot data quality issues, implementing corrective actions as necessary.

Professional & Technical Skills:

• Must To Have Skills: Proficiency in PySpark.
• Good To Have Skills: Experience with Data Engineering.
• Strong understanding of data modeling and database design principles.
• Familiarity with cloud platforms such as AWS or Azure for data storage and processing.
• Experience with data warehousing solutions and big data technologies.

Additional Information:

• The candidate should have minimum 5 years of experience in PySpark.
• This position is based at our Indore office.
• A 15 years full time education is required.
• 15 years full time education