Data Engineer

Infobeans; Indore District, Pune District; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Infobeans is seeking a Data Engineer with strong Python and PySpark skills to design and implement robust data pipelines in a fast-paced environment.

The candidate should have deep SQL knowledge, experience with Databricks and Azure Data Factory, and a passion for scalable, well-documented code from scratch.

Qualifications

• Proficient in Python and PySpark with strong SQL skills.
• Experience building scalable data pipelines and data models.
• Ability to write clean, modular, well-documented code.

Responsibilities

• Design, develop, and maintain Python and PySpark solutions.
• Write complex SQL queries for data transformation and validation.
• Build scalable, reusable code frameworks with modular design.

Skills

Python

PySpark

SQL

Databricks

Azure Data Factory

Tools

Databricks

Azure Data Factory

Job description

Data Engineer

Overview:

We are seeking a highly skilled Python and PySpark Developer who is passionate about building robust data solutions from the ground up. The ideal candidate should have a deep understanding of core programming concepts, be highly proficient in Python and PySpark, and possess strong SQL skills. We're looking for someone who thinks out of the box, understands the value of reusable and scalable solutions, and is comfortable working in dynamic, fast-paced environments. Experience with Databricks and Azure Data Factory (ADF) is needed.

Responsibilities:

• Design, develop, and maintain Python and PySpark.
• Write complex SQL queries for data transformation, validation, and analysis.
• Build scalable, reusable code frameworks with a dynamic and modular mindset.
• Develop solutions from scratch with clean, well-documented code.
• Optimize and refactor existing code for better performance and scalability.
• Collaborate closely with data engineers, analysts, and business stakeholders to deliver impactful solutions.
• Implement best practices in data engineering, coding standards, and deployment.
• Work with Databricks and ADF to orchestrate and manage data workflows.