Senior Data Engineer

ThinkWise Consulting LLP; Hyderabad; Presencial; INR 2.500.000 - 4.500.000; Jornada completa

Descripción de la vacante

ThinkWise Consulting LLP is seeking a Senior Data Engineer to design, develop, and optimize scalable cloud data platforms and modern ETL/ELT pipelines. The role focuses on data lakehouse and data warehouse solutions using Fabric, Databricks, or Snowflake.

You will work with PySpark, SQL, Python/Scala and cloud data services to enable analytics, governance, and reporting, collaborating with data scientists, analysts, and stakeholders to deliver reliable data solutions.

Formación

• Bachelor's or Master's in CS/IT/Engineering.
• 6–10 years of data engineering experience.
• Hands-on with SQL, PySpark, Python/Scala.
• Experience with cloud data services (Azure/AWS/GCP).
• Experience with Delta Lake, Spark, Snowflake, Fabric.

Responsabilidades

• Design, develop, and maintain scalable data pipelines, data models, and ETL/ELT processes.
• Build and optimize data lake, lakehouse, and data warehouse solutions using Databricks, Snowflake, or Fabric.
• Develop data processing solutions using PySpark, Python, Scala, and SQL.
• Work with large datasets to improve data processing performance, reliability, and scalability.
• Develop and maintain batch and real-time data pipelines as required.
• Implement data transformation, cleansing, validation, and integration processes.
• Work with Delta Lake, Spark, Snowflake, Fabric and other modern data engineering technologies.
• Support analytics and self-service reporting through Power BI.

Conocimientos

Data modeling

Analytical thinking

Problem solving

Independent work

Collaboration

Educación

Bachelor's/Master’s degree in CS/IT/Engineering

Herramientas

SQL

PySpark

Python

Scala

Power BI

Databricks

Snowflake

Microsoft Fabric

Git

CI/CD

Descripción del empleo

We are hiring a **Senior Data Engineer** to design, develop, and optimize scalable cloud data platforms and modern ETL/ELT pipelines using **Microsoft Fabric, Databricks, or Snowflake**.

The role involves building and maintaining data lakehouse and data warehouse solutions, developing high-performance data pipelines, enabling analytics and reporting, and contributing to data engineering best practices.

Strong hands-on experience in **PySpark, SQL, Python/Scala, and cloud data platforms** is required.

What You Will Do

Following are the high-level responsibilities that you will play, but are not limited to:

• Design, develop, and maintain scalable data pipelines, data models, and ETL/ELT processes.
• Build and optimize data lake, lakehouse, and data warehouse solutions using **Microsoft Fabric, Databricks, or Snowflake**.
• Develop data processing solutions using **PySpark, Python, Scala, and SQL**.
• Work with large and complex datasets to improve data processing performance, reliability, and scalability.
• Develop and maintain batch and real-time data pipelines as required.
• Implement data transformation, cleansing, validation, and integration processes.
• Work with **Delta Lake, Spark, Snowflake, Microsoft Fabric**, and other modern data engineering technologies.
• Support business analytics and self-service reporting through **Power BI** and other visualization platforms.
• Collaborate with data scientists, analysts, BI developers, and business stakeholders to deliver reliable data solutions.
• Implement data quality, governance, security, and monitoring practices across data pipelines.
• Troubleshoot and resolve data pipeline, performance, and production issues.
• Contribute to the development of coding standards, reusable frameworks, and data engineering best practices.
• Participate in code reviews and ensure high-quality, maintainable, and scalable solutions.
• Implement CI/CD practices for data engineering workflows and deployments.
• Stay updated with emerging data engineering technologies and contribute to continuous improvement of the data ecosystem.

Required Qualifications

• Bachelor's/Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 6–10 years of experience** in data engineering, data platform development, or related areas.

Strong hands-on experience in one or more of the following:

• Microsoft Fabric** – Data Factory, Lakehouse, Data Warehouse
• Snowflake** – Data Warehousing, Snowpipe, Performance Optimization
• Power BI** – Data Modeling, DAX, Report Development
• Strong proficiency in **SQL**.
• Strong programming skills in **Python or Scala**.
• Hands‑on experience with **PySpark/Apache Spark** and large‑scale data processing.
• Experience working with **Azure, AWS, or GCP** cloud data services.
• Strong understanding of data modeling, ETL/ELT concepts, data warehousing, and data lake/lakehouse architectures.
• Understanding of data governance, data quality, security, and access control.
• Experience with **CI/CD, Git, and DevOps practices** for data engineering workflows.
• Strong analytical and problem‑solving skills.
• Ability to work independently and collaborate effectively with cross-functional teams.