Data Engineer (Databricks, Snowflake & ETL)

Datagaps; Hyderabad; Hybrid; INR 900,000 - 1,500,000; Full time

Job summary

Datagaps in Hyderabad, India is seeking a Data Engineer with 2 to 4 years of experience to design, develop, and maintain modern data solutions using Databricks and Snowflake. You will build scalable ETL/ELT pipelines, optimize data ingestion and transformations, and collaborate with clients to translate requirements into technical solutions while adhering to data warehousing best practices.

Strong SQL, PySpark, and data modeling skills are essential, along with good communication to interact

Qualifications

• 2 to 4 years of Data Engineering experience.
• Hands-on Databricks and Snowflake experience.
• ETL/ELT development and SQL proficiency.
• Strong data warehousing knowledge and PySpark/Spark experience.
• Experience with large datasets and data modeling.
• Client-facing communication and stakeholder management.
• Agile environment experience.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines.
• Build data ingestion, transformation, and loading processes.
• Develop data workflows using Databricks and Snowflake.
• Work with structured and unstructured data from multiple source systems.
• Ensure data quality, integrity, and consistency across platforms.
• Troubleshoot and optimize existing data pipelines for performance.

Skills

Databricks

Snowflake

ETL/ELT Development

SQL

PySpark

Apache Spark

Data Modeling

Data Warehousing

Tools

Azure Data Factory

Azure Data Lake

AWS Glue

Python

Git/GitHub

CI/CD

Power BI

Tableau

Job description

Job Summary

We are seeking a highly motivated Data Engineer with 2 to 4 years of experience in designing, developing, and maintaining modern data solutions. The ideal candidate should possess strong hands-on experience with Databricks, Snowflake, ETL/ELT processes, SQL, and Data Warehousing concepts.

In addition to technical expertise, the candidate should have excellent communication and stakeholder management skills, with the ability to interact directly with clients, gather requirements, provide updates, and contribute to solution discussions.

Key Responsibilities

Data Engineering

• Design, develop, and maintain scalable ETL/ELT pipelines.
• Build and optimize data ingestion, transformation, and loading processes.
• Develop data workflows using Databricks and Snowflake.
• Work with structured and unstructured data from multiple source systems.
• Ensure data quality, integrity, and consistency across platforms.
• Troubleshoot and optimize existing data pipelines for performance and scalability.

Databricks & Snowflake

• Develop and maintain Databricks notebooks and workflows.
• Build data transformation solutions using PySpark and Spark.
• Design and optimize Snowflake data models, schemas, and queries.
• Implement best practices for performance tuning and cost optimization.
• Work with Delta Lake and modern data lakehouse architectures.

Client Interaction

• Directly interact with customers to understand business and technical requirements.
• Participate in client meetings, status updates, and solution discussions.
• Translate business requirements into technical solutions.
• Provide regular progress updates and support issue resolution.
• Collaborate with onsite/offshore teams and stakeholders.

Data Management

• Develop reusable data processing frameworks.
• Validate data quality and perform root cause analysis for data issues.
• Support production deployments and post-production activities.
• Prepare technical documentation and design documents.

Required Skills

Technical Skills

• 2 to 4 years of Data Engineering experience.
• Strong hands-on experience in Databricks.
• Strong hands-on experience in Snowflake.
• Experience in ETL/ELT Development.
• Strong SQL and query optimization skills.
• Good understanding of Data Warehousing concepts.
• Experience with PySpark and Apache Spark.
• Experience working with large datasets.
• Good knowledge of data modeling concepts.

Client-Facing Skills

• Experience interacting directly with clients/business stakeholders.
• Strong communication and presentation skills.
• Ability to gather and understand requirements.
• Ability to independently manage assigned tasks.

Preferred Skills

• Azure Data Factory (ADF)
• Azure Data Lake
• AWS Glue
• Python
• Git/GitHub
• CI/CD knowledge
• Power BI or Tableau exposure

Preferred Candidate Profile

• 2 to 4 years of experience in Data Engineering.
• Strong expertise in Databricks, Snowflake, ETL, and SQL.
• Excellent client communication and stakeholder management skills.
• Ability to work independently and deliver solutions in a fast-paced environment.
• Strong analytical and problem-solving skills.
• Experience working in Agile environments.