Data Engineer - DataBricks (Gujarat - Onsite)

CloudLabs Inc; Navsari; Remote; INR 1,200,000 - 2,400,000; Full time

Benefits offered by this job

Remote work available

Job summary

CloudLabs Inc is seeking an experienced Data Engineer to design scalable data pipelines, build Lakehouse architectures, and support enterprise analytics initiatives. The role requires 4–5 years of data engineering experience with Databricks, Spark, PySpark, and SQL.

Remote work is available, with collaboration across global teams. The ideal candidate will implement Delta Lake practices, ensure data governance, and optimize large-scale data workloads in a fast-paced environment.

Qualifications

• 4–5 years of experience in Data Engineering.
• Hands-on experience with Databricks and Apache Spark.
• Strong proficiency in PySpark and SQL.
• Experience with Delta Lake and Lakehouse architecture.
• Experience working on AWS, Azure platforms.
• Knowledge of Data Warehousing and Data Modeling concepts.
• Hands-on experience in orchestrating workflows using Airflow, Azure Data Factory, or similar tools.
• Experience with Git-based version control and CI/CD practices.

Responsibilities

• Design and develop scalable data pipelines using Databricks and Spark.
• Build and maintain Lakehouse architectures and data platforms.
• Develop and optimize ETL/ELT processes using PySpark and SQL.
• Implement Delta Lake best practices for data reliability and performance.
• Integrate data from multiple structured and unstructured sources.
• Collaborate with business teams, architects, and data scientists to deliver data solutions.
• Monitor data pipelines and troubleshoot data processing issues.
• Implement data governance, security, and compliance controls.
• Perform query optimization and performance tuning for large-scale data workloads.

Skills

Databricks

Apache Spark

PySpark

SQL

Delta Lake

Lakehouse architecture

AWS

Azure

Data Warehousing

Data Modeling

Airflow

Azure Data Factory

Git

CI/CD

Kafka

Event Hubs

Kinesis

Tools

Airflow

Azure Data Factory

Job description

CloudLabs Inc was founded in 2014 in USA with the mission to provide exceptional IT & Business consulting services at a competitive price, to help clients realize the best value from their investments. Within a short span, CloudLabs evolved from pure-play consulting into a transformative partner for Business Acceleration Advisory, Transformative Application Development & Managed Services - enabling digital transformations, M&A transitions, Automation & Process-driven optimizations & complex Integration initiatives for enterprises across the globe. As a Strategic Planning & Implementation Partner for global companies, CloudLabs has seen a 200% uptake in winning high-value, high-impact and high-risk projects that are critical for the business.

With offices in the US, Canada & India and with the team of 250+ experienced specialists, CloudLabs is now at an inflection point and ready for its next curve of progress.

Please write & follow us here:

Website: cloudlabsit.com

LinkedIn: CloudLabs Inc

Email us: [e-mail]

Experience Required: 4+ years in Senior Project Manager

Start Date: Immediate or as soon as available

What we offer:

• We welcome candidates rejoining the workforce after career break/parental leave and support their journey to reacclimatize too corporate.
• Opportunity to work remotely is available.

About the Role

We are looking for an experienced Data Engineer with strong expertise in Databricks and Big Data technologies. The candidate will be responsible for building scalable data pipelines, implementing modern Lakehouse architectures, and supporting enterprise analytics initiatives.

Key Responsibilities

• Design and develop scalable data pipelines using Databricks and Spark.
• Build and maintain Lakehouse architectures and data platforms.
• Develop and optimize ETL/ELT processes using PySpark and SQL.
• Implement Delta Lake best practices for data reliability and performance.
• Integrate data from multiple structured and unstructured sources.
• Collaborate with business teams, architects, and data scientists to deliver data solutions.
• Monitor data pipelines and troubleshoot data processing issues.
• Implement data governance, security, and compliance controls.
• Perform query optimization and performance tuning for large-scale data workloads.

Required Skills:

• Must have 4 to 5 years of experience in Data Engineering.
• Hands-on experience with Databricks and Apache Spark.
• Strong proficiency in PySpark and SQL.
• Experience with Delta Lake and Lakehouse architecture.
• Experience working on AWS, Azure platforms.
• Knowledge of Data Warehousing and Data Modeling concepts.
• Hands-on experience in orchestrating workflows using Airflow, Azure Data Factory, or similar tools.
• Experience with Git-based version control and CI/CD practices.

Good to Have

• Experience with Snowflake.
• Knowledge of streaming technologies such as Kafka, Event Hubs, or Kinesis.
• Strong technical and analytical capabilities.
• Excellent debugging and troubleshooting skills.
• Effective communication and collaboration skills.
• Ability to work in a fast-paced, agile environment.