Data Engineer

Awign; Bengaluru Urban; Remote; INR 1,200,000 - 2,000,000; Full time

Job summary

A leading technology firm is looking for a Remote Data Engineer with 5-7 years of experience in Databricks, PySpark, and Data Fabric concepts. Candidates should possess strong skills in designing data ingestion pipelines using Azure Data Factory and have a solid understanding of data transformation and optimization techniques. This full-time role offers an opportunity to work closely with cross-functional teams and contribute to enterprise data transformation initiatives.

Qualifications

• 5-7 years of experience in data engineering, particularly with Databricks.
• Strong hands-on engineering skills and understanding of modern data architectures.
• Working knowledge of Data Fabric concepts.

Responsibilities

• Design and implement data ingestion pipelines using Azure Data Factory.
• Optimize Databricks queries and performance for data ingestion.
• Collaborate with cross-functional teams including data analysts and scientists.

Skills

T-SQL

Databricks

Azure Data Factory

PySpark

C#

Job description

Remote Data Engineer position (2 openings). Minimum 5‑7 years experience. Remote work.

Screening Checklist

• Proficiency in interpreting data transformation logic written in T‑SQL and implementing equivalent processes within Databricks.
• Ability to design and implement data ingestion pipelines using Azure Data Factory (from source to RAW layer).
• Basic knowledge of C# and SQL (reading code suffices).
• Experience in collecting and analyzing performance metrics to optimize data ingestion pipelines.
• Competence in performing performance optimizations for Databricks read/write queries as needed.

Job Overview

We are currently seeking experienced Data Engineers (5‑7 years) with strong expertise in Databricks, PySpark, and Data Fabric concepts to contribute to an ongoing enterprise data transformation initiative. The ideal candidates will have solid hands‑on engineering skills, a good understanding of modern data architectures, and the ability to work collaboratively within cross‑functional teams.

Key capabilities & expectations

• Strong experience in understanding and translating data transformation logic written in TSQL and implementing efficient transformations in Databricks using PySpark, aligned with Data Fabric design principles.
• Hands‑on experience in designing and implementing data ingestion pipelines using Azure Data Factory, enabling reliable data movement from source systems to the RAW and curated data layers within a Data Fabric ecosystem.
• Working knowledge of Data Fabric concepts, including metadata‑driven pipelines, data integration Sullivan, orchestration, data lineage, and governance.
• Experience in monitoring, collecting, and analyzing pipeline performance metrics to identify inefficiencies and support optimization of data ingestion and processing workflows.
• Practical experience in performance tuning and optimization of Databricks read and write operations, including partitioning, file formats, and query optimization techniques.
• Ability to collaboration closely with senior engineers and architects, contribute to design discussions, follow best practices, and support the continuous improvement of the data platform.
• Strong problem‑solving skills, eagerness to learn, and the ability to work effectively with cross‑functional teams, including data analysts, data scientists, and business stakeholders.

This role is ideal for professionals looking to deepen their expertise in Databricks and Data Fabric architectures while contributing to scalable, well‑governed, and high-performance enterprise Sobre data solutions.

Seniority Level

Mid‑Senior level

Employment Type

Full‑time

Job Function

Information Technology