Data Engineer

Cnx

Bengaluru

On-site

INR 1,200,000 - 1,500,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Concentrix seeks an experienced Data Engineer in Bangalore to design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, Spark, and Delta Lake. You will build reliable data workflows, ensure data quality, and optimize performance in Azure cloud services.

You will collaborate with data architects, analysts, and business stakeholders, implement CI/CD practices, and maintain documentation while upholding security and governance standards.

Qualifications

  • 4–6 years of relevant experience in data engineering and cloud‑based data platforms.
  • Strong hands‑on experience with Azure Databricks, PySpark, Apache Spark, Python, and advanced SQL.
  • Proven expertise in developing scalable ETL/ELT pipelines for batch and large‑scale data processing.

Responsibilities

  • Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Databricks, PySpark, Python, and SQL.
  • Build and optimize batch and large‑scale data‑processing solutions using Apache Spark.
  • Develop data ingestion and transformation workflows for structured and semi‑structured data.
  • Implement reliable data‑processing frameworks using Delta Lake capabilities.
  • Perform data cleansing, validation, reconciliation, and quality checks to ensure accuracy and consistency.
  • Optimize Spark jobs, Databricks notebooks, SQL queries, clusters, and data pipelines for performance and cost efficiency.
  • Troubleshoot pipeline failures, data‑quality issues, production incidents, and performance bottlenecks.
  • Implement incremental data processing, partitioning, caching, join optimization, and file‑management techniques.
  • Develop reusable and configurable data‑engineering components.
  • Implement version control and automated deployment processes using CI/CD and DevOps practices.
  • Collaborate with data architects, analysts, application teams, and business stakeholders to translate requirements into technical solutions.
  • Maintain technical documentation covering pipeline architecture, transformation logic, dependencies, and operational procedures.
  • Ensure solutions comply with organizational standards for security, governance, monitoring, and data protection.

Skills

SQL
Python
ETL/ELT development
Data pipelines design
Performance optimization
Data governance/security

Tools

Azure Databricks
PySpark
Apache Spark
Delta Lake
Azure Data Lake Storage
Azure Data Factory
Git
CI/CD
DevOps practices

Job description

Job Title: Data Engineer Job Description We're Concentrix. The intelligent transformation partner. Solution-focused. Tech-powered. Intelligence-fueled. The global technology and services leader that powers the world’s best brands, today and into the future. We’re solution-focused, tech-powered, intelligence-fueled. With unique data and insights, deep industry expertise, and advanced technology solutions, we’re the intelligent transformation partner that powers a world that works, helping companies become refreshingly simple to work, interact, and transact with. We shape new game-changing careers in over 70 countries, attracting the best talent. In our Information Technology and Global Security team, you will deliver the latest technology infrastructure, transformative software solutions and industry-leading global security for our staff and clients. You will work with the best in the world to design, implement and strategize IT, security, application development, innovation, and solutions in today’s hyperconnected world. You will be part of the technology team that is core to our vision of develop, build and run the future of Integrated Services. Our game-changers around the world have devoted their careers to ensuring every relationship is exceptional. And we’re proud to be recognized with awards such as "World's Best Workplaces," “Best Companies for Career Growth,” and “Best Company Culture,” year after year. We embrace our game-changers with open arms, people from diverse backgrounds, who are curious and willing to learn. Your natural talent to help others and go beyond WOW for our customers will fit right in with what we do and who we are. Join us and be part of this journey towards greater opportunities and brighter futures.

Job Overview

We are seeking an experienced and results-driven Data Engineer with 4–6 years of relevant experience in designing, developing, and optimizing scalable data solutions using Azure Databricks and related technologies. The ideal candidate should have strong hands‑on expertise in Azure Databricks, PySpark, Apache Spark, SQL, Python, ETL/ELT development, Delta Lake, Azure cloud services, CI/CD, and performance optimization. The role requires practical experience in building reliable, secure, and production‑grade data pipelines.

Key Responsibilities
  • Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Databricks, PySpark, Python, and SQL.
  • Build and optimize batch and large-scale data‑processing solutions using Apache Spark.
  • Develop data ingestion and transformation workflows for structured and semi‑structured data.
  • Implement reliable data‑processing frameworks using Delta Lake capabilities.
  • Perform data cleansing, validation, reconciliation, and quality checks to ensure accuracy and consistency.
  • Optimize Spark jobs, Databricks notebooks, SQL queries, clusters, and data pipelines for performance and cost efficiency.
  • Troubleshoot pipeline failures, data‑quality issues, production incidents, and performance bottlenecks.
  • Implement incremental data processing, partitioning, caching, join optimization, and file‑management techniques.
  • Develop reusable and configurable data‑engineering components.
  • Implement version control and automated deployment processes using CI/CD and DevOps practices.
  • Collaborate with data architects, analysts, application teams, and business stakeholders to translate requirements into technical solutions.
  • Maintain technical documentation covering pipeline architecture, transformation logic, dependencies, and operational procedures.
  • Ensure solutions comply with organizational standards for security, governance, monitoring, and data protection.
Required Technical Skills
  • 4–6 years of relevant experience in data engineering and cloud‑based data platforms.
  • Strong hands‑on experience with Azure Databricks, PySpark, Apache Spark, Python, and advanced SQL.
  • Proven expertise in developing scalable ETL/ELT pipelines for batch and large‑scale data processing.
  • Solid understanding of Delta Lake, including Delta tables, ACID transactions, schema evolution, merge/upsert operations, and table optimization.
  • Experience with Azure data services such as Azure Data Lake Storage and Azure Data Factory.
  • Strong knowledge of Spark and Databricks performance optimization, including partitioning, caching, join strategies, shuffle optimization, file sizing, and cluster configuration.
  • Experience with Git‑based version control, CI/CD pipelines, and DevOps practices for automated deployment across environments.
  • Good understanding of data quality, security, monitoring, troubleshooting, and production support.

Location: IND Bangalore - MTP C4, 3rd Flr

Language Requirements: Time Type: Full time

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Concentrix • Bengaluru

On-site
INR 1,500,000 - 2,500,000
Data Engineer-Information Technology
Data Engineer-Information Technology

Emcure Pharmaceuticals Limited • Pune District

On-site
INR 1,400,000 - 2,000,000
Data Engineer(Azure, ADF, Databricks, PySpark, Unity Catalog, SQL) || 6 to 10 Years || Bangalore
Data Engineer(Azure, ADF, Databricks, PySpark, Unity Catalog, SQL) || 6 to 10 Years || Bangalore

Innova ESI • Bengaluru

On-site
INR 800,000 - 1,500,000
Databricks Developer
Databricks Developer

Kumaran Systems • India

On-site
INR 900,000 - 1,500,000
Databricks
Databricks

Infosys • Bengaluru

On-site
INR 3,000,000 - 5,500,000
Data Engineer(Azure, ADF, Databricks, PySpark, Unity Catalog, SQL) || 6 Years || Bangalore
Data Engineer(Azure, ADF, Databricks, PySpark, Unity Catalog, SQL) || 6 Years || Bangalore

Innova ESI • Bengaluru

On-site
INR 800,000 - 1,600,000
Databricks Engineer
Databricks Engineer

DATAECONOMY Inc • Pune District

On-site
INR 2,500,000 - 4,000,000
Data Engineer
Data Engineer

Expleo Group • Chennai

On-site
INR 900,000 - 1,300,000
IN_Senior Associate_ Databricks Senior Data Engineer_GCC_Advisory_Bangalore
IN_Senior Associate_ Databricks Senior Data Engineer_GCC_Advisory_Bangalore

PwC • Bengaluru

On-site
INR 1,800,000 - 2,400,000
Data Engineer
Data Engineer

Intact Insurance • India

On-site
INR 1,500,000 - 2,300,000