Data Engineer

Thoucentric Technology Private Limited; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Thoucentric is seeking a data engineer in Bengaluru to design, develop, and maintain robust data pipelines using Azure Data Factory (ADF). The role involves building ETL/ELT processes, enabling batch and near real-time data flows, and integrating ADF with Blob Storage, Synapse, and Databricks.

You will collaborate with data analysts to understand data requirements, ensure data quality, and optimize pipeline performance.

Qualifications

• Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF).
• Create, schedule, and monitor ADF pipelines; implement batch and near real-time flows.
• Integrate ADF with Blob Storage, Synapse, Databricks, Key Vault, and other services.
• Write performant SQL queries and data transformations; analyze large datasets.
• Implement logging, alerting, retry logic for production stability.
• Implement data validation and quality checks; maintain data dictionaries.

Responsibilities

• Collaborate with data analysts to understand data requirements and structure transformations accordingly.
• Ensure data is accurate, consistent, and aligned with business logic.
• Troubleshoot pipeline issues and optimize performance; diagnose data quality problems.
• Document pipeline logic, data flows, and troubleshooting procedures.
• Work with data architects, BI developers, and business users.

Skills

Azure Data Factory

ETL/ELT

SQL queries

Spark

Databricks

Data pipelines

Data quality

Data validation

Pipeline monitoring

Performance optimization

Tools

Blob Storage

Synapse

Databricks

Key Vault

Job description

At Thoucentric, we offer end-to-end consulting solutions designed to address the most pressing business challengesacross industries. Leveraging deep domain expertise, cutting-edge technology, and a results-driven approach, wehelp organizations streamline operations, enhance decision-making, and accelerate growth. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.

We help clients with Business Consulting, Program & Project Management, Digital Transformation, Product Management, Process & Technology Solutioning and Execution including Analytics & Emerging Tech areas cutting across functional areas such as Supply Chain, Finance & HR, Sales & Distribution across US, UK, Singapore and Australia. Our unique consulting framework allows us to focus on execution rather than pure advisory. We are working closely with marquee names in the global consumer & packaged goods (CPG) industry, new age tech and start-up ecosystem.

We have been certified as "Great Placeto Work" by AIM and have been ranked as "50 Best Firms for Data Scientists to Work For".

We have an experienced consulting team of over 500+ world-class business and technology consultants based across six global locations, supporting clients through their expert insights, entrepreneurial approach and focus on delivery excellence. We have also built point solutions and products through Thoucentric labs using AI/ML in the supply chain space.

Job Description

Exp-3-5 years

• Design, develop, and maintain robust data pipelinesusing Azure Data Factory (ADF)
• Build ETL/ELT processes that ingest,transform, and load data from various sources into data lakes, warehouses, or databases
• Implement both batch and near real-time data flows

• Create, schedule, and monitor ADF pipelines
• Implement complex data workflows, including dependency management,parameterization, and dynamic pipeline design
• Integrate ADF with otherAzure services (Blob Storage, Synapse, Databricks, Key Vault, etc.)
• Implement logging, alerting, and retry logic for production stability

• Analyze large datasets to identify trends, issues, andtransformation needs
• Write performant SQL queries and transformations
• Collaborate with data analysts to understand data requirements and structuretransformations accordingly
• Ensure data is accurate, consistent, and alignedwith business logic

• Quickly diagnose and resolve data quality issues,pipeline failures, or performance bottlenecks
• Perform root cause analysisfor pipeline errors and implement long-term fixes
• Debug and optimize Sparkor SQL transformations in environments like Databricks

• Implement data validation and quality checks within pipelines
• Perform reconciliation and sanity checks across data sources
• Maintain datadictionaries and metadata repositories

• Work closely with data architects, analysts, BIdevelopers, and business users
• Translate business data needs into technical implementations
• Document pipeline logic, data flows, and troubleshooting procedures

• Work closely with data architects, analysts, BIdevelopers, and business users
• Translate business data needs into technical implementations
• Document pipeline logic, data flows, and troubleshooting procedures

• Optimize ADF pipeline execution time and cost
• Improve query performance using indexing, partitioning, and efficienttransformation logic
• Recommend improvements in data infrastructure andtooling

What a Consulting role at Thoucentric will offer you?

Opportunity to define your career path and not as enforced by a manager

A great consulting environment with a chance to work with Fortune 500 companies and startups alike

A dynamic but relaxed and supportive working environment that encourages personal development

Be part of One Extended Family. We bond beyond work - sports, get-togethers, common interests etc. Work in a very enriching environment with Open Culture, Flat Organization and Excellent Peer Group

Be part of the exciting Growth Story of Thoucentric!