Data Engineer

ThoughtSol Infotech Ltd.; Dadri; On-site; INR 1,800,000 - 2,600,000; Full time

Job summary

ThoughtSol Infotech Ltd. is seeking a skilled Data Engineer to manage, monitor, and optimize Azure-based data lakehouse infrastructure.

You will handle ADLS, Databricks, Synapse, and ADF, implement security, backups, and cost optimization, collaborating with DevOps for CI/CD and maintaining documentation.

Applicants should have 3+ years of cloud administration experience and strong scripting with Python or Shell.

Qualifications

• Minimum 3 years of relevant experience in Cloud Administration or Azure Infrastructure Management.
• Experience with Microsoft Azure cloud services and cloud storage management.
• Familiarity with Azure Data Lake Storage (ADLS) and data platforms.
• Knowledge of Azure Databricks, ADF, and Synapse Analytics.
• Understanding of data lakehouse architecture and cloud-based data platforms.
• Experience in cloud infrastructure monitoring, administration, and troubleshooting.
• Knowledge of cloud security, IAM, and RBAC.
• Experience in backup management, data recovery, retention, and tiering.

Responsibilities

• Manage, maintain, and monitor Azure-based data lakehouse infrastructure.
• Follow established infrastructure standards and reference architectures.
• Administer ADLS, Databricks, and other Azure data services.
• Implement security measures, access controls, and data protection policies.
• Manage backup schedules, retention policies, tiering, and recovery procedures.
• Monitor pipelines, backups, and ensure successful backup/recovery activities.
• Monitor performance, availability, and health of cloud services.
• Identify and troubleshoot infrastructure, storage, connectivity, and performance issues.
• Optimize resource utilization and storage costs across Azure resources.
• Support administration of ADF, Databricks, and Synapse Analytics.
• Configure resources per organizational policies and CI/CD with Azure DevOps.
• Maintain documentation and operational procedures.

Skills

Azure cloud administration

Security & IAM

Backup & recovery

Performance monitoring

CI/CD with Azure DevOps

SQL & scripting (Python/Shell)

Cost optimization

Networking basics

Tools

Azure Data Lake Storage (ADLS)

Azure Databricks

Azure Data Factory (ADF)

Azure Synapse Analytics

Job description

We are looking for a skilled Data Engineer with hands‑on experience in managing, maintaining, and monitoring cloud‑based data lakehouse infrastructure, preferably on Microsoft Azure.

The ideal candidate should have a strong understanding of Azure Data Services, cloud infrastructure, data storage, security, backup management, and performance monitoring. The candidate will be responsible for ensuring the availability, security, reliability, and optimal performance of the data lakehouse environment.

Key Responsibilities

• Manage, administer, and maintain cloud infrastructure and services supporting the data lakehouse environment on Microsoft Azure.
• Follow and implement established infrastructure standards, guidelines, and reference architectures.
• Administer and manage Azure Data Lake Storage (ADLS), Azure Databricks, and other relevant Azure data services.
• Implement and maintain security measures, access controls, and data protection policies for the data lakehouse environment.
• Manage backup schedules, data retention policies, data tiering, and recovery procedures.
• Monitor data pipelines, review backup logs, and ensure the successful execution of backup and recovery activities.
• Monitor the performance, availability, and health of cloud infrastructure and data lakehouse services.
• Identify and troubleshoot infrastructure, storage, connectivity, and performance‑related issues.
• Monitor Azure resources and optimize resource utilization, storage capacity, and operational costs.
• Support the administration and maintenance of Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.
• Ensure proper configuration and management of cloud resources in accordance with organizational policies.
• Work with Azure DevOps for infrastructure deployment, configuration management, and CI/CD activities.
• Maintain technical documentation, operational procedures, backup records, and infrastructure configurations.
• Stay updated with industry trends, emerging technologies, and best practices to continuously improve data lakehouse infrastructure and capabilities.
• Perform any other duties and responsibilities as assigned.

Required Skills

• Minimum 3 years of relevant experience in Cloud Administration, Data Lakehouse Administration, or Azure Infrastructure Management.
• Hands‑on experience with Microsoft Azure cloud services.
• Good understanding of Azure Data Lake Storage (ADLS) and cloud storage management.
• Knowledge of Azure Databricks, Azure Data Factory (ADF), and Azure Synapse Analytics.
• Understanding of data lakehouse architecture and cloud‑based data platforms.
• Experience in cloud infrastructure monitoring, administration, and troubleshooting.
• Knowledge of cloud security, identity and access management (IAM), and role‑based access control (RBAC).
• Experience in backup management, data recovery, retention policies, and storage tiering.
• Understanding of performance monitoring, resource optimization, and capacity management.
• Knowledge of Azure Monitor, Log Analytics, and monitoring tools.
• Basic understanding of networking, virtual machines, storage, and other Azure infrastructure components.
• Familiarity with Azure DevOps, CI/CD, and deployment processes.
• Good understanding of SQL and basic scripting using Python or Shell.
• Strong analytical, troubleshooting, and problem‑solving skills.
• Ability to work independently and collaborate effectively with technical teams and stakeholders.

Good to Have

• Microsoft Azure certifications such as Azure Administrator Associate or Azure Solutions Architect.
• Experience managing enterprise‑level data lakehouse environments.
• Exposure to Azure security, governance, and compliance frameworks.
• Experience with infrastructure automation and Infrastructure as Code (IaC).
• Knowledge of disaster recovery planning and business continuity processes.
• Experience working with cross‑functional teams and client stakeholders.
• Understanding of data engineering workflows and end‑to‑end data platform operations.