Azure Senior Data Engineer - S

Tata Consultancy Services; Mumbai; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Tata Consultancy Services in India is seeking an experienced Azure Senior Data Engineer to design and implement end-to-end data pipelines. You will collaborate with engineers, researchers, and data scientists to gather requirements, build scalable solutions, and optimize performance using Databricks, PySpark, and Azure services.

The role requires 8-12 years of experience, strong Python, SQL, and Agile practices, with a focus on data lake architectures.

Qualifications

• Develop and implement data pipelines using Databricks, ADF, Python, and related libraries.
• Strong knowledge of Pandas, NumPy, PySpark, and SQL.
• Working knowledge in DevOps and Databricks for real-world problems.
• Experience with Azure cloud data lake storage and Azure services.
• SQL for ETL processes and data extraction, transformation, loading.
• Team player with Agile experience and familiarity with JIRA/ADLM tools.
• Ability to prioritize, status, and multitask.

Responsibilities

• Collaborate with engineers, researchers, and data scientists to gather requirements and develop end-to-end pipelines.
• Independently develop, code or modify, test, and debug applications of varying complexity.
• Conduct experiments and data analyses to ensure robustness, scalability, and accuracy.
• Optimize and fine-tune pipelines for performance and efficiency.

Skills

Databricks

ADF

Python

Pandas

NumPy

PySpark

SQL

JIRA

ADLM Tools

Azure

DevOps

Agile

Tools

Databricks

Azure

Python

SQL

Job description

Role- Azure Senior Data Engineer

Desired Experience Range- 8-12 Years

Location- Chennai, Bangalore, Kolkata, Pune, Hyderabad, Delhi.

Must Have-

• Develop and implement Data pipelines and algorithms using Databricks, ADF, Python programming language, jointly with knowledge about Pandas, NumPy, Pyspark, SQL and similar libraries.
• Should have working knowledge in devops and should be well versed in Databricks to solve real-world problems.
• Should have working knowledge in Azure cloud data lake storage and azure services.
• Utilize SQL for data Extraction, Transformation, and Loading (ETL) processes.
• Work well within a team environment.
• Familiar with Agile development methodology
• Experience with JIRA, ADLM Tools
• Ability to prioritize, status, and multitask.

Roles & Responsibility -

• Work closely with engineers, researchers, and data scientists to gather requirements, perform data analysis, and develop end-to-end pipelines.
• Independently develop, code or modify, test, and debug application programs of varying degree of complexity
• Conduct experiments and perform data analysis to ensure the robustness, scalability, and accuracy of provided solutions
• Optimize and fine-tune for performance and efficiency.