Azure Data Engineer/azure data architect/Lead data engineer

Tata Consultancy Services; Hyderabad, Chennai District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services in Hyderabad seeks a Lead Data Engineer to guide a team, design ELT pipelines, and oversee data architecture across Azure and Google cloud platforms. You will drive CI/CD releases, ensure data quality, and translate complex data requirements for stakeholders.

Strong experience in data modeling, data warehouse design, and Master Data Management is required, along with leadership and communication skills for high-tempo projects.

Qualifications

• Experience managing Azure Data Lake structure and security.
• Proven data modeling experience with star and snowflake schemas.
• Ability to translate business requirements from non-technical stakeholders.
• Experience across full development lifecycle from architecture to deployment.
• Experience in Master Data Management solutions.
• Experience in data warehouse analysis and design with data warehousing methods.
• Unit testing of SQL scripts, stored procedures, ETL/ELT modules.
• Knowledge of Agile methodology and software development lifecycle.
• GIT, Release Management and Azure DevOps knowledge.
• Experience with cloud data platforms (Azure and Google) and data governance tools.

Responsibilities

• Lead and mentor junior data engineers.
• Design, build and deploy ELT pipelines and data management processes.
• Operationalise ELT job workflows with reliable error handling and rollback framework.
• Oversee data architecture and pipelines platforms, managing data flow from sources.
• Develop, configure, and document database architecture.
• Monitor and improve data loading performance.
• Work with CI/CD pipelines to manage releases.
• Communicate technical terms to non-technical stakeholders; explain change rationale.
• Follow agile evolution: design, refactor, CI/CD, TDD.
• Collaborate with Business Partner Team and Stakeholders as SME; identify risks and mitigations.
• Provide production support for data load jobs.
• Write customized queries to generate periodic reports.
• Maintain or upgrade existing applications.

Skills

Leadership

Data modeling

SQL scripting

ETL/ELT design

Communication

Agile methods

Tools

Azure Data Lake

Azure Synapse

Azure Data Factory

Azure Purview

Azure Batch

Azure DevOps

Google BigQuery

Google Cloud Run

Job description

Role & responsibilities

• Lead and mentor junior data engineers
• Design, build and deploy ELT pipelines and data management processes.
• Operationalise ELT job workflow with reliable error/exception handling and rollback framework.
• Build and oversee data architecture & pipelines platforms and systems, managing the flow of data from internal and external sources by leveraging both distributed and local structures.
• Participate in developing, designing, configuring, and documenting database architecture.
• Monitor and improve data loading performance.
• Working with CI/CD pipelines to manage and execute releases
• Clearly communicate technical terms to non-technical people and help them understand why change might be required to achieve a specific goal or to complete a project
• Conform to agile development practices Evolutionary design, refactoring, continuous integration/delivery, test-driven development.
• Work collaboratively with Business Partner Team and Business Stakeholder during projects scoping and feasibility phases as a SME for concept investigation/viability and technical impact; Identify and communicate technical risks and mitigations.
• Provide production support for data load jobs.
• Write customized query to generate automatic periodic reports.
• Maintain or upgrade existing applications.

Attend key design meetings and provide support.

Preferred candidate profile

• Experience in managing Azure Data Lake structure and security
• Proven experience with data modeling, and working with star schema, snowflake schema
• Ability to understand and communicate business requirements from non-technical stakeholders while simultaneously managing their expectations.
• Experience with the full development life cycle of an application stack - from architecture through test and deployment.
• Experience in Master Data management solutions.
• Experience in data warehouse analysis and design, with full knowledge of data warehouse methodologies and data modelling.
• Unit testing low level components SQL scripts, Stored Procedures, ETL/ELT modules; Integration testing data transition points; Performance, load and stress testing.
• Knowledge and experience in software development methodology and toolset for implementing, lifecycle and management - Agile methodology.
• GIT code repository and branching strategies; Microsoft Release Management and any deployment tools; Azure DevOps knowledge.
• Experience with Azure cloud solutions; Knowledge of Azure Data Factory, Azure Automation, Azure Data Lake, Azure Synapse, Azure Purview, Azure KeyVault, and Azure Batch.
• Experience with Google Cloud Platform solutions; knowledge of Google BigQuery, Google Storage, and Google CloudRun
• Experience with APIs.
• Team player
• Organized and detail oriented
• Excellent communication, presentation, and writing skills
• Thrives in a fast paced, constantly changing environment