Data Engineer

Artihcus

Hyderabad

On-site

INR 1,200,000 - 1,800,000

Part time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

A technology consulting firm in Hyderabad is seeking a Mid-Senior level professional for a contract position in Azure DevOps. The role involves developing and implementing data pipelines, managing CI/CD processes, and ensuring quality in data transformation. Candidates should have expertise in Azure Data Factory and Databricks, with solid experience in the end-to-end ETL cycle. This position offers a chance to work in a dynamic environment focusing on business development within the IT industry.

Qualifications

Extensive knowledge in Azure data services and data pipelines.
Experience in handling ETL processes using Azure tools.
Ability to design and manage CI/CD pipelines.

Responsibilities

Develop and implement data pipelines.
Participate in team meetings to review project status.
Ensure compliance with change control processes.

Skills

Azure DevOps (CI/CD)

Spark

Scala

Data Factory

Databricks

Job description

Secondary - Azure DevOps (CI/CD), Spark, Scala

Responsibilities and Scope

Developing and implementing data pipelines to extract, transform and load ETL data from various sources into Azure data services. This includes using Azure Data Factory, Azure Databricks or other tools to orchestrate data workflows and data movement.
Build, Test and Run of data assets tied to tasks and user stories from the Azure DevOps instance of Enterprise Data Analytics.
Bring a level of technical expertise of the Big Data space that contributes to the strategic roadmaps for Enterprise Data Architecture, Global Data Cloud Architecture and Global Business Intelligence Architecture and contributes to the development of the broader Enterprise Data Analytics Engineering community.
Actively participate in regularly scheduled contact calls to transparently review the status of inflight projects, priorities of backlog projects, and review adoption of previous deliveries from Enterprise Data Analytics with the Enterprise Data Analytics management team.
Handle break fixes and participate in a rotational on‑call schedule. On‑call includes monitoring of scheduled jobs and ETL pipelines.
Actively participate in team meetings to transparently review the status of inflight projects and their progress.
Follow standard practice and frameworks on each project from development to testing and then productionizing each within the appropriate environment laid out by Data Architecture.
Challenge self and others to make an impact that matters and help team connect their contributions with broader purpose.
Set expectations to the team, align the work based on the strengths and competencies, and challenge them to raise the bar while providing the support.
Extensive knowledge of multiple technologies, tools and processes to improve the design and architecture of the assigned applications.
Design, implement and manage CI/CD pipelines for software development teams.
Understanding of web applications and CI/CD processes.
Knowledge sharing and documentation.
Contribute to produce and maintain processes, procedures, operational and architectural documentation.
Change Control: ensure compliance with processes and adherence to standards and documentation.
Work with Deloitte Technology leadership and service teams in reviewing documentation and aligning KPIs to critical steps in our service operations.
Active participation in ongoing training within the BI space.
Ingest the data into target from source using ADF or ADB.
Apply transformations if any based on the business requirements.
The candidate should have worked on end‑to‑end ETL cycle using ADF or ADB.
Source could be API and Database, Rest API, 90 Oracle etc.
Quality check to be done post data transfer.
Logic apps for SharePoint, Key Vault event triggers are used – good to have.
The candidate will have to work in IC role.