65039 Data Migration Lead (with Databricks/Azure)

Half the Sky Australia; Singapore; On-site; SGD 130,000 - 180,000; Full time

Benefits offered by this job

Employee share purchasing

Flexible work arrangements

Diversity and inclusion

Competitive pay

Job summary

Avanade is seeking a Data Migration Lead to architect and drive large-scale data migration programs using Databricks and Azure. You will design data pipelines, data streams, and APIs, while guiding a team of data engineers and analysts.

Strong Azure Data Factory/Data Lake/Databricks expertise is essential, with a track record of delivering enterprise-scale data solutions. The role requires extensive experience in cloud data architectures, governance, and collaboration with business stakeholders

Qualifications

• 12+ years of relevant professional experience
• 4+ years architecting & implementing large-scale data projects through Azure Data services
• Strong collaboration and communication skills
• Ability to translate business use cases into technical solutions
• Data profiling, cataloging, and mapping for technical design

Responsibilities

• Translate business requirements to technical solutions leveraging strong business acumen
• Support planning and implementation of data design services, providing sizing and configuration assistance
• Deliver data transformations and modernizations of enterprise data solutions using Azure data technologies
• Design and build modern data pipelines and data streams
• Expose data to end users via Power BI, Azure API Apps, or other visualization platforms
• Provide technical direction to data engineers, data analysts, data integration developers, and DBAs
• Ensure quality of architecture and design deliverables
• Prepare High-Level Design (HLD) documents
• Manage and resolve design integration risks with stakeholders
• Develop data warehouse schematics and architectures for data access and analytics
• Travel as needed

Skills

Azure Data Services

Business Acumen

Communication

Data Governance

CI/CD

Tools

Azure Data Factory

Azure Data Lake

Azure SQL DW

Azure SQL

Databricks

PowerShell

ARM templates

Hadoop

Spark

ETL/ELT tools (AWS Glue, Talend)

MS SQL

DynamoDB

Job description

65039 Data Migration Lead (with Databricks/Azure)

As a Data Engineering Manager at Avanade, you will use comprehensive modern data engineer techniques and methods with Advanced Analytics to support business decisions for Avanade clients. You can collect, aggregate, and analyze structured/unstructured data from multiple internal and external sources and patterns, insights, and trends to decision-makers. Your goal is to support the use of data-driven insights to help our Clients achieve business outcomes and objectives. In this role, you will collect, aggregate, store, and reconcile data in support of Client business decisions. You will help design and build data pipelines, data streams, reporting tools, information dashboards, data service APIs, data generators and other end-user information portals and insight tools. You will be a critical part of the data supply chain, ensuring that stakeholders can access and manipulate data for routine and ad hoc analysis to drive business outcomes using Advanced Analytics.

Key Role Responsibilities:

Day-to-day, you will

• Translate business requirements to technical solutions leveraging strong business acumen
• Analyzes current business practices, processes, and procedures and identifies future business opportunities for leveraging Microsoft Azure Data & Analytics Services.
• Support the planning and implementation of data design services, providing sizing and configuration assistance and performing needs assessments.
• Delivery of data transformations and modernizations of enterprise data solutions using Azure cloud data technologies.
• Design and Build Modern Data Pipelines and Data Streams.
• Design and Build Data Service APIs
• Responsible for designing and providing technical direction to data engineers, data analysts, data integration developers, and database administrators.
• Responsible for the quality assurance of all architecture and design deliverables (e.g., technical documents)
• Responsible for preparing High-Level Design (HLD) document
• Manage, highlight, and resolve any design integration risks/issues with relevant business and stakeholder
• Develop and maintain data warehouse schematics, layouts, architectures, and relational/non-relational databases for data access and Advanced Analytics.
• Expose data to end users using PowerBI, Azure API Apps, or any other modern visualization platform or experience.
• Implement effective metrics and monitoring processes.
• Manage Data Governance in the enterprise and improve data governance, data quality, and master data practice
• Act as an Advisor for Data Governance processes, frameworks, standards, and architectures
• Travel as needed

Key Role Skill & Capability Requirements:

Your technical and non-technical skills include:

• Minimum 12 years experience with 4+ years of experience working with large complex enterprises, architecting & implementing large-scale data projects through Azure Data services
• Enterprise-scale technical experience with cloud and hybrid infrastructures, architecture designs, data migrations, and technology management. required
• Collaboration and Communication. Known for driving decisions collaboratively, resolving conflicts, and ensuring follow-through with exceptional verbal and written communication skills.
• Demonstrated experience in turning business use cases and requirements into technical solutions.
• Ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flow.
• Ability to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration.
• Expert knowledge of Azure Data Factory, Azure Data Lake, Azure SQL DW, Azure SQL, and Data Bricks is required.
• Knowledge of Azure IoT, Azure HDInsight + Spark, Azure Cosmos DB, Azure Stream Analytics & Python is a plus.
• End-to-end data migration from legacy data platform to cloud data services experience is plus
• Experience in Big Data management & processing using tools such as Hadoop, Spark
• Experience in Data Transformation using ETL/ELT tools such as AWS Glue, Factory, Talend, EAI
• Experience in database and data warehousing with SQL, NoSQL & RDBMS such as MS SQL, AWS DynamoDB, etc.
• Knowledge of Lambda and Kappa architecture patterns.
• Knowledge of Master Data Management (MDM) and Data Quality tools and processes
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals.
• Working experience with Visual Studio, PowerShell Scripting, and ARM templates
• Ability to orchestrate, lead, and influence remote teams, ensuring successful implementation of customer projects

Preferred Certifications:

Preferred Years of Work Experience:

You likely have about 12+ years of relevant professional experience.

Preferred Years of Management Experience:

5+ years managing & leadership of 3 or more associates in management consulting or industry.

What we offer you

Come for the distinctive experiences you have helping forward-thinking corporations, non-profits, and governments push the boundaries of digital innovation. Stay for the limitless learning opportunities that encourage you to master Microsoft and pursue big ideas. Enjoy ambitious growth for yourself as part of Avanades people-first culture with benefits like employee share purchasing, flexible work arrangements, a commitment to diversity and inclusion, and competitive pay.