Senior Azure Data Engineer

ESR Healthcare; Remote; USD 120,000 - 180,000; Full time

Job summary

An established industry player is seeking a highly experienced Azure Architect with over a decade of expertise in data architecture and cloud solutions. This role demands a deep understanding of Azure services and the ability to lead complex data governance initiatives. You will be responsible for designing and implementing scalable data solutions, ensuring seamless data migration, and fostering a collaborative team environment. If you thrive in a challenging, innovative setting and possess exceptional leadership skills, this opportunity is perfect for you to make a significant impact on transformative projects.

Qualifications

• 10+ years of experience in Azure, Big data, and data warehouse architecture.
• Proven ability to lead teams and manage client service delivery effectively.

Responsibilities

• Design and architect Azure Data Factory and Azure Databricks solutions.
• Lead the Data Governance Program and manage data migration processes.

Skills

ADF

Azure Data Bricks

PySpark

Spark SQL

PL/SQL

Python

Power BI

Java

C##

Unix shell/Perl scripting

Tools

Azure Data Factory

Azure Databricks

Azure SQL DB/DW

Data Factory

Azure Stream Analytics

Azure Analysis Service

HDInsight

Databricks

Azure Data Catalog

ML Studio

Job description

Experience level: 10+ years

Mandatory skills: ADF, Azure Data Bricks, PySpark, Spark SQL, PL/SQL, Python

Job Description:

Core Skills required: ADF, Azure Data Bricks, PySpark, Spark SQL, PL/SQL, Python

1. At least 5 years of consulting or client service delivery experience on Azure
2. At least 10 years of experience in Azure, Big data, database and data warehouse architecture and delivery
3. Minimum of 5 years of professional experience in 2 of the following areas:
1. Solution/technical architecture in the Azure cloud
2. Big Data/analytics/information analysis/database management in the cloud
3. IoT/event-driven/microservices in the cloud
4. Experience with private and public cloud architectures, pros/cons, and migration considerations
5. Design and architect an Azure Data Factory, Azure Databricks, Azure Data Lake
6. Lead the Data Governance Program
7. Extensive hands-on experience implementing data migration and data processing using Azure services: Serverless Architecture, Azure Storage, Azure SQL DB/DW, Data Factory, Azure Stream Analytics, Azure Analysis Service, HDInsight, Databricks, Azure Data Catalog, ML Studio, AI/ML, Azure Functions, ARM Templates, Azure DevOps, CI/CD, etc.
8. Cloud migration methodologies and processes including tools like Azure Data Factory, Event Hub, etc.
9. Familiarity with the Technology stack available in the industry for data management, data ingestion, capture, processing and curation: Kafka, StreamSets, Attunity, GoldenGate, Map Reduce, Hadoop, Hive, Hbase, Cassandra, Spark, Flume, Hive, Impala, etc.
10. Familiarity with Networking, Windows/Linux virtual machines, Container, Storage, ELB, AutoScaling is a plus
11. Experience developing and deploying ETL solutions on Azure
12. Strong in Power BI, Java, C##, Spark, PySpark, Unix shell/Perl scripting
13. Familiarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data Catalog, etc.
14. Multi-cloud experience a plus - Azure, AWS, Google
15. Proven ability to build, manage and foster a team-oriented environment
16. Proven ability to work creatively and analytically in a problem-solving environment
17. Excellent communication (written and oral) and interpersonal skills
18. Excellent leadership and management skills
19. Excellent organizational, multi-tasking, and time-management skills
20. Proven ability to work independently and lead the onsite offshore team towards the project plan