Azure Data Engineer (1385)

Sharp Decisions; Charlotte (NC); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Sharp Decisions is seeking a Senior Data Engineer/Quality Engineer to join our Data Strategy team in a hybrid role in Charlotte, NC. You will help establish a reference data system for securities and pricing data and work on Azure-based data platform initiatives.

Ideal candidates have 10+ years in Python/PySpark, strong SQL, and hands-on experience with Azure Data Factory, Databricks, and Collibra data quality/governance. Financial services experience is a plus.

Qualifications

• 10+ years of software/data engineering experience required.
• Expertise with Python/PySpark and Azure cloud services required.
• Experience with data quality, governance tools (Collibra) and data lineage is a plus.

Responsibilities

• Lead the development of a reference data system for securities and pricing data.
• Contribute to the Azure-based data platform and data strategy initiatives.
• Collaborate with SMBC/Nikko teams to implement standards and governance.

Skills

Python

PySpark

Azure Cloud

Azure Data Factory

Azure Lake Gen 2

Azure Databricks

Power BI

SQL

DevOps CI/CD

ETL/ELT

Data Quality

Data Governance

Data Lineage

Collibra

Financial services

Tools

Git

Jenkins

Job description

RESOURCE TYPE: W2 Only

LOCATION: Hybrid - Charlotte, NC

Experience Level: Senior (10+ Years)

Role Objectives

• These roles will be part of the Data Strategy team spanning across the SMBC Capital Markets and Nikko securities teams, SMBC Americas Division’s broker-dealer and swap-dealer entities.
• These roles will be involved in the active development of the data platform in close coordination with the SMBC team, beginning with the establishment of a reference data system for securities and pricing data, and later moving to other data domains.
• The consulting team will need to follow internal developments standards to contribute to the overall agenda of the Data Strategy team.
• The implementation of this strategic platform on SMBC’s Azure Cloud Platform will require solutions and know-how as listed in the qualifications below.

Qualifications and Skills

• 10+ years of experience in software development using Python, PySpark and its frameworks.
• Proven experience as a Data Engineer/Data Quality Engineer with experience in Azure cloud.
• Experience implementing solutions using -
• Azure cloud services
• Azure Data Factory
• Azure Lake Gen 2
• Azure Databases
• Azure Databricks
• Data Quality Framework/Solution
• Collibra Data Quality or any Data Quality tools (IDQ/IBM/Ab-Initio)
• Collibra Data Governance and Data lineage knowledge.
• Power BI
• Well versed with Azure Databricks
• Strong SQL skills with RDMS or noSQL databases
• Familiarity with the DevOps lifecycle (git, Jenkins, etc.),CI/CD processes
• Good understanding of ETL/ELTprocesses
• Experience in financial services industry, financial
• instruments, asset classes and market data are a plus.