Azure Data Engineer (1385)

Sharp Decisions; Charlotte (NC); Hybrid; USD 140,000 - 200,000; Full time

Job summary

Sharp Decisions seeks a Senior Data Engineer to lead the development of a reference data platform for securities and pricing data in a hybrid Charlotte, NC environment. You will partner with SMBC teams to design and implement Azure-based data services and governance capabilities.

The role requires 10+ years in Python/PySpark with strong SQL and data quality governance experience, including Azure Databricks and Power BI. AW2-backed engagement with global data initiatives is expected.

Qualifications

• 10+ years of software development experience with Python and PySpark.

Responsibilities

• Develop and advance the reference data platform for securities and pricing data.
• Collaborate with SMBC teams to implement an Azure-based data platform.
• Adhere to internal standards and contribute to the Data Strategy roadmap.

Skills

Python

PySpark

Azure

Azure Data Factory

Azure Databricks

Data Quality

Data Governance

Collibra Data Quality

Collibra Data Governance

Power BI

SQL

NoSQL

DevOps

CI/CD

ETL/ELT

Tools

Azure

Azure Data Factory

Azure Lake Gen 2

Azure Databases

Azure Databricks

Git

Jenkins

Job description

RESOURCE TYPE: W2 Only
LOCATION:Hybrid - Charlotte, NC
Experience Level: Senior (10+ Years)

Role Objectives

• These roles will be part of the Data Strategy team spanning across the SMBC Capital Markets and Nikko securities teams, SMBC Americas Division’s broker-dealer and swap-dealer entities.
• These roles will be involved in the active development of the data platform in close coordination with the SMBC team, beginning with the establishment of a reference data system for securities and pricing data, and later moving to other data domains.
• The consulting team will need to follow internal developments standards to contribute to the overall agenda of the Data Strategy team.
• The implementation of this strategic platform on SMBC’s Azure Cloud Platform will require solutions and know-how as listed in the qualifications below.

Qualifications and Skills

• 10+ years of experience in software development using Python, PySpark and its frameworks.
• Proven experience as a Data Engineer/Data Quality Engineer with experience in Azure cloud.
• Experience implementing solutions using -
  
  • Azure cloud services
  • Azure Data Factory
  • Azure Lake Gen 2
  • Azure Databases
  • Azure Databricks
  • Data Quality Framework/Solution
  • Collibra Data Quality or any Data Quality tools (IDQ/IBM/Ab-Initio)
  • Collibra Data Governance and Data lineage knowledge.
  • Power BI
• Well versed with Azure Databricks
• Strong SQL skills with RDMS or noSQL databases
• Familiarity with the DevOps lifecycle (git, Jenkins, etc.),CI/CD processes
• Good understanding of ETL/ELTprocesses
• Experience in financial services industry, financial
• instruments, asset classes and market data are a plus.