Azure Data Engineer

Sharp Decisions; Charlotte, Northern (NC, KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Sharp Decisions is seeking a Senior Data Engineer to lead the development of a data platform on Azure for securities and pricing data, collaborating with SMBC teams. The role involves building reference data systems, ensuring data quality, governance, and lineage across domains.

Candidates should have 10+ years in software/Data engineering, strong Azure and PySpark skills, and experience with ETL/ELT and BI tools. Hybrid work in Charlotte, NC is offered.

Qualifications

• 10+ years of software development experience with Python/PySpark.
• Proven Data Engineer/Data Quality Engineer with Azure cloud experience.
• Experience implementing Azure services: Data Factory, Lake Gen 2, Databases, Databricks.
• Knowledge of Collibra Data Quality/Governance and data lineage.
• Strong SQL skills with RDMS/NoSQL databases and Power BI familiarity.
• Familiar with DevOps lifecycle and CI/CD processes; ETL/ELT expertise.
• Experience in financial services domain is a plus.

Responsibilities

• Develop a reference data platform for securities and pricing data on Azure.
• Coordinate with SMBC teams to implement the data strategy roadmap.
• Ensure adherence to internal standards and data governance practices.

Skills

Python

PySpark

Azure Cloud

Azure Data Factory

Azure Data Lake Gen 2

Azure Databricks

SQL

NoSQL

Data Quality

Data Governance

Data Lineage

Power BI

CI/CD

ETL/ELT

DevOps

Financial Services

Tools

Azure Data Factory

Collibra

Jenkins

Git

Job description

RESOURCE TYPE: W2 Only
LOCATION:Hybrid – Charlotte, NC
Experience Level: Senior (10+ Years)

Role Objectives

• These roles will be part of the Data Strategy team spanning across the SMBC Capital Markets and Nikko securities teams, SMBC Americas Division’s broker-dealer and swap-dealer entities.
• These roles will be involved in the active development of the data platform in close coordination with the SMBC team, beginning with the establishment of a reference data system for securities and pricing data, and later moving to other data domains.
• The consulting team will need to follow internal developments standards to contribute to the overall agenda of the Data Strategy team.
• The implementation of this strategic platform on SMBC’s Azure Cloud Platform will require solutions and know-how as listed in the qualifications below.

Qualifications and Skills

• 10+ years of experience in software development using Python, PySpark and its frameworks.
• Proven experience as a Data Engineer/Data Quality Engineer with experience in Azure cloud.
• Experience implementing solutions using –
  • Azure cloud services
  • Azure Data Factory
  • Azure Lake Gen 2
  • Azure Databases
  • Azure Databricks
  • Data Quality Framework/Solution
  • Collibra Data Quality or any Data Quality tools (IDQ/IBM/Ab-Initio)
  • Collibra Data Governance and Data lineage knowledge.
  • Power BI
• Well versed with Azure Databricks
• Strong SQL skills with RDMS or noSQL databases
• Familiarity with the DevOps lifecycle (git, Jenkins, etc.),CI/CD processes
• Good understanding of ETL/ELTprocesses
• Experience in financial services industry, financial
• instruments, asset classes and market data are a plus.