Azure Data Factory with Data Bricks

BinTech Group; Jersey City (NJ); On-site; USD 125,000 - 160,000; Full time

Job summary

BinTech Group in Jersey City, NJ is seeking a Senior Azure Cloud Engineer to design and implement data processing solutions on Azure Databricks. You will own large parts of projects from inception to production, collaborating with architects, engineers, data scientists and business stakeholders to deliver enterprise-grade data platforms.

Responsibilities include building batch and real-time pipelines, integrating diverse data sources, and advancing patterns for streaming analytics on Azure.

Qualifications

• Experience building data pipelines in Azure cloud and Databricks.
• Ability to own projects end-to-end with stakeholders.
• Strong collaboration with architects, engineers and data scientists.

Responsibilities

• Develop notebooks in Azure Databricks using Python, R, SQL.
• Build large-scale batch and real-time data pipelines with data processing frameworks in Azure cloud platform.
• Partner with architects, engineers, information analysts, business, and technology stakeholders for developing and deploying enterprise grade platforms that enable data-driven solutions.
• Sr. Azure Cloud Engineer coordinating with platform, APIM, Azure DevOps, and Support.
• Own projects end-to-end with business stakeholders, IT teams, data stewards, and 3rd party companies.
• Enable streaming data use-cases and standard design patterns in a big data environment.
• Collaborate with data scientists to integrate algorithms and models into automated processes.
• Review and provide feedback on core code changes and support production deployment.
• Provide technical guidance on technology implementation and architectural changes.
• Analyze and design technical solutions on Azure to address business needs.
• Develop, test, and modify software to improve efficiency of data platforms and applications.
• Integrate data from a variety of sources, ensuring data quality and accessibility.

Skills

Azure Databricks

Python

R

SQL

Azure Data Factory

Azure DevOps

Big data

Tools

Azure Functions

Azure Data Lake Storage

Azure Event Grid

Azure Monitor

Azure Service Bus

SQL Server

Oracle/PL-SQL

Job description

PRIMARY RESPONSIBILITIES ( Azure DataBricks)

The job responsibilities are described herein:

• Develop notebooks in Azure Databrciks using Python, R, SQL programming languages.
• Build large-scale batch and real-time data pipelines with data processing frameworks in Azure cloud platform.
• Partner with architects, engineers, information analysts, business, and technology stakeholders for developing and deploying enterprise grade platforms that enable data-driven solutions.
• Sr. Azure Cloud Engineer coordinating with platform, APIM, Azure DevOps, and Support.
• Ability to own projects from end to end, working with business stakeholders, IT teams, data stewards, and 3rd party companies to ensure accurate and insightful outcomes.
• Enable streaming data use-cases and standard design patterns in a big data environment.
• Collaborating with data scientists to integrate algorithms and models into automated processes. Applying expertise, judgment, and precedents to contribute to the resolution of moderately complex problems.
• Review and provide feedback on core code changes and support production deployment.

Provide technical guidance on technology implementation and architectural changes.

Analyze and design technical solutions on Azure to address business needs.

Develop, test, and modify software to improve efficiency of data platforms and applications.

Integrating data from a variety of sources, assuring that they adhere to data quality and accessibility standards.

CORE TECHNOLOGIES

Azure DataBricks, Proficiency in Python; R; Azure data factory, Azure Functions, Azure Data Lake storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, Azure Service Bus, Python, SQL Server, Oracle / PL-SQL, T-SQL and SQL stored procedures, Azure DevOpsx

PRIMARY RESPONSIBILITIES ( Azure DataBricks)

Azure DataBricks, Proficiency in Python; R; Azure data factory, Azure Functions, Azure Data Lake storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, Azure Service Bus, Python, SQL Server, Oracle / PL-SQL, T‑​​​​​​​…\u2026