Data Engineer / Support Analyst

ALLTECH CONSULTING SVC INC; New Jersey; On-site; USD 120,000 - 150,000; Full time

Job summary

ALLTECH CONSULTING SVC INC seeks an experienced Data Engineer / Support Analyst in New Jersey to manage and support data pipelines, integrate Bloomberg market data, and develop ETL processes using Azure and Python.

Ideal candidates have at least 10 years in financial services with strong technical skills in Databricks and Azure Data Factory. Excellent problem-solving and communication skills are essential for success in this pivotal role.

Qualifications

• 10+ years of experience in data engineering or production support within financial services.
• Hands-on experience with Azure Databricks, Azure Data Factory, and Azure Storage.
• Strong Python and SQL programming skills.

Responsibilities

• Manage and support batch processes and data pipelines in Azure Databricks.
• Integrate and process Bloomberg market data feeds into analytics platforms.
• Monitor and resolve data issues related to applications.

Skills

Expertise in Databricks

Azure cloud services

Bloomberg terminal

Data integration

ETL pipelines

Python programming

SQL programming

FTP/SFTP file transfers

Incident management

Stakeholder management

Tools

Azure Databricks

Azure Data Factory

Bloomberg data feeds

Git

CI/CD pipelines

Azure DevOps

Job description

Job Description:

Job Summary:
We are seeking an experienced Data Engineer / Support Analyst with strong expertise in Databricks, Azure cloud services, Bloomberg terminal and BBG data integration, and batch job support. The ideal candidate will have a background in financial markets, with hands-on experience managing data pipelines, FTP file transfers, and real-time/batch processing in a production environment.

Key Responsibilities:
* Manage and support batch processes and data pipelines in Azure Databricks and Azure Data Factory.
* Integrate and process Bloomberg market data feeds and files into analytics platforms.
* Monitor, troubleshoot, and resolve data and system issues related to applications and market data ingestion.
* Develop, automate, and optimize ETL pipelines using Python, Spark, and SQL.
* Manage FTP/SFTP file transfers between internal systems and external vendors.
* Ensure data quality, completeness, and timeliness for downstream reporting systems.
* Collaborate with desks, application support, and infrastructure teams to resolve incidents and enhance data workflows.
* Provide L2/L3 production support for critical market data systems.

Required Skills & Experience:
* 10+ years of experience in data engineering or production support within financial services.
* Hands-on experience with Azure Databricks, Azure Data Factory, and Azure Storage, Logic Apps, Fabric.
* Strong Python and SQL programming skills.
* Experience with Bloomberg data feeds (BPIPE, TSIP,SFTP, B-Quant).

* Experience with Git, CI/CD pipelines, and Azure DevOps.
* Proven ability to support batch jobs, troubleshoot failures, and manage job scheduling.
* Experience handling FTP/SFTP file transfers and automation (e.g., using scripts or managed file transfer tools).
* Solid understanding of market data concepts, and financial instruments.
* Excellent communication, problem-solving, and stakeholder management skills.

Nice to Have:
* Experience with Power BI or other visualization tools.
* Familiarity with Kafka, Event Hubs, or real-time streaming architectures.
* Knowledge of ITIL / incident management best practices.

* Knowledge of building AI Agents and LLMs