Azure Databricks Data Engineer for Finance Pipelines & Bloomberg

Cloud Analytics Technologies, LLC; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Cloud Analytics Technologies, LLC in New York seeks a senior data engineer to manage batch processes and data pipelines in Azure Databricks and Data Factory, and to integrate Bloomberg feeds for trading/analytics platforms.

You will develop and optimize ETL pipelines with Python, Spark, and SQL, oversee secure FTP/SFTP transfers, and ensure data quality for downstream systems. Collaboration with operations and infrastructure teams is essential.

Qualifications

• Hands-on experience with Azure Databricks and Data Factory in financial services.
• Strong Python and SQL programming skills with data engineering focus.
• Experience with Bloomberg data feeds (BPIPE, TSIP, SFTP) required.

Responsibilities

• Manage and support batch processes and data pipelines in Azure Databricks and Azure Data Factory.
• Integrate Bloomberg market data feeds and files into trading or analytics platforms.
• Monitor, troubleshoot, and resolve data and system issues related to trading applications and market data ingestion.
• Develop, automate, and optimize ETL pipelines using Python, Spark, and SQL.
• Manage FTP/SFTP file transfers between internal systems and external vendors.
• Ensure data quality, completeness, and timeliness for downstream trading and reporting systems.
• Collaborate with operations, application support, and infrastructure teams to resolve incidents and enhance data workflows.

Skills

Azure Databricks

Azure Data Factory

Python

SQL

Bloomberg data feeds

Data engineering

Troubleshooting

Tools

Azure Storage

Logic Apps

Fabric

SFTP

Job description

Cloud Analytics Technologies, LLC in New York seeks a senior data engineer to manage batch processes and data pipelines in Azure Databricks and Data Factory, and to integrate Bloomberg feeds for trading/analytics platforms.

You will develop and optimize ETL pipelines with Python, Spark, and SQL, oversee secure FTP/SFTP transfers, and ensure data quality for downstream systems. Collaboration with operations and infrastructure teams is essential.