Senior Data Engineer - Databricks & Bloomberg Trading Data

Valiant Technologies LLC; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Valiant Technologies LLC in New York, NY seeks a senior Data Engineer specializing in financial services data pipelines. You will own batch processing, integration of Bloomberg market feeds, and end-to-end ETL automation using Python, Spark, and SQL.

You will monitor, troubleshoot, and optimize data workflows across trading platforms, data ingestion, and reporting systems, working with operations, application support, and infrastructure teams.

Qualifications

• 10+ years of data engineering or production support experience in financial services.
• Hands-on with Azure Databricks, Data Factory, and storage services.
• Strong Python and SQL programming skills.
• Experience with Bloomberg data feeds (BPIPE, TSIP, SFTP).
• Ability to monitor, troubleshoot, and optimize data pipelines.

Responsibilities

• Manage and support batch processes and data pipelines in Azure Databricks and Data Factory.
• Integrate Bloomberg feeds into trading or analytics platforms.
• Monitor and resolve issues in trading applications and data ingestion.
• Develop and automate ETL pipelines using Python, Spark, and SQL.
• Ensure data quality, timeliness and completeness for downstream systems.
• Collaborate with operations, support, and infrastructure teams.
• Manage FTP/SFTP file transfers between internal systems and external vendors.

Skills

Python

SQL

Azure Databricks

Azure Data Factory

Spark

ETL

Data quality

Tools

Azure Storage

Logic Apps

Fabric

Bloomberg feeds

Job description

Valiant Technologies LLC in New York, NY seeks a senior Data Engineer specializing in financial services data pipelines. You will own batch processing, integration of Bloomberg market feeds, and end-to-end ETL automation using Python, Spark, and SQL.

You will monitor, troubleshoot, and optimize data workflows across trading platforms, data ingestion, and reporting systems, working with operations, application support, and infrastructure teams.