Data Engineer - Databricks

Cloud Analytics Technologies, LLC; New York (NY); Sur place; USD 140 000 - 190 000; Plein temps

Résumé du poste

Cloud Analytics Technologies, LLC in New York seeks a senior data engineer to manage batch processes and data pipelines in Azure Databricks and Data Factory, and to integrate Bloomberg feeds for trading/analytics platforms.

You will develop and optimize ETL pipelines with Python, Spark, and SQL, oversee secure FTP/SFTP transfers, and ensure data quality for downstream systems. Collaboration with operations and infrastructure teams is essential.

Qualifications

• Hands-on experience with Azure Databricks and Data Factory in financial services.
• Strong Python and SQL programming skills with data engineering focus.
• Experience with Bloomberg data feeds (BPIPE, TSIP, SFTP) required.

Responsabilités

• Manage and support batch processes and data pipelines in Azure Databricks and Azure Data Factory.
• Integrate Bloomberg market data feeds and files into trading or analytics platforms.
• Monitor, troubleshoot, and resolve data and system issues related to trading applications and market data ingestion.
• Develop, automate, and optimize ETL pipelines using Python, Spark, and SQL.
• Manage FTP/SFTP file transfers between internal systems and external vendors.
• Ensure data quality, completeness, and timeliness for downstream trading and reporting systems.
• Collaborate with operations, application support, and infrastructure teams to resolve incidents and enhance data workflows.

Connaissances

Azure Databricks

Azure Data Factory

Python

SQL

Bloomberg data feeds

Data engineering

Troubleshooting

Outils

Azure Storage

Logic Apps

Fabric

SFTP

Description du poste

Job description
Manage and support batch processes and data pipelines in Azure Databricks and Azure Data Factory.
Integrate and process Bloomberg market data feeds and files into trading or analytics platforms.
Monitor, troubleshoot, and resolve data and system issues related to trading applications and market data ingestion.
Develop, automate, and optimize ETL pipelines using Python, Spark, and SQL.
Manage FTP/SFTP file transfers between internal systems and external vendors.
Ensure data quality, completeness, and timeliness for downstream trading and reporting systems.
Collaborate with operations, application support, and infrastructure teams to resolve incidents and enhance data workflows.
10+ years of experience in data engineering or production support within financial services or trading environments.
Hands-on experience with Azure Databricks, Azure Data Factory, and Azure Storage, Logic Apps, Fabric.
Strong Python and SQL programming skills.
Experience with Bloomberg data feeds (BPIPE, TSIP,SFTP).

CLOUD ANALYTICS TECHNOLOGIES is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Equal Opportunity Employer

CLOUD ANALYTICS TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Some of our Business Ventures - Knowledge Analytics

Do restrictive necessities, your capability for advanced analytics or your business model need custom-made applications and infrastructure configurations? opt for our hosted managed services to urge comprehensive analytics solutions supported industry-leading Cloud Analytics Technologies, LLC code - delivered by the Cloud Analytics Technologies, LLC consultants UN agency understand these solutions best. Through a mix of with-it technology and dedicated service and support, Cloud Analytics Technologies, LLC hosted managed services will facilitate you: