Senior Data Engineer - Databricks, PySpark, Bloomberg

TechDigital Group; New York (NY); On-site; USD 80,000 - 120,000; Full time

Job summary

An established industry player is seeking a skilled data engineer to design and maintain scalable data pipelines using Azure technologies. This role involves integrating data from various sources into the Azure Databricks platform, implementing ETL processes, and ensuring data quality and integrity. The successful candidate will collaborate with data scientists and analysts to facilitate data-driven decision-making and assist with data visualization using Power BI. Join a dynamic team and contribute to impactful projects in a fast-paced environment where your expertise will drive innovation and success.

Qualifications

• Experience in developing scalable data pipelines using Azure technologies.
• Strong skills in data integration and ETL processes.

Responsibilities

• Design and maintain scalable data pipelines using Azure Databricks and Data Lake.
• Collaborate with data scientists to support data-driven decision-making.

Skills

Azure Data Bricks

Azure Data Lake

Python

Pyspark

Bloomberg data acquisition

Tools

Azure Data Factory

Power BI

Job description

Required skills and qualifications
An associates or technical certification in Electronics, preferred
Familiarity with equipment such as RF test equipment, controlled impedance measurements, Vector Network analyzer, thickness measurements, thermal conductivity measurements, optical microscopes, mechanical testing
Understanding basic electronic theory and material testing.
Demonstrate experience with lab-based equipment.
Attention to detail, good verbal, and written communication skills.
Ability to follow written and verbal instructions
Ability to work within a small team.
Some light manufacturing experience, a plus

Responsibilities:
Responsibilities

Testing materials for quality control and assurance

Basic electrical measurements using various instruments

Create reports based on measurements and data logging of items test

Maintenance of electronic equipment and other process analyzers.

Conduct testing per written SOP.

Organize data collection.

Ability to carry out tasks through multiple process steps, testing, and data reporting.

Work Hours
M-F 8 hours/ day ( flex time)
10 Paid Holidays