Databricks Data Engineer – Scalable Pipelines & Governance

Amalgamated Bank Of Ny; New York (NY); Hybrid; USD 155,000 - 170,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Amalgamated Bank Of Ny seeks a Data Engineer to design and maintain scalable data pipelines on Databricks, enabling advanced analytics across the organization.

The ideal candidate has 2+ years of Data Engineering experience, strong SQL/Python/Spark skills, and familiarity with Azure cloud services. The role emphasizes data integrity, security, and collaboration with data scientists and stakeholders.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or equivalent experience.
• 2+ years of experience as a Data Engineer, preferred proven expertise in using Databricks.
• Strong proficiency in data processing tools and languages such as SQL, Python, and Spark.
• Experience with cloud platforms, particularly Azure, and integration with Databricks.
• Understanding of data architecture principles and experience with ETL tools and methodologies.
• Excellent problem-solving skills, with the ability to manage multiple priorities in a fast-paced environment.
• Strong communication skills, with the ability to collaborate effectively with cross-functional teams.

Responsibilities

• Design and develop scalable data pipelines using Databricks, ensuring efficient data processing and integration from various sources.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet their needs.
• Implement best practices for data management, storage, and retrieval, focusing on optimizing performance and ensuring data integrity.
• Work with large datasets and perform data cleansing, transformation, and aggregation to support analytics and reporting functions.
• Monitor and troubleshoot data pipeline performance, ensuring high availability and reliability.
• Stay updated with the latest advancements in data engineering and Databricks technologies to continuously improve our data infrastructure.
• Document data processes and workflows, ensuring transparency and knowledge sharing across the team.
• Ensure data security and compliance with regulatory requirements and industry best practices.
• Perform other duties as directed.

Skills

Databricks

SQL

Python

Spark

Data pipelines

Data cleansing

Excellent communication

Education

Bachelor’s degree in Computer Science, Information Technology, Engineering, or equivalent

Tools

Databricks

Spark

Azure

Job description

Amalgamated Bank Of Ny seeks a Data Engineer to design and maintain scalable data pipelines on Databricks, enabling advanced analytics across the organization.

The ideal candidate has 2+ years of Data Engineering experience, strong SQL/Python/Spark skills, and familiarity with Azure cloud services. The role emphasizes data integrity, security, and collaboration with data scientists and stakeholders.