Cloud Data Engineer - Azure & Databricks Pipelines

Talentify; Chicago (IL); On-site; USD 125,000 - 140,000; Full time

Job summary

Mesirow, an independent financial services firm based in Chicago, is seeking a data engineer to modernize our data architecture and build a cloud-native estate on Azure and Databricks. You will own ETL pipelines and enable data-driven decisions across the firm.

Collaborate with data stewards and product teams, apply security-by-design, and advance testing, CI/CD, and IaC practices in a fast-moving environment.

Qualifications

• 2–5 years of experience designing, developing, and deploying data pipelines and modern cloud data solutions.
• Strong proficiency in Python programming and PySpark.
• Hands-on experience with Databricks (notebooks, jobs, Delta Lake) and Unity Catalog.
• Experience building and operating orchestration pipelines with Azure Data Factory (ADF).
• DevOps experience, Git-based workflows and CI/CD.
• Self-starter with strong ownership, problem-solving skills, and the ability to manage multiple priorities in a fast-moving environment.
• Strong communication and collaboration skills, with the ability to translate business requirements into practical data solutions and work effectively with both technical and non-technical stakeholders.

Responsibilities

• Own technical solutions that evolve our data architecture/integration patterns while also solving for changing business needs.
• Develop and maintain ETL pipelines to transform raw data into usable formats for data analysis, reporting and a centralized data repository.
• Evolve our data pipeline infrastructure using cloud native tools such as Azure Data Factory, Databricks or similar toolsets.
• Collaborate with other data stewards, product and business team members to ensure that our data is accurate, reliable, and consistent.
• Participate in an agile team dynamic; own testing through test automation, drive DevOps capabilities and infrastructure-as-code practices to create repeatable and controlled deployments across environments.
• Build trusted, well-governed data products that can support reporting, analytics, automation, and emerging AI use cases.
• Apply security-by-design principles, including least-privilege access, secrets management, encryption, data classification, and appropriate protection of sensitive data.
• Partner with data owners and stewards to implement data quality, classification, lineage, access controls, and governance standards across the data platform.

Skills

Python programming

PySpark

Data governance

Communication

Ownership

Problem-solving

Collaboration

Tools

Databricks

Unity Catalog

Azure Data Factory

CI/CD

Git

Job description

Mesirow, an independent financial services firm based in Chicago, is seeking a data engineer to modernize our data architecture and build a cloud-native estate on Azure and Databricks. You will own ETL pipelines and enable data-driven decisions across the firm.

Collaborate with data stewards and product teams, apply security-by-design, and advance testing, CI/CD, and IaC practices in a fast-moving environment.