Kafka Data Engineer

ALLTECH CONSULTING SVC INC; Quebec; On-site; USD 56,919 - 78,263; Full time

Job summary

An innovative firm is seeking a Kafka Data Engineer to join their dynamic TEDRA team within the Institutional Securities Technology Division. This role involves working with transactional big data for real-time processing and contributing to the development of data stores and archives. The ideal candidate will thrive in an agile environment, demonstrating strong collaboration and ownership while delivering multiple projects across various business areas. If you're passionate about data engineering and eager to work on cutting-edge technology, this opportunity is perfect for you.

Qualifications

• 2-5 years of experience in data engineering and database technology.
• Strong knowledge of Kafka and scripting languages like Python.

Responsibilities

• Configure Kafka plants to support applications processing.
• Build monitoring tools to predict and prevent outages.
• Collaborate with teams to investigate production issues.

Skills

Kafka

Python

Unix / K-Shell

Agile Development

Data Engineering

Data Visualization

Machine Learning

Job description

Experience: 2-5 years

Business Background:
Position is for a role in TEDRA department. TEDRA is part of the Institutional Securities Technology (IST) Division. It is responsible for maintaining, distributing, and revenue, risk, and reference data (client, product, and pricing). As the authoritative source of key data sets, we are at the forefront of database technology and are heavily involved in data engineering, data science, data visualization, and machine learning efforts across the Firm.

Position Introduction:
This is a Kafka data engineer role in the team responsible for developing on the firm’s Capture data stores that holds the transactional big data for real-time and archive processing and getting it into the archives and data lake. The global team consists of highly technical team members who are adaptable to both hands-on development and project management. We deliver multiple projects for multiple business areas in parallel. The business owners and subject matter experts will be globally distributed, making communication and pro-activity important. You will be expected to work closely with our operations partners on requirements for projects.

The development will be performed using an agile methodology which is based on scrum (time boxing, daily scrum meetings, retrospectives, etc.) and XP (continuous integration, refactoring, unit testing, etc.) best practices. Candidates must therefore be able to work collaboratively, demonstrate good ownership, and be able to work well in teams.

Responsibilities:
– Configure and setup Kafka plants to support applications processing
– Build application specific monitoring and observability tools to help predict and prevent outages
– Collaborate with application and infrastructure teams to investigate production issues and provide solutions
– Provide support coverage during early Asia hours when needed

Technical Skills Requirement:
– Practiced knowledge of Kafka
– Strong with scripting languages such as: Python and Unix / K-Shell
– Knowledge of database concepts is desirable
– Familiar with lifecycle of data in an investment banking operation
– Experienced in Agile development process