Sr. Data Engineer-Java

Highbrow LLC; Toronto; Hybrid; CAD 100,000 - 130,000; Full time

Job summary

A leading technology firm is seeking a Sr. Data Engineer proficient in Java and Big Data to enhance their data pipeline architecture. This role focuses on ensuring data quality, optimizing performance, and supporting strategic solutions within a collaborative Agile environment. Ideal candidates have over 10 years of experience and a strong background in both relational and NoSQL databases.

Qualifications

• 10+ years of relevant experience in data engineering.
• Experience designing scalable applications and microservices.
• 5+ years of Capital Markets experience.

Responsibilities

• Create and maintain optimal data pipeline architecture.
• Ensure data quality and integrity across various data sources.
• Optimize data pipelines for performance and scalability.

Skills

Java

Big Data

Hadoop

Spark

Kafka

Agile

Job description

Job Title: Sr. Data Engineer-Java

Job ID: 2024-12900

Job Location: Toronto, ON (Hybrid – 2 days/week onsite)

Job Travel Location(s):

# Positions: 2

Employment Type: W2

Duration:Long Term

# of Layers:0

Work Eligibility:All Work Authorizations are Permitted – No Visa Transfers

Key Technology:Java, Big Data

Job Responsibilities:

• Create and maintain optimal data pipeline architecture
• Implement data products curated by our Chief Data Office, as well as custom data models for fit for use.
• Ensure data quality and integrity across various data sources and systems to ensure data accuracy, completeness, and reliability.
• Optimize data pipelines for performance and scalability.
• Provide technical support to promptly resolve escalated incidents/outages.
• Develop and document a detailed solution design, impart your subject matter expertise throughout life cycle.
• Take business, Enterprise Architecture, system performance and development standards requirements, then develop functional, technical and user interface designs for an application and/or system.
• Find ways to keep costs low, help come up with strategic solutions to support cost effectiveness and enhance stakeholder experience.
• Conduct code reviews to address quality, standards compliance, reusability and ease of maintenance, Operational Readiness Reviews, and support gating and review signoffs for solution design.
• Ensure design leverages existing reusable components, traces back to business requirements, and that new modules are designed with reusability in mind.
• Keep up to date with the latest industry trends and technologies related to data engineering.

Skills and Experience Required:

Required:

• 10+ years of relevant experience in a related field of job function.
• Experience with: Java, Spring,
• Experience with big data tools: Hadoop, HDFS, ADLS, ADF, Spark, Kafka, Databricks, Dremio etc.
• Experience with relational SQL and NoSQL databases, including Cassandra.
• Experience designing production grade, scalable applications and microservices.
• 5+ years of Capital Markets experience.
• Experience working on Agile Teams

Desired:

• Experience in Python and/or Scala