Senior Data Engineer - Scalable AI & Big Data

Hackajob Ltd; Belfast City District; On-site; GBP 61,000 - 101,000; Full time

Job summary

Hackajob Ltd is recruiting a data engineering specialist to design and develop scalable data processing components and persistence layers for production-grade systems. You will work in agile teams, providing leadership on major data-centric components and ensuring reliable delivery in demanding environments.

Proficiency in Java/Scala/Python, SQL, and experience with cloud data platforms (AWS/Azure/GCP/Databricks) are essential, as is familiarity with streaming architectures like Kafka.

Qualifications

• Strong software development experience in Java, Scala, or Python.
• Experience with data-processing platforms from AWS, Azure, GCP, or Databricks.
• Experience building large-scale data processing components for production systems.
• Proficient in SQL and SQL extensions for analytical queries.
• Familiarity with ETL/ELT data processing pipelines and design patterns.
• Knowledge of distributed data processing frameworks, data stores, and formats.
• Ability to write quality, testable code and experience with automated testing.
• Experience with CI/CD techniques and practices.
• Interest in AI technologies.

Responsibilities

• Design and develop data processing and data persistence software components for data-scale solutions.
• Work in agile teams and provide leadership on significant data-system components.
• Design and develop large-scale data processing software for production environments.
• Develop data processing software for deployment in big data technologies.
• Deliver full software lifecycle: design, code, test, and defect resolution.
• Collaborate with architects and engineers to meet non-functional needs.
• Maintain high code quality and work with operations for readiness.
• Advise on effort estimates and technical implications of user stories.
• Coach and mentor team members.

Skills

Java

Scala

Python

SQL

Testing

CI/CD

AI interest

Performance tuning

Data visualization

Data transformations

Data best practices

Tools

AWS

Azure

GCP

Databricks

Kafka

Job description

Hackajob Ltd is recruiting a data engineering specialist to design and develop scalable data processing components and persistence layers for production-grade systems. You will work in agile teams, providing leadership on major data-centric components and ensuring reliable delivery in demanding environments.

Proficiency in Java/Scala/Python, SQL, and experience with cloud data platforms (AWS/Azure/GCP/Databricks) are essential, as is familiarity with streaming architectures like Kafka.