Python Data Engineer - Production Pipelines (Dublin)

Needl; Dublin; In loco; EUR 70.000 - 100.000; Tempo pieno

Descrizione del lavoro

Needl in Dublin is seeking a Software Engineer to join the Data Engineering team. The role focuses on building robust data processing systems with Python, designing scalable pipelines and handling large datasets in a production environment.

You will work with cloud-based data technologies, relational databases and distributed processing frameworks, contributing to reliable, high-performance data infrastructure.

Competenze

• 3-5+ years of professional software engineering experience building and maintaining production systems.
• Strong hands-on Python development in production-level codebases.
• Experience with at least one additional language such as Java, C# or Scala.
• Strong SQL skills and solid Relational DB knowledge.
• Experience designing, building or maintaining batch data processing pipelines.
• Experience troubleshooting and supporting production applications.
• Interest in building reliable, scalable data infrastructure.

Mansioni

• Design, develop and maintain scalable data pipelines and data processing applications.
• Build robust, production-quality software primarily using Python.
• Work with large datasets, relational databases and distributed processing systems.
• Develop and optimise batch processing workflows, ensuring reliability, performance and data integrity.
• Improve the efficiency, scalability and maintainability of existing data infrastructure.
• Work with modern cloud-based data technologies and distributed computing frameworks.
• Collaborate with engineers and stakeholders to deliver high-quality solutions.
• Take ownership of code throughout the SDLC, including testing and deployment.

Conoscenze

Python development

SQL

Production software

Batch data processing

Problem solving

Distributed data technologies

Strumenti

Databricks

Delta Lake

Apache Airflow

Cloud platforms

Descrizione del lavoro

Needl in Dublin is seeking a Software Engineer to join the Data Engineering team. The role focuses on building robust data processing systems with Python, designing scalable pipelines and handling large datasets in a production environment.

You will work with cloud-based data technologies, relational databases and distributed processing frameworks, contributing to reliable, high-performance data infrastructure.