Talent Pipeline for opportunities within Data Engineering

Data Science Careers; København; On-site; DKK 600,000 - 900,000; Full time

Job summary

A prominent data engineering firm in Copenhagen is seeking a skilled Data Engineer to design and develop data handling systems. The ideal candidate will excel in building data ingestion pipelines, transforming data for analysis, and optimizing performance in cloud environments. Strong proficiency in programming languages such as Python and SQL, as well as familiarity with ETL tools like Apache Airflow, is required. This is a great opportunity to contribute to data-driven projects in a collaborative environment.

Qualifications

• Proficiency in programming languages like Python, Java, Scala, or SQL.
• Familiarity with ETL tools such as Apache NiFi, Apache Airflow, and Talend.
• Knowledge of databases (SQL and NoSQL) and data warehouses like Amazon Redshift and Snowflake.
• Understanding of big data frameworks such as Apache Hadoop, Apache Spark, and Apache Kafka.
• Experience with cloud platforms like AWS, Azure, or Google Cloud.

Responsibilities

• Build data pipelines to ingest data from various sources.
• Clean, transform, and structure raw data for analysis.
• Select and manage appropriate storage solutions.
• Design and implement data processing pipelines.
• Create data models and schemas for data analysts and scientists.

Skills

Python

Java

Scala

SQL

Apache NiFi

Apache Airflow

Talend

Informatica

Amazon Redshift

Snowflake

Hadoop

AWS S3

Apache Spark

Apache Kafka

Git

Job description

Talent Pipeline for opportunities within Data Engineering

Copenhagen, Capital Region of Denmark, Denmark

About the job Talent Pipeline for opportunities within Data Engineering

Sign up for our talent pipeline for a job within Data Engineering.

A data engineer is a professional who specializes in the design, development, and maintenance of systems and infrastructure for handling and processing data. Their primary role is to ensure that data is collected, stored, and made available for analysis in a reliable, efficient, and scalable manner. Data engineers play a crucial role in enabling data-driven decision-making within organizations. Here are some key responsibilities and skills associated with a data engineer:

Responsibilities:

• Data Ingestion: Data engineers are responsible for building pipelines to ingest data from various sources, such as databases, external APIs, logs, and streaming platforms.
• Data Transformation: They clean, transform, and structure raw data to make it suitable for analysis. This process involves data cleansing, normalization, and data quality assurance.
• Data Storage: Data engineers select and manage appropriate storage solutions, including relational databases, NoSQL databases, data warehouses, or data lakes, to store and organize the data securely.
• Data Processing: They design and implement data processing pipelines to perform operations like aggregation, filtering, and data enrichment. This ensures that data is prepared for analysis efficiently.
• Data Modeling: Data engineers create data models and schemas that define the structure of the data, making it easier for data analysts and data scientists to work with the data.
• ETL (Extract, Transform, Load): Building and maintaining ETL processes is a core responsibility, where data is extracted from source systems, transformed, and then loaded into the target storage.
• Data Pipeline Orchestration: They manage and monitor data pipelines, ensuring that they run reliably and efficiently. Automation and scheduling are critical for data pipeline management.
• Data Quality Assurance: Data engineers implement data validation and data quality checks to identify and rectify inconsistencies or errors in the data.
• Performance Optimization: Optimizing data storage and processing for performance and cost efficiency is essential, particularly in large-scale data environments.
• Collaboration: Data engineers collaborate closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

Skills and Tools:

• Programming Languages: Proficiency in languages like Python, Java, Scala, or SQL for data processing and pipeline development.
• Data Integration Tools: Familiarity with ETL tools like Apache NiFi, Apache Airflow, Talend, or Informatica.
• Data Storage Technologies: Knowledge of databases (SQL and NoSQL), data warehouses (e.g., Amazon Redshift, Snowflake), and data lake solutions (e.g., Hadoop, AWS S3).
• Big Data Technologies: Understanding of big data frameworks such as Apache Hadoop, Apache Spark, and Apache Kafka.
• Cloud Platforms: Experience with cloud platforms like AWS, Azure, or Google Cloud for scalable and cost-effective data storage and processing.
• Version Control: Proficiency in using version control systems like Git for code management.
• Data Modeling: Knowledge of data modeling techniques and tools such as ER diagrams, and familiarity with tools like Apache Avro and Apache Parquet for data serialization.
• Data Quality and Monitoring: Tools and practices for data quality assurance, monitoring, and alerting.
• Problem-Solving: Strong problem-solving skills to troubleshoot and optimize data pipelines.

Submit your LinkedIn profile or Resume and we will contact you when we see a match with your experience and the recruitment needs from our customers.