Data Engineering Specialist

Caderno Nacional; Brasília; Presencial; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A tech-driven company in Brasília is looking for a Data Architect to design and maintain data pipelines for threat intelligence. Candidates should have strong Python skills, experience with data validation, and familiarity with orchestration tools. Join a collaborative team where you can work on challenging projects that drive business growth.

Qualificações

• Proven experience as a Data Architect or similar role.
• Familiar with data pipelines and cloud platforms.
• Excellent attention to detail.

Responsabilidades

• Design and develop data pipelines for threat intelligence.
• Implement data validation processes for quality assurance.
• Collaborate with analysts on data requirements.

Conhecimentos

Data validation processes

Data pipeline optimization

Problem-solving skills

Strong proficiency in Python

Communication and collaboration skills

Ferramentas

Apache Airflow

Docker

Kubernetes

AWS

Azure

Descrição da oferta de emprego

Responsibilities

• Design and Development: Design, develop, and maintain data pipelines for ingesting threat intelligence data from various sources into our data ecosystem.
• Data Validation: Implement data validation processes to ensure data accuracy, completeness, and consistency.
• Collaboration with Analysts: Collaborate with analysts to understand data requirements and design appropriate solutions.
• Automation: Develop automation scripts and workflows for data export processes to external systems or partners.
• Pipeline Optimization: Optimize and enhance existing data pipelines for improved performance and scalability.
• Troubleshooting: Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity.
• Documentation: Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
• Guidance: Provide guidance and support to other team members on data architecture best practices and methodologies.

Requirements

• Experience: Proven experience as a Data Architect or similar role, with a focus on data ingest, validation, and export automation.
• Python Proficiency: Strong proficiency in Python.
• Data Pipeline Orchestration Tools: Experience with data pipeline orchestration tools such as Apache Airflow, Apache NiFi, or similar.
• Cloud Platforms: Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform.
• Data Validation Techniques: Experience with data validation techniques and tools for ensuring data quality.
• Containerization Technologies: Experience building and deploying images using containerization technologies such as Docker and Kubernetes.
• Problem-Solving Skills: Excellent problem-solving skills and attention to detail.
• Communication and Collaboration Skills: Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Why Work With Us

• As a member of our team, you will have the opportunity to work on challenging projects that drive business growth.
• We offer a collaborative and dynamic work environment, with opportunities for professional development and advancement.

How to Apply

If you are interested in this position, please submit your resume and a cover letter outlining your qualifications and experience.

We look forward to hearing from you!