Somente para BR [COPE] - Senior Data Engineer

FoodChain ID Group, Inc; Brasil; Presencial; BRL 120.000 - 180.000; Tempo integral

Resumo da oferta

A leading company in the food technology sector seeks a Data Engineer skilled in Python and SQL to optimize large datasets and enhance data processing capabilities. The ideal candidate will have significant experience in data modeling and troubleshooting within an Agile environment. This role involves collaborating with various stakeholders and requires strong technical expertise to lead initiatives in data engineering.

Qualificações

• Minimum 7 years' experience using Python for large datasets.
• Proficiency in Python 3+ and libraries (Pandas, NumPy, SciPy).
• Deep SQL experience with Oracle, Postgres, or MS SQL Server.

Responsabilidades

• Create, extract, transform, and optimize datasets using Python and SQL.
• Develop and maintain data models, schemas, and documentation.
• Collaborate with data scientists and API developers.

Conhecimentos

Python

SQL

Data Modeling

Data Warehousing

Troubleshooting

Agile Development

Formação académica

College degree in computer science, software development, or related field

Ferramentas

MongoDB

AWS

Azure

Terraform

Azure DevOps

Descrição da oferta de emprego

· Use Python and SQL to efficiently create, extract, transform, and optimize very large datasets.

· Troubleshoot data and data processes and develop solutions for improvement.

· Develop and maintain data models, schemas, and documentation.

· Contribute as a hands-on engineer while sometimes taking the lead in the technical design and solution.

· Collaborate daily with data scientists, food scientists, product owners, team members, API developers, architects, and DevOps engineers.

· Advise other developers, both junior and senior, and share best practices.

This job description is not intended to be all-inclusive. Employees may be asked to perform additional tasks as required, that may or may not be related to their primary duties.

Required Qualifications:

· Minimum 7 years’ experience using Python for analyzing, extracting, creating, and transforming large datasets.

· Proficiency in Python 3+ and common Python libraries for data engineering and data science, specifically Pandas, NumPy, and SciPy.

· Deep experience with SQL and relational data using Oracle, Postgres, or MS SQL Server.

· Solid understanding of database design principles, data modeling, and data warehousing concepts.

· Excellent troubleshooting skills and instincts.

· Curious, self-motivated, and self-directed; comfortable working within an Agile software development team with short, iterative delivery cycles.

· College degree or equivalent experience in computer science, software development, engineering, information systems, math, food science, or other applicable field of study.

Preferred Qualifications:

· NoSQL database design and development using MongoDB, AWS DynamoDB, or Azure Cosmos DB.

· Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and services related to data storage/processing

· Exposure to Terraform or other Infrastructure-as-Code tooling.

· Proficient in Azure DevOps for source code and pipeline management.