Data Engineer (BigQuery & MongoDB)

Demetria; São Paulo; Teletrabalho; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

Uma startup em crescimento que atua na interseção entre ciência de dados e agricultura busca um Engenheiro de Dados (BigQuery & MongoDB) para se juntar à equipe. O candidato ideal terá um forte histórico em design e manutenção de pipelines de dados, bem como experiência em ambientes de nuvem e tecnologias da informação. Com opções flexíveis de trabalho remoto, esta posição oferece um ambiente dinâmico para crescimento e desenvolvimento de carreira.

Serviços

Ambiente de trabalho flexível

Salário competitivo e benefícios

Oportunidades de crescimento e desenvolvimento

Qualificações

• FormaçãO superior em Ciência da Computação ou área relacionada.
• Experiência comprovada como Engenheiro de Dados ou função similar.
• Forte experiência em BigQuery e MongoDB.

Responsabilidades

• Projetar e manter pipelines de dados robustos para processar dados de múltiplas fontes.
• Colaborar com equipes multifuncionais para a integração de dados.
• Utilizar ferramentas de automação como Apache Airflow para garantir fluxo de dados.

Conhecimentos

SQL

BigQuery

MongoDB

Python

Docker

Kubernetes

Apache Airflow

Data Modeling

Data Governance

Formação académica

Bacharel em Ciência da Computação ou Sistemas de Informação

Descrição da oferta de emprego

Join to apply for the Data Engineer (BigQuery & MongoDB) role at Demetria

Continue with Google Continue with Google

2 days ago Be among the first 25 applicants

Join to apply for the Data Engineer (BigQuery & MongoDB) role at Demetria

Get AI-powered advice on this job and more exclusive features.

Sign in to access AI-powered advices

Continue with Google Continue with Google

Continue with Google Continue with Google

Continue with Google Continue with Google

Continue with Google Continue with Google

Continue with Google Continue with Google

Continue with Google Continue with Google

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• Proven experience as a Data Engineer or similar role, with a strong background in designing and maintaining data pipelines.
  
  

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• Proven experience as a Data Engineer or similar role, with a strong background in designing and maintaining data pipelines.
  
  

Proficiency in SQL and working with cloud-based data warehouses (BigQuery experience preferred).

• Experience with MongoDB for NoSQL database management.
• Strong experience with Docker for containerization, along with orchestration tools such as Kubernetes (preferred).
• Familiarity with workflow orchestration tools such as Apache Airflow or Prefect.
• Experience with Google Cloud Platform (GCP) services (Cloud Functions, Cloud Run, BigQuery, Dataflow).
• Strong programming skills in Python or other data-related programming languages.
• Experience working with large datasets, and data integration from multiple sources.
• Knowledge of data modeling, data governance, and data architecture best practices.
• Familiarity with CI/CD pipelines for automated deployment of data infrastructure.
  
  

11 Fluent English is a must

• Fluent Spanish and is a plus
  
  

Preferred Qualifications

Experience with data processing from IoT devices or sensors, particularly spectrometry or other scientific instrumentation.

Familiarity with data visualization tools like Tableau or Google Looker Studio.

Experience managing data in cloud environments like GCP, AWS, or Azure.

Previous experience in a startup environment, particularly in the agriculture or coffee industry, is a plus.

What We Offer

Opportunity to work with a passionate team in a rapidly growing startup at the intersection of data science and agriculture.

Flexible work environment, with remote work options.

Competitive salary and benefits.

Opportunities for career growth and development.

Key Responsibilities

Data Pipeline Development: Design, build, and maintain robust ETL pipelines to collect, process, and store data from various sources, including spectrometry data from our NIR scans, user feedback, and external data inputs.

Data Integration: Collaborate with cross-functional teams to integrate data from multiple sources, including MongoDB, BigQuery, APIs, and Excel files, ensuring unified access for the data science team.

Database Management: Set up and manage scalable data storage solutions using cloud-based data warehouses (BigQuery), as well as NoSQL databases (MongoDB) for real-time and unstructured data processing.

Containerization and Deployment: Use Docker to containerize data pipelines and services for smooth deployment across environments. Manage containers and orchestration for scalable and repeatable workflows.

Data Quality Assurance: Implement processes to clean, validate, and monitor data quality, ensuring consistency and accuracy across all datasets used by the data science and product teams.

Collaboration with Data Scientists: Work alongside data scientists to understand their data needs, enabling effective model development, testing, and deployment by providing clean, well-organized datasets.

Optimization and Scalability: Design data solutions that can scale with the business, ensuring fast, reliable data access for real-time analytics and model training.

Cloud Infrastructure Management: Work with Google Cloud Platform (GCP) to set up and manage cloud infrastructure for data storage and processing, optimizing the use of GCP services like Cloud Functions, Cloud Run, Google Dataflow and BigQuery.

Data Security and Compliance: Ensure data handling follows industry best practices for security, privacy, and compliance with applicable regulations.

Workflow Automation: Implement workflow automation and orchestration tools like Apache Airflow or Prefect to ensure smooth and consistent data flow across the entire pipeline.

A vaga aceita trabalho remoto?

Sim

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Software Development

Referrals increase your chances of interviewing at Demetria by 2x

Sign in to set job alerts for “Data Engineer” roles.

Continue with Google Continue with Google

Continue with Google Continue with Google

Software Engineer (Python) Career Opportunities at Dev.Pro - 01

Senior Data Engineer, Business Analytics - Brazil

Pleno Data Engineer - Expertise Databricks | Jupyter

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.