Senior Data Engineer with Data Science/MLOps background

N-iX; Brasil; Teletrabalho; BRL 20.000 - 80.000; Tempo integral

Resumo da oferta

N-iX is seeking a proactive Senior Data Engineer to join its vibrant team in Brazil. This role involves designing and maintaining sophisticated data pipelines, leveraging extensive knowledge in data architecture, machine learning, and cloud technologies to support AI-driven initiatives. Ideal candidates will demonstrate strong problem-solving skills and a passion for tackling complex data challenges, contributing to impactful data solutions.

Serviços

Flexible working format

Competitive salary

Personalized career growth

Professional development tools

Education reimbursement

Corporate events and team buildings

Qualificações

• 5+ years of experience in data engineering.
• Proficient in Python and PySpark.
• Experience with cloud-based data services.

Responsabilidades

• Design and maintain scalable data pipelines in Palantir Foundry.
• Collaborate with teams to optimize data workflows.
• Monitor and troubleshoot data pipeline performance.

Conhecimentos

Python

PySpark

Data Modeling

ETL/ELT Concepts

Cloud Technologies

Machine Learning

Data Governance

Analytical Skills

Formação académica

Degree in Computer Science or related field

Ferramentas

Palantir Foundry

AWS Glue

Azure Data Factory

Docker

Kubernetes

Descrição da oferta de emprego

Senior Data Engineer with Data Science/MLOps background

Senior Data Engineer with Data Science/MLOps background

3 weeks ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

Senior Data Engineer

We are seeking a proactive Senior Data Engineer to join our vibrant team. As a Senior Data Engineer, you will play a critical role in designing, developing, and maintaining sophisticated data pipelines, Ontology Objects, and Foundry Functions within Palantir Foundry. Your background in machine learning and data science will be valuable in optimizing data workflows, enabling efficient model deployment, and supporting AI-driven initiatives. The ideal candidate will possess a robust background in cloud technologies, data architecture, and a passion for solving complex data challenges.

Senior Data Engineer

We are seeking a proactive Senior Data Engineer to join our vibrant team. As a Senior Data Engineer, you will play a critical role in designing, developing, and maintaining sophisticated data pipelines, Ontology Objects, and Foundry Functions within Palantir Foundry. Your background in machine learning and data science will be valuable in optimizing data workflows, enabling efficient model deployment, and supporting AI-driven initiatives. The ideal candidate will possess a robust background in cloud technologies, data architecture, and a passion for solving complex data challenges.

Key Responsibilities:

• Collaborate with cross-functional teams to understand data requirements, and design, implement and maintain scalable data pipelines in Palantir Foundry, ensuring end-to-end data integrity and optimizing workflows.
• Gather and translate data requirements into robust and efficient solutions, leveraging your expertise in cloud-based data engineering. Create data models, schemas, and flow diagrams to guide development.
• Develop, implement, optimize and maintain efficient and reliable data pipelines and ETL/ELT processes to collect, process, and integrate data to ensure timely and accurate data delivery to various business applications, while implementing data governance and security best practices to safeguard sensitive information.
• Monitor data pipeline performance, identify bottlenecks, and implement improvements to optimize data processing speed and reduce latency.
• Collaborate with Data Scientists to facilitate model deployment and integration into production environments.
• Support the implementation of basic ML Ops practices, such as model versioning and monitoring.
• Assist in optimizing data pipelines to improve machine learning workflows.
• Troubleshoot and resolve issues related to data pipelines, ensuring continuous data availability and reliability to support data-driven decision-making processes.
• Stay current with emerging technologies and industry trends, incorporating innovative solutions into data engineering practices, and effectively document and communicate technical solutions and processes.
  
  

Tools and skills you will use in this role:

• Palantir Foundry
• Python
• PySpark
• SQL
• TypeScript
  
  

Required:

• 5+ years of experience in data engineering, preferably within the pharmaceutical or life sciences industry;
• Strong proficiency in Python and PySpark;
• Proficiency with big data technologies (e.g., Apache Hadoop, Spark, Kafka, BigQuery, etc.);
• Hands-on experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow);
• Expertise in data modeling, data warehousing, and ETL/ELT concepts;
• Hands-on experience with database systems (e.g., PostgreSQL, MySQL, NoSQL, etc.);
• Proficiency in containerization technologies (e.g., Docker, Kubernetes);
• Familiarity with ML Ops concepts, including model deployment and monitoring.
• Basic understanding of machine learning frameworks such as TensorFlow or PyTorch.
• Exposure to cloud-based AI/ML services (e.g., AWS SageMaker, Azure ML, Google Vertex AI).
• Experience working with feature engineering and data preparation for machine learning models.
• Effective problem-solving and analytical skills, coupled with excellent communication and collaboration abilities;
• Strong communication and teamwork abilities;
• Understanding of data security and privacy best practices;
• Strong mathematical, statistical, and algorithmic skills.
  
  

Nice to have:

• Certification in Cloud platforms, or related areas;
• Experience with search engine Apache Lucene, Webservice Rest API;
• Familiarity with Veeva CRM, Reltio, SAP, and/or Palantir Foundry;
• Knowledge of pharmaceutical industry regulations, such as data privacy laws, is advantageous;
• Previous experience working with JavaScript and TypeScript.
  
  

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits
• not applicable for freelancers

Seniority level

• Seniority level

  Not Applicable

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology and Engineering

• Industries

  Real Estate, Financial Services, and Capital Markets

Referrals increase your chances of interviewing at N-iX by 2x

Get notified about new Senior Data Engineer jobs in Brazil.

[DATA PRODUCTS MISC] Senior Business Analyst

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.