Mid-level Data Analyst

Jobtailor; São Paulo; Presencial; BRL 180 000 - 260 000; Tempo integral

Resumo da oferta

Jobtailor in São Paulo is seeking a Senior Data Engineer to lead the preparation, cleaning, and organization of data for analytical use. You will develop robust data pipelines with PySpark, Scala Spark, and Apache NiFi, while ensuring governance, quality, and LGPD compliance.

You will work closely with the development team and Product Owner to translate complex business rules into efficient data structures and supply reliable data for strategic decisions.

Qualificações

• Develop, optimize, and maintain complex SQL and HQL queries for large-scale ETL.

Responsabilidades

• Lead data pipelines development & maintenance using PySpark, Scala Spark, and NiFi.
• Ensure data governance, quality, and LGPD compliance.
• Collaborate with development team and Product Owner.
• Handle data quality, validation, and real-time processing with Kafka.
• Translate business rules into efficient data structures to support decision making.

Conhecimentos

Data Pipeline Development

SQL & HQL

Dimensional Modeling

Data Quality Tools

LGPD Compliance

Databricks

Ferramentas

PySpark

Scala Spark

Apache NiFi

Jupyter Notebooks

Databricks

Kafka

Descrição da oferta de emprego

• Will take a leading role in preparing, cleaning, sanitizing and organizing data for analytical use.
• Will lead the development and maintenance of robust data pipelines using PySpark, Scala Spark and Apache NiFi.
• Will ensure consistency, governance and reliability of information through the implementation of validation techniques and data quality tools.
• Will work closely with the development team and the Product Owner.

Requirements

• Responsible for developing, optimizing and maintaining complex SQL and HQL queries for extracting, transforming and manipulating large-scale data.
• Proficient in structuring Data Warehouse, Data Lake and Lakehouse environments, autonomously applying advanced concepts of dimensional modeling, partitioning and relational organization within the Medallion Architecture.
• Ensures consistency, governance and reliability of information through the implementation of validation techniques and data quality tools, also ensuring compliance with LGPD (Brazilian General Data Protection Law) guidelines and principles.
• Works on analytical data pipelines using Jupyter Notebooks and the Databricks platform, with familiarity with real-time processing using data streaming tools such as Kafka.
• Works closely with the development team and the Product Owner (PO), translating complex business rules and requirements into efficient data structures that directly support strategic decision-making.
• Performs corrective and evolutionary maintenance safely, ensuring proper closure and handover of services to technical and business stakeholders.
• Requires a strong analytical mindset, sharp critical thinking for evaluating data scenarios, clear communication and a deep business-oriented perspective.

Core Competencies

Demonstrates expertise in developing and maintaining data pipelines using PySpark, Scala Spark, and Apache NiFi, while ensuring data quality and compliance with LGPD guidelines. Proficient in SQL and HQL for large-scale data manipulation and structuring Data Warehouse, Data Lake, and Lakehouse environments.

Highest-signal resume keywords

• Data Pipeline Development
• SQL and HQL Query Optimization
• Data Warehouse and Lakehouse Structuring
• Data Quality Tools Implementation
• LGPD Compliance

Hard Skills

• PySpark
• Scala Spark
• Apache NiFi
• SQL
• HQL
• Dimensional Modeling
• Data Quality Tools
• Jupyter Notebooks
• Databricks
• Kafka

Soft Skills

• Analytical Mindset
• Critical Thinking
• Clear Communication
• Business-Oriented Perspective

Industry Keywords

• Data Governance
• Data Consistency
• Data Validation
• Medallion Architecture
• Real-Time Processing