Senior Data Engineer (Databricks)

Anova; Porto; Híbrido; EUR 55 000 - 78 000; Tempo integral

Resumo da oferta

Anova is hiring a Data Engineer (Databricks) to join our R&D Software team in Porto. This hybrid role blends remote work with occasional in‑office collaboration to build reliable data assets used by reporting, analytics and ML across the company.

You’ll own end‑to‑end Databricks ETL, translate business goals into scalable data solutions, ensure data quality and governance, and collaborate across teams to deliver value with AI-assisted tooling.

Qualificações

• Bachelor's degree in a related field is required.
• 5+ years of experience in data engineering or a closely related role, with hands-on production experience in Databricks (6–8 years preferred).
• Significant experience building data workloads in Databricks, good understanding of PySpark and Delta Lake.
• Strong SQL with advanced features and query tuning.
• Experience with streaming or incremental ingestion (Structured Streaming, Auto Loader) and ensuring idempotency, checkpointing and schema evolution.
• Data modelling for BI and analytics.

Responsabilidades

• Build and run the Databricks pipelines that turn real-time telemetry and platform data into reliable, well-governed data assets.
• Deliver Databricks ETL projects end to end, from requirements through to pipelines running in production.
• Translate business goals into data solutions and help stakeholders make the right choices about data.
• Contribute to technical decisions, take a significant share of the implementation, and monitor pipelines once live.
• Publish and maintain data assets as master data for the organization.

Conhecimentos

SQL

PySpark

Data modelling

CI/CD

Streaming data

English proficiency

Git

Data quality testing

Agentic coding tools awareness

Formação académica

Bachelor's degree in Computer Science, Data Engineering, Data Science, or a related quantitative field

Ferramentas

Databricks

PySpark

Delta Lake

Descrição da oferta de emprego

Join us on the R&D Software team as a Data Engineer (Databricks), and help shape the future of safer, more efficient, and more reliable operations across the globe. Start your journey with Anova today!

Where you’ll work: This is a hybrid role based out of our Porto office. In practice, most of your work can be done remotely, with occasional in-office time in Porto for team collaboration — a flexibility our engineers consistently tell us they value.

Job Duties And Responsibilities

You will build and run the Databricks pipelines that turn real-time telemetry and platform data into reliable, well-governed data assets — the master data that reporting, analytics and machine learning across Anova all depend on.

Collaborate for success

• Deliver Databricks ETL projects end to end, from requirements through to pipelines running in production.
• Translate business goals into data solutions and help stakeholders make the right choices about data.
• Contribute to technical decisions, take a significant share of the implementation, and monitor the pipelines you own once they are live.

Build the One Anova data stream

• Work with real-time telemetry from industrial IoT sensors deployed across the globe.
• Build the BI aggregations that bring data from across platforms together into consistent, reusable data assets.
• Your pipelines are the backbone for our internal natural-language digital assets that lets any employee query Anova's data without writing SQL. The reliability, freshness and clarity of what you publish directly determines whether that experience can be trusted.
• Publish and maintain data assets as master data for the organization.

Engineer with AI assistance

• Use agentic coding tools — Claude Code, Copilot, Cursor and similar — as a normal part of daily delivery.
• Hold AI-generated code to the same bar as any other code. You are accountable for what you ship.
• Keep repositories, tests and documentation structured so both people and agents can work in them effectively.

Advocate for quality

• Contribute to and continuously adapt best practices and Ways of Working around data engineering, testing and pipeline operations.
• Maintain clear data lineage and definitions for the assets you own — as AI agents increasingly query this data directly, untraceable or ambiguous data becomes a governance risk, not just a data-quality one.
• Treat data quality as a feature: tests, expectations and monitoring, so problems surface before stakeholders find them.

Minimum Requirements -

• Bachelor's degree in Computer Science, Data Engineering, Data Science, or a related quantitative field or equivalent combination of education and experience
• 5+ years of experience in data engineering or a closely related role, with hands‑on production experience in Databricks (6–8 years preferred).
• Significant experience building data workloads in Databricks, with a very good understanding of PySpark and Delta Lake.
• Strong SQL — window functions, complex joins and query tuning are everyday tools for you.
• Experience with streaming or incremental ingestion (Structured Streaming, Auto Loader, or equivalent) and the patterns that keep it correct: idempotency, checkpointing and schema evolution.
• Data modelling for BI and analytics.
• Good understanding of testing and CI/CD for Databricks workflows, alongside the software engineering and DevOps basics — git, code review, linters, unit tests and CI/CD pipelines are things you use daily.
• Data quality practice: testing data as well as code, using pipeline expectations, dbt tests or similar.
• Comfortable using agentic coding tools, with a clear view of where they help and where they need supervision.
• Proficient in written and spoken English.

Preferred Qualifications -

• Databricks platform depth beyond the basics: Lakeflow pipelines (formerly Delta Live Tables), Lakeflow Jobs, Unity Catalog for governance and lineage, and infrastructure as code with Declarative Automation Bundles or Terraform.
• Performance and cost optimization on Databricks: cluster sizing, Photon, liquid clustering, and partitioning.
• The wider Azure data ecosystem: Event Hubs or Data Factory.
• Master data management or data governance practice: clear ownership, stewardship and agreed definitions for shared data assets.
• Domain experience in industrial, energy or IoT settings.