Data Engineer

AgileEngine, LLC.

Brasil

Remote

BRL 120,000 - 240,000

Full time

39 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Learning budget
Competitive compensation
100% remote

Job summary

AgileEngine, LLC. is seeking a Data Engineer to build pipelines, storage, and retrieval systems for a data platform powering AI-driven forecasts. The role requires 5+ years of Python software engineering, strong SQL, data modeling, and experience with production ingestion pipelines and RAG.

You will design scalable data storage, indexing, and graph-based retrieval, and support AI agents with embedding and retrieval workflows. Remote work is available for Brazil-based candidates.

Qualifications

  • 5+ years of experience with Python software engineering, including testing and production debugging.
  • Strong SQL, database and data modeling skills, with large datasets optimization.
  • Experience building production ingestion and transformation pipelines with monitoring.
  • Experience processing unstructured data, including embeddings and retrieval.
  • Practical experience with vector databases or vector search systems for retrieval quality.

Responsibilities

  • Build reliable ingestion pipelines, handle updates, deduplication, and backfills.
  • Design the data platform for storage and indexing supporting SQL and vector search.
  • Make data useful to AI agents with document processing and embedding pipelines.
  • Implement bitemporal models and point-in-time queries to prevent data leakage.
  • Own data quality, provenance, schemas, lineage, and timestamp semantics.
  • Improve performance, cost, and access controls across datasets and workflows.

Skills

Python engineering
SQL
Data modeling
Unstructured data
RAG
Vector databases
English (upper-intermediate)

Tools

Databricks
Snowflake
AWS
Vector search

Job description

About the role

We are looking for a Data Engineer to build the pipelines, storage, and retrieval systems behind a data platform that helps AI agents produce better forecasts.

The mandatory requirements are 5+ years of experience with Python software engineering, strong SQL, database and data modeling skills, production ingestion pipelines, unstructured data processing and RAG, vector databases or vector search, and an upper-intermediate English level.

What you will do
  • Build reliable ingestion pipelines. Acquire and process diverse datasets, handling incremental updates, deduplication, schema changes, retries, backfills, and monitoring.
  • Design the data platform. Develop storage and indexing architectures that support SQL queries, keyword and vector search, and graph-based retrieval.
  • Make data useful to AI agents. Build document processing, chunking, embedding, and retrieval pipelines that deliver relevant context quickly and efficiently.
  • Support historical reasoning and evaluation. Implement bitemporal models and point-in-time queries that distinguish when something happened from when it became known, preventing future information from leaking into historical tests.
  • Own data quality and provenance. Define schemas, validation rules, source attribution, lineage, and timestamp semantics. Investigate inconsistencies and make data limitations visible to downstream users.
  • Improve performance, cost, and access controls. Optimize queries and pipelines, measure retrieval quality, and provide secure access across datasets and agent workflows.
Must haves
  • 5+ years of experience with Python software engineering, including testing, maintainable system design, and production debugging.
  • Strong SQL, database, and data modeling skills, with experience optimizing queries and working with large datasets.
  • Experience building and operating production ingestion and transformation pipelines, including monitoring and recovery from partial failures.
  • Experience processing unstructured data, including document extraction, chunking, embeddings, and retrieval-augmented generation (RAG).
  • Practical experience with vector databases or vector search systems, including indexing, metadata filtering, and retrieval performance. This includes methods to tune search quality and relevance.
  • Careful judgment about data correctness and meaning: timestamps, units, identifiers, missing values, revisions, and source reliability.
  • The ability to take an ambiguous requirement, make architectural tradeoffs, and deliver a working system.
  • Upper-intermediate English level.
Nice to haves
  • Databricks, Snowflake, or comparable lakehouse and warehouse platforms.
  • AWS infrastructure, object storage, queues, and distributed processing.
  • Financial, economic, news, or other frequently revised datasets.
  • Bitemporal modeling, historical snapshots, and reproducible backtesting.
  • Search relevance, hybrid retrieval, and RAG evaluation.
  • Graph databases, entity resolution, and knowledge graphs.
  • TurboPuffer or other vector databases.
Perks
  • Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
  • Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
  • Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
  • Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
  • Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
  • Well-being & support: access local well-being programs and people-focused support tailored to your location
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Belo Horizonte

Hybrid
BRL 494,000 - 713,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Sorocaba

Hybrid
BRL 603,000 - 933,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Campinas

Hybrid
BRL 494,000 - 713,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • São Paulo

Hybrid
BRL 494,000 - 658,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine • Recife

Hybrid
BRL 384,000 - 494,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Rio de Janeiro

Hybrid
BRL 494,000 - 713,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine • São Bernardo do Campo

Remote
BRL 494,000 - 658,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Salvador

Remote
BRL 494,000 - 713,000
Professional growth
Competitive USD pay
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Florianópolis

Hybrid
BRL 439,000 - 658,000
Professional growth
Competitive compensation
Exciting projects
+1
Data Engineer ID99039
Data Engineer ID99039

AgileEngine, LLC. • Curitiba

Hybrid
BRL 603,000 - 905,000
Professional growth
Competitive compensation
Exciting projects
+1