Senior Data Engineer — Remote & Flexible Hours

AgileEngine; Orlando (FL); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine in Orlando, FL is seeking a Senior Data Engineer to design scalable data lakes, warehouses, and lakehouses supporting a thematic research platform processing large volumes of financial data daily. You will implement Python-based ETL/ELT pipelines, orchestrate workflows with Airflow, develop ingestion from third‑party APIs, and work with Snowflake, Spark, and AWS to deliver high‑performance data infrastructure.

The role blends hands‑on engineering with technical consulting.

Qualifications

• US work authorization required
• Bachelor’s degree or equivalent in a technical field
• 5+ years of Python development experience
• 5+ years with data processing libraries (Pandas, Polars, PySpark or DuckDB)
• 2+ years with Spark and Snowflake
• Expert-level Airflow or similar orchestration
• Knowledge of Medallion Architecture and columnar file formats
• Experience with third-party API data ingestion
• Cloud proficiency with AWS, GCP, and Snowflake; strong SQL
• Strong cross-functional collaboration and communication skills
• Focus on data integrity and end-to-end pipeline quality
• Proactive mindset and AI tooling awareness

Responsibilities

• Design and implement Python data engineering solutions.
• Build scalable Data Lakes, Data Warehouses, and Data Lakehouses.
• Develop ETL/ELT pipelines with Python and Airflow.
• Create ingestion workflows from diverse 3rd party APIs.
• Optimize file formats and storage for performance.
• Collaborate with AI teams to accelerate analytics and ML.
• Translate business goals into data architecture roadmaps.
• Use Terraform to provision AWS/on-prem infrastructure

Skills

Python

Pandas

Polars

PySpark

DuckDB

Airflow

Snowflake

Spark

SQL

NoSQL

AWS

GCP

Education

Bachelor’s degree in Computer Science/Engineering or related field

Tools

Data pipelines

OpenSearch/Elasticsearch

Terraform

Job description

AgileEngine in Orlando, FL is seeking a Senior Data Engineer to design scalable data lakes, warehouses, and lakehouses supporting a thematic research platform processing large volumes of financial data daily. You will implement Python-based ETL/ELT pipelines, orchestrate workflows with Airflow, develop ingestion from third‑party APIs, and work with Snowflake, Spark, and AWS to deliver high‑performance data infrastructure.

The role blends hands‑on engineering with technical consulting.