Senior Data Engineer - Data Lakes & AI/ML Infra (Remote)

AgileEngine; Austin (TX); Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Professional growth

Competitive pay (USD)

Exciting projects

Flexible schedule

Job summary

AgileEngine in Austin is seeking a Senior Data Engineer to design and build scalable data lakes, data warehouses, and lakehouses supporting a thematic research platform for financial data. You will implement Python ETL/ELT pipelines, orchestrate with Airflow, and work with Snowflake, Spark, and AWS to deliver high-performance data infrastructure.

The role combines hands-on engineering with technical consulting, translating business goals into data architecture roadmaps and collaborating across

Qualifications

• 5+ years of Python experience in data engineering.
• 5+ years of data processing libraries (Pandas, PySpark, DuckDB, Polars).
• 2+ years with Big Data technologies (Spark, Snowflake).
• Expert-level pipeline orchestration with Airflow.
• Strong communication and cross-functional collaboration.

Responsibilities

• Design and implement Python data engineering solutions.
• Build scalable Data Lakes, Data Warehouses, and Data Lakehouses.
• Develop ETL/ELT processes at scale using Python and Airflow.
• Create ingestion workflows from diverse 3rd party APIs.
• Collaborate with stakeholders to translate business goals into roadmaps.

Skills

Python

Data processing

Airflow

SQL

Team collaboration

English (Upper)

Education

Bachelor's degree in CS/Engineering

Tools

Spark

Snowflake

Terraform

OpenSearch

Elasticsearch

AWS

Job description

AgileEngine in Austin is seeking a Senior Data Engineer to design and build scalable data lakes, data warehouses, and lakehouses supporting a thematic research platform for financial data. You will implement Python ETL/ELT pipelines, orchestrate with Airflow, and work with Snowflake, Spark, and AWS to deliver high-performance data infrastructure.

The role combines hands-on engineering with technical consulting, translating business goals into data architecture roadmaps and collaborating across