Senior Data Engineer

Wise Skulls; Austin (TX); In loco; USD 130.000 - 170.000; Tempo pieno

Descrizione del lavoro

Wise Skulls LLC in Austin, TX is seeking a Senior Data Engineer to design, build, and maintain reliable data pipelines powering analytics and AI-driven insights. You will work with Spark and Snowflake to move and transform data at scale, and apply AI tools to visualize data for business decisions.

The role emphasizes data quality, collaboration with analysts and product teams, and leveraging MCP and LLM-based tools to automate workflows.

Competenze

• 5–8 years of experience in data engineering and ETL/pipeline development.
• Proficient with Spark and Snowflake.
• Hands-on Python for data processing and scripting.

Mansioni

• Design, build, and maintain scalable ETL/ELT pipelines for batch and near-real-time data.
• Use Spark to transform and process large datasets efficiently.
• Model, load, and optimize data in Snowflake for performance and cost.
• Implement validation, monitoring, and error handling to ensure data quality.
• Create AI-powered dashboards and visualizations for business insights.
• Collaborate with analysts, product managers, and stakeholders to translate requirements into data solutions.
• Document pipelines, data lineage, and transformations.

Conoscenze

ETL/ELT pipelines

Spark

Snowflake

Python

AI viz tools

LLM tools

MCP

Strumenti

Airflow

AWS

Azure

GCP

Git

CI/CD

Data governance

Data cataloging

Descrizione del lavoro

We're looking for a Senior Data Engineer to design, build, and maintain reliable data pipelines that power analytics and AI-driven insights. You'll work with Spark and Snowflake to move and transform data at scale, and use AI tools to build visualizations and dashboards that make that data useful to the business.

Key Responsibilities

• - Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines for batch and, where needed, near-real-time data.
• - Data Processing: Use Spark to transform and process large datasets efficiently.
• - Data Warehousing: Model, load, and optimize data in Snowflake for performance and cost.
• - Data Quality & Reliability: Implement validation, monitoring, and error handling to keep pipelines accurate and dependable.
• - AI-Powered Visualization: Use AI tools to build dashboards and reports that communicate insights clearly.
• - AI Enablement: Apply AI agents, MCP (Model Context Protocol), and LLM-based tools to automate workflows and speed up data tasks.
• - Documentation: Document pipelines, data lineage, and transformations.
• - Collaboration: Work with analysts, product managers, and business stakeholders to turn requirements into data solutions.

Required Qualifications

• - 5-8 years of experience in data engineering, including ETL and data pipeline development
• - Working knowledge of Apache Spark
• - Hands-on experience with Snowflake
• - Basic Python for data processing and scripting
• - Experience using AI tools to create visualizations and dashboards
• - Awareness of AI concepts, including agents, MCP, and LLM-based tools

Preferred

• - Orchestration tools (e.g., Airflow)
• - Cloud platform experience (AWS, Azure, or GCP)
• - Git and CI/CD practices
• - Familiarity with data governance and cataloging