Data Engineer

Headfarmer; Phoenix (AZ); On-site; USD 120,000 - 180,000; Full time

Job summary

Headfarmer is seeking a Senior Data Engineer to mature a medallion data architecture (Bronze/Silver/Gold) and enable AI/ML analytics. You will own ingestion pipelines, data quality, and provenance views while mentoring junior engineers and partnering with the Data Architecture Director on architectural decisions.

The role emphasizes API-driven ingestion, data reconciliation, and production-grade Python across AWS/Azure/GCP environments. On-site in Phoenix preferred.

Qualifications

• 3+ years of production data engineering experience.
• Experience with a modern cloud data warehouse (BigQuery, Snowflake, Redshift, or similar).
• Advanced SQL focusing on window functions, CTEs, performance tuning, and messy CRM data.
• Python at production level; advanced Python is a plus.
• API integration and landing API output for downstream processing.
• AI-Augmented workflows experience in production engineering environments.

Responsibilities

• Build and maintain ingestion pipelines from source systems into Bronze/Silver/Gold medallion architecture.
• Own data quality checks gating data from Bronze to Silver.
• Accelerate table creation for API-based ingestion sources.
• Hardening of the Silver layer and provenance views for accurate metrics.
• Reconcile dashboards and metrics across multiple data sources.
• Mentor junior engineers and collaborate with the Data Architecture Director on complex decisions.
• Establish engineering practices and automate code review and CI/CD for pipelines.
• Build and operate cloud-based ingestion/orchestration jobs with robust failure handling.

Skills

Advanced SQL

Python

API integration

Data reconciliation

Performance tuning

Window functions

Common Table Expressions

Multi-cloud

Mentorship

Data modeling

Tools

dbt

Dataform

BigQuery

Snowflake

Redshift

Power BI

Job description

Our client is a fast-growing, high-volume field-service business that is in the middle of transitioning its data warehousing function from an offshore vendor to an in-house, onshore team. They run their operations on a CRM, using a medallion data architecture (Bronze/Silver/Gold) to feed executive dashboards, FP&A reporting, and operational decision-making across dozens of markets.

The data sources span CRM, accounting, FP&A, marketing, and telemetry systems. The platform works, but it's young — we need a senior engineer who can help mature it, close data quality gaps, accelerate table creation for our API ingestion, and lay a trustworthy foundation for upcoming AI/ML predictive analytics and agentic AI initiatives.

We are an AI-forward team. We use AI tools daily for code review, drafting, and sanity-checking work — but we need someone who knows how to use AI to increase efficiency without losing the judgment to verify what it produces. You don't need to have used any specific tool (Claude, ChatGPT, Copilot, etc.) — what matters is that you can catch AI output that's wrong before it ships.

This role also carries a real mentorship component: you'll work alongside junior team members who know the business and CRM deeply, delegate simpler validation work to them, and help upskill them as they shadow you and the Data Architecture Director on more complex work.

What You'll Own

• Ingestion Pipelines: Perform API calls against source systems, land data in a staging zone, and build the dataflows that move it into our Bronze/Silver/Gold medallion architecture.
• Data Quality: Own the quality checks that gate data moving from Bronze to Silver, ensuring what reaches reporting layers is trustworthy.
• Accelerate Table Creation: Build out the tables needed to support our growing set of API-based ingestion sources.
• Silver Layer Hardening: Help mature our bitemporal silver layer and build provenance views that surface discrepancies affecting reported metrics.
• Reconciliation: Reconcile dashboards and metrics across multiple sources of truth using consistent methodology.
• Mentorship & Collaboration: Work hands-on with junior team members — delegating appropriate tasks, reviewing their work, and helping build their skills over time. Collaborate closely with the Data Architecture Director on more complex architectural decisions.
• Process Maturity: Help establish stronger engineering practices as we move from an offshore vendor model to an in-house team — including code review processes for the Bronze/Silver/Gold pipeline, since this currently isn't automated and is a known gap.
• Infrastructure & Orchestration: Build and operate cloud-based ingestion and orchestration jobs with sensible failure handling.

Required Experience

• 3+ years of production data engineering experience, ideally with a modern cloud data warehouse (BigQuery, Snowflake, Redshift, or similar). Note: this team is genuinely flexible on cloud platform — GCP, Azure, and AWS experience are all valued, and our Data Architecture Director has deep multi-cloud experience himself.
• Advanced SQL is the top technical priority — window functions, CTEs, performance tuning, and comfort working with messy, real-world CRM data (odd timestamp formats, SCD-versioned records, IDs that don't behave as expected, etc.).
• Python at a working/production level. Advanced Python is a plus but not a hard requirement — moderate proficiency with a track record of shipping production code is acceptable.
• API integration experience — pagination, authentication, rate limits, and reliably landing API output for downstream processing.
• Data reconciliation experience — comfortable holding multiple datasets in your head, identifying discrepancies, and clearly explaining root causes.

Required: AI-Augmented Workflow Experience

This is a genuine requirement, not a buzzword. We're looking for someone who:

• Has real, hands-on experience using AI tools (Claude, ChatGPT, Copilot, or similar — tool-agnostic, no specific platform required) to increase efficiency in production engineering work.
• Understands AI as an assistant for completing tasks and sanity-checking, not a replacement for engineering judgment.
• Can identify when AI output is wrong — hallucinated table/column names, subtly incorrect SQL syntax — before it reaches production.
• Is comfortable working in an environment that's actively building toward more automated code review and CI/CD practices as we mature past our current manual process.

Nice to Have

• CRM domain knowledge (field service vertical — jobs, appointments, timesheets data models).
• Power BI or similar BI tool experience.
• Bitemporal data warehouse design patterns (Kimball, Data Vault/Linstedt, or a pragmatic hybrid).
• Experience with dbt, Dataform, or similar transformation tooling.
• Prior experience mentoring junior engineers or analysts.