ETL Data Platform Developer

Cloud Analytics Technologies, LLC; Redmond (WA); On-site; USD 150,000 - 190,000; Full time

Job summary

Cloud Analytics Technologies, LLC is seeking a senior data engineer to design and implement ETL pipelines using Microsoft Fabric and Azure Data Factory in Redmond, WA. You will build a metadata-driven Lakehouse architecture and an orchestration layer to enable cross-agent querying, while ensuring data quality, lineage, and governance across all data flows.

The role requires strong expertise in Fabric components, Azure SQL, Data Lake Gen2, Power BI, and Python/Spark/Scala.

Qualifications

• 7+ years of experience in data engineering with a focus on Microsoft Azure Data stack and Fabric technologies.
• Proficient in Microsoft Fabric components (Lakehouse, Dataflows Gen2, Pipelines, Notebooks) and Azure Data Factory.
• Strong SQL, Python and Spark/Scala skills with experience in structured and semi-structured data.
• Experience building metadata-driven architectures with governance and security in mind.
• Familiarity with agent-based architectures and conversational AI integration is a plus.

Responsibilities

• Design and implement ETL pipelines using Microsoft Fabric and Azure Data Factory.
• Build and maintain a metadata-driven Lakehouse architecture to support multiple consumption patterns.
• Develop agent-specific data lakes and an orchestration layer for cross-agent querying.
• Enable interactive data consumption via Power BI, Azure OpenAI, and other analytics tools.
• Ensure data quality, lineage, and governance across all ingestion and transformation processes.
• Collaborate with product teams to understand data needs and deliver scalable solutions.
• Optimize performance and cost across storage and compute layers.

Skills

Microsoft Fabric

Azure Data Factory

Azure SQL

Azure Data Lake Storage Gen2

Power BI

SQL

Python

PySpark

Scala

JSON/XML/CSV/Parquet

Data governance

Data modeling

Security best practices

Agent-based architectures

Job description

Key Responsibilities

• Design and implement ETL pipelines using Microsoft Fabric (Dataflows, Pipelines, Lakehouse ,warehouse, sql) and Azure Data Factory.
• Build and maintain a metadata driven Lakehouse architecture with threaded datasets to support multiple consumption patterns.
• Develop agent specific data lakes and an orchestration layer for an uber agent that can query across agents to answer customer questions.
• Enable interactive data consumption via Power BI, Azure OpenAI, and other analytics and AI tools.
• Ensure data quality, lineage, and governance across all ingestion and transformation processes.
• Collaborate with product teams to understand data needs and deliver scalable solutions.
• Optimize performance and cost across storage and compute layers.

Required Qualifications

7+ years of experience in data engineering with a focus on Microsoft Azure Data stack and Fabric technologies.

Strong expertise in:

• Executive written and oral communication
• Microsoft Fabric (Lakehouse, Dataflows Gen2, Pipelines, Notebooks)
• Azure Data Factory, Azure SQL, Azure Data Lake Storage Gen2
• Power BI and/or other visualization tools
• SQL, Python and PySpark/ Scala
• Experience working with structured and semi structured data (JSON, XML, CSV, Parquet).
• Proven ability to build metadata driven architectures and reusable components.
• Familiarity with agent based architectures and conversational AI integration is a plus.
• Strong understanding of data modeling, data governance, and security best practices.

CLOUD ANALYTICS TECHNOLOGIES is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Equal Opportunity Employer

CLOUD ANALYTICS TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Some of our Business Ventures - Knowledge Analytics

Do restrictive necessities, your capability for advanced analytics or your business model need custom‑made applications and infrastructure configurations? opt for our hosted managed services to urge comprehensive analytics solutions supported industry‑leading Cloud Analytics Technologies, LLC code - delivered by the Cloud Analytics Technologies, LLC consultants UN agency understand these solutions best. Through a mix of with‑it technology and dedicated service and support, Cloud Analytics Technologies, LLC hosted managed services will facilitate you: