Data Architect

C5i; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

C5i is seeking an experienced Data Architect/Engineer to design and govern a scalable data platform on Azure Databricks, implementing Delta Lake, Medallion patterns, and data products.

You will build ingestion pipelines, orchestrate with Airflow, and collaborate with distributed teams, including on-site support at the client office in Long Island City, NY 1–2 days per week.

Qualifications

• 8+ years of data engineering experience with hands-on Databricks
• Strong SQL and Python/PySpark skills
• Experience with Airflow orchestration and dbt Core
• Knowledge of Medallion Architecture and data quality

Responsibilities

• Design and govern a scalable data platform on Azure Databricks
• Define lakehouse architecture with Delta Lake and Unity Catalog
• Develop ingestion pipelines using APIs, databases, files, and Lakeflow/Auto Loader
• Establish CI/CD, Git workflows, automated testing, and data monitoring
• Collaborate with client and distributed teams, including on-site at Long Island City

Skills

Databricks

Azure Databricks

SQL

Python / PySpark

Airflow

dbt Core

CI/CD

Git

Data governance

Medallion Architecture

Tools

Lakeflow Connect

Auto Loader

Delta Lake

Job description

C5i is a pure-play AI & Analytics provider that combines the power of human perspective with AI technology to deliver trustworthy intelligence. The company drives value through a comprehensive solution set, integrating multifunctional teams that have technical and business domain expertise with a robust suite of products, solutions, and accelerators tailored for various horizontal and industry-specific use cases. At the core, C5i’s focus is to deliver business impact at speed and scale by driving adoption of AI-assisted decision-making.

C5i caters to some of the world’s largest enterprises, including many Fortune 500 companies. The company’s clients span Technology, Media, and Telecom (TMT), Pharma & Lifesciences, CPG, Retail, Banking, and other sectors. C5i has been recognized by leading industry analysts like Gartner and Forrester for its Analytics and AI capabilities and proprietary AI-based platforms.

Global offices

United States | Canada | United Kingdom | United Arab of Emirates | India

Essential Duties and Responsibilities

• These tasks reflect the core responsibility of the role in designing and governing a scalable, reusable data platform.
• Defines and maintains the lakehouse architecture on Azure Databricks, using Delta Lake and Unity Catalog.
• Designs and applies Medallion Architecture (Bronze, Silver, Gold) patterns, data products, and semantic models.
• Establishes reference designs, modeling conventions, and reusable patterns aligned with enterprise standards.

2. Data Ingestion, Orchestration & Transformation

• These tasks ensure reliable, well-structured data flows into and through the platform.
• Designs ingestion pipelines using APIs, databases, files, and Databricks-native capabilities such as Lakeflow Connect and Auto Loader.
• Defines orchestration using Apache Airflow and data transformation and modeling using dbt Core.
• Builds and guides development in SQL and Python/PySpark.
• These tasks ensure the platform is trusted, tested, and maintainable.
• Defines and implements data quality frameworks across pipelines and data products.
• Establishes CI/CD, Git-based workflows, automated testing, and data pipeline monitoring.

4. Partner Review & Technical Governance

• These tasks ensure partner and vendor work meets enterprise standards.
• Reviews partner architecture, code, data models, and pipelines for alignment with enterprise standards and reusable design.
• Provides technical guidance and feedback to distributed and partner engineering teams.
• These tasks reflect collaboration with client and internal teams.
• Works with client stakeholders, including on-site at the client office in Long Island City, NY 1–2 days a week.
• Applies AI-assisted/spec-driven development practices across the delivery lifecycle.

Requirements

• 8+ years of overall experience, including strong hands‑on Databricks experience
• Strong experience with Azure Databricks, Delta Lake, Unity Catalog, SQL and Python/PySpark
• Hands‑on experience with Apache Airflow for orchestration and dbt Core for data transformation and modeling
• Experience building ingestion pipelines using APIs, databases, files and Databricks-native capabilities such as Lakeflow Connect and Auto Loader
• Strong understanding of Medallion Architecture (Bronze, Silver, Gold), data products, semantic modeling and data quality frameworks
• Experience with CI/CD, Git, automated testing and data pipeline monitoring
• Ability to review partner architecture, code, data models and pipelines, ensuring alignment with enterprise standards and reusable design
• Experience with AI-assisted/spec-driven development and working with distributed engineering teams

C5i is proud to be an equal opportunity employer. We are committed to equal employment opportunity regardless of race, color, religion, sex, sexual orientation, age, marital status, disability, gender identity, etc. If you have a disability or special need that requires accommodation, please keep us informed about the same at the hiring stages for us to factor necessary accommodations.