Expert Software Architect

Tcognition, Inc.; City of Albany (NY); On-site; USD 120,000 - 172,000; Full time

Job summary

Tcognition, Inc. is seeking a Data Warehouse Architect Lead for Michigan's data initiatives. The role focuses on building and maintaining data pipelines, data lake hydration, and data product development with strong governance and security practices.

You will lead a team to design scalable data architectures, collaborate with cross-functional partners, and ensure delivery of trusted data assets for analytics and ML applications. Local Michigan on-site presence is required for critical hours.

Qualifications

• 4+ years hand-on experience in hydrating the enterprise Data Lake on a Cloud Service Provider (AWS, Azure, or Google) from relational, semi-structured and unstructured data sources by building Extract and Load Data Pipelines.
• 4+ years hands-on experience in Transforming datasets in a Data Lake on a Cloud Service Provider to deliver Data Products with traceability of attributes to the Critical Data Elements in the systems of record.
• 2+ years experience enforcing Data Sharing Policies to Data Products to ensure users access appropriate versions at subscription time.

Responsibilities

• Oversee the design, development, and maintenance of data pipelines for analytics and operational systems.
• Architect and optimize data infrastructure, including storage, orchestration frameworks, and ETL/ELT processes.
• Ensure data quality, reliability, and security with standards, monitoring, and governance practices.
• Lead and mentor data engineering staff, review designs/code, and support professional growth.
• Collaborate with data scientists, analysts, product teams, and stakeholders to meet organizational needs.

Skills

Data Warehouse

Cloud Services

AWS

Azure

Google Cloud

Data Lake

Data Pipelines

ETL

Data Transformation

Data Products

SQL

Python

R

Spark

Terraform Scripting

DAGs Scripting

YAML scripting

Docker

Education

Bachelor's Degree

Tools

Juniper Notebooks

NOSQL DB

Graph DB

Parquet DB

Lakehouse

Medallion Architecture

Kafka

Kubernetes

Glue

Databricks

Power BI

Tableau

Redshift Spectrum

Sagemaker

DBT

Datahub

SageMaker

GitHub

DevOps

Job description

Build your career with the freedom you deserve. Explore our stable, rewarding remote work opportunities and unlock a world of flexibility and professional growth.

Code your career with tCognition

Experience: 2-5

Number of Positions: 1

Skills Required: Data Warehouse, Cloud Services, AWS, Azure, Google Cloud, Data Lake, Data Pipelines, ETL, Data Transformation, Data Products

The State of Michigan is looking for a Data Warehouse Architect 3

Max Bill Rate: $106.00/hour

Vendor submissions will close on 11/19 at 10 AM ET. Late submissions will not be accepted.

Top Skills & Years of Experience

• Clear and concise written and verbal communication is crucial to success and productivity in this role

Required Education: Bachelors Degree

Required Skills

• Minimum 4 years of hand-on experience is required in hydrating the enterprise Data Lake on a Cloud Service Provider (AWS, Azure, or Google) from relational, semi-structured and unstructured data sources by building Extract and Load Data Pipelines.
• Minimum of 4 years hands-on experience is required in Transforming the datasets in a Data Lake on a Cloud Service Provider to deliver Data Products with traceability of their attributes / features to the Critical Data Elements in the systems of record.
• Minimum of 2 years experience is required in enforcing Data Sharing Policies to Data Products to ensure that User get to see only the version of the Data Product the User is entitled to get access to at the time of subscription.

Preferred Skills

• O Proficiency desirable in DevOps tools including Github/Jenkins/Terraform/YAML/DAGs/Docker/Databricks/SageMaker etc. for building Data/ML pipelines to extract/store data from multiple sources to parquet destinations.
• O Experience desirable in promoting Data Engineering Standards and Best Practices among the community of developers to influence the enterprise towards building scalable and supportable Data/ML Products delivery and their usage by the community of data, analytics, and ML product users.
• O Experience desirable in implementing Data Governance and Data Security tools available on a CSP to automate the enforcement of Data Sharing Policies at the point of subscription of a Data Product by a subscriber.

Other Desired Skills

• o Must have a BS in Computer Science / Data Science / Information Systems with an overall 4+ years of experience in designing and implementing data, analytics, and AI solutions for large enterprises by building data pipelines using the available data platform tools of any Cloud Service Provider (CSP) such as AWS/Microsoft/Google.
• o Experience of minimum 4 years in modeling and leveraging Relational data services with demonstrated proficiency in Python/Pysparks/SQL programming and tuning SQL queries to extract data from multiple sources ( relational DB, flat files, CSV, HTML, JSON, etc) sources into Lakehouse Medallion architecture in parquet formats
• o Demonstrated experience and expertise of minimum 4 years in programming/scripting languages including Python, R, Spark, etc.
• o Experience in designing and delivering Star-schema based operational data store is required to enable visualization of program activities and outcomes in a visualization tool such as Power BI.

Key Programming Languages: SQL, Python, R, Spark, Terraform Scripting, DAGs Scripting, YAML scripting, Docker

Other Technologies, Concepts and Frameworks: Juniper Notebooks, NOSQL DB, Graph DB, Parquet DB, Lakehouse and Medallion Architecture concepts, Kafka, Kubernetes, Glue, Databricks, Power BI, Tableau, Redshift Spectrum, Sagemaker, DBT, Datahub, SageMaker, Machine Learning, GitHub, DevOps

Data Engineering Lead excels in data modeling, Lakehouse, and ML products delivery using an agile DevOps framework with a mastery of distributed data architectures essential for data, analytical and operational ML applications with demonstrated proficiency in frameworks including DataBricks, Sagemaker, Spark, Kafka, Kubernetes, SQL, NoSQL DBs, Graph DBs, and visualization tools such as Power BI/Tableau.

The Data Engineer Lead serves as the primary State of Michigan technology lead engaging with the UoM proof of concept team to support best practice adoption within the state hosting environment and provides training to DAV data engineers. This role’s responsibilities include:

• Oversee the design, development, and maintenance of data pipelines that move, transform, and prepare data for analytics and operational systems.
• Architect and optimize data infrastructure, including storage solutions, orchestration frameworks, and ETL/ELT processes.
• Ensure data quality, reliability, and security by establishing standards, monitoring systems, and implementing validation and governance practices.
• Lead and mentor data engineering staff, providing technical guidance, reviewing designs/code, and supporting professional growth.
• Collaborate with cross functional partners, such as data scientists, analysts, product teams, and business stakeholders, to ensure data solutions meet organizational needs and strategic goals.

Local candidates vs non-local: Open to candidates local to the Lansing Michigan area and those willing to relocate. Candidates that relocate must relocate before day one and be ready to be on-site from day one at their own expense (non-negotiable)

Hybrid Work Schedule: Wednesday and Thursday are REQUIRED in-office days. This is NON-NEGOTIABLE and there is NO remote only option.

Assignment duration: Projected to go until 11/2027. Ultimate duration and possibility of extension will be dependent on business need.

Interview Process: Virtual and In-Person. 1st round is virtual, 2nd round is in-person. For the virtual round, Candidates should join from a laptop and be prepared to share their screen if requested. A screenshot photo of candidate will be required for any interviews as well as a vendor present at beginning of virtual interview to validate candidate

ID - DHW - Business Systems Analyst/Programmer