Senior Cloud Data Engineer / Data Architect

TechTrend, Inc.; Reston (VA); Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Life Insurance

Short Term Disability Insurance

Long Term Disability Insurance

401(k) Plan (matching component)

Gym access at TechTrend HQ

Job summary

TechTrend, Inc. seeks a Senior Cloud Data Engineer / Data Architect to design, build, modernize, and support enterprise-scale cloud data platforms in the Microsoft Azure ecosystem.

The role balances architecture with hands-on data engineering using Azure Databricks, ADLS Gen2, Synapse, Data Factory, SQL, Python/PySpark, and related Azure services. The candidate will work across architecture, data engineering, analytics, governance, security, and AI/ML integration to translate requirements into

Qualifications

• 7+ years in data engineering/architecture or related roles.
• 4+ years designing or implementing cloud-based data platforms.
• Proven expertise with Azure data services and Databricks.
• Strong SQL and Python/PySpark engineering skills.
• Experience in data modeling, governance, and security patterns.

Responsibilities

• Architect and implement modern cloud data platforms and Lakehouse architectures on Azure and Databricks.
• Design scalable data lakes, warehouses, and analytics architectures for varied data types.
• Build and optimize batch and streaming pipelines using Azure Databricks, ADLS Gen2, Synapse.
• Define ingestion patterns from databases, APIs, files, and on-prem sources.
• Institute data governance, lineage, RBAC, and security controls.

Skills

Azure Databricks

Azure Data Factory

Azure Synapse

ADLS Gen2

Python/PySpark

SQL

Spark/PySpark

Delta Lake

Unity Catalog

RBAC

Tools

Terraform

GitHub

Azure DevOps

PowerShell

Microsoft Purview

CI/CD

Job description

TechTrend is seeking a Senior Cloud Data Engineer / Data Architect to design, build, modernize, and support enterprise-scale cloud data platforms, primarily within the Microsoft Azure ecosystem.

This is a senior, hands-on technical role. The ideal candidate will combine strong data architecture expertise with the engineering depth necessary to build and troubleshoot production data solutions using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure Data Factory, SQL, Python/PySpark, and related Azure Data & AI services.

The candidate should be comfortable working across architecture, data engineering, analytics, governance, security, and AI/ML integration and translating business and mission requirements into scalable production solutions.

What We're Looking For

We are not looking for someone who is purely an architect or purely a traditional ETL developer. The successful candidate will be a hands-on Data Engineer / Architect who can move comfortably between: Architecture → Data Modeling → Engineering → Databricks → Azure → Governance → AI Enablement → Production Operations. Selected candidate should be able to whiteboard the target architecture with stakeholders and then get into Databricks or Azure and help the team build, troubleshoot, optimize, and operationalize it.

Responsibilities

• Architect and implement modern cloud data platforms and Lakehouse architectures using Azure and Databricks.
• Design scalable data lakes, Lakehouse, data warehouse, and analytical architectures supporting structured, semi-structured, and unstructured data.
• Build and optimize batch and streaming data pipelines using technologies such as Azure Databricks, Spark/PySpark, Azure Data Factory, Synapse pipelines, Event Hubs, and related services.
• Design and implement solutions using ADLS Gen2, Delta Lake, Databricks SQL, Unity Catalog, Azure Synapse Analytics, and Azure SQL.
• Design ingestion and integration patterns for data originating from databases, APIs, files, applications, cloud platforms, and legacy/on-premises systems.
• Define data modeling strategies including dimensional models, curated data products, transformation layers, and analytical datasets.
• Establish data governance, lineage, cataloging, access-control, data-quality, and security patterns, including Unity Catalog and Microsoft Purview where appropriate.
• Design secure data architectures using Microsoft Entra ID, managed identities, RBAC, Key Vault, private endpoints, and least-privilege principles.
• Work with AI/ML teams to prepare governed enterprise data for machine learning, Generative AI, RAG, analytics, and other AI-enabled use cases.
• Support integration between Databricks and Azure AI/ML capabilities, APIs, applications, reporting platforms, and enterprise systems.
• Evaluate performance, scalability, availability, security, and cost characteristics of proposed architectures.
• Develop reusable engineering patterns, reference architectures, notebooks, pipelines, IaC, technical standards, and deployment documentation.
• Troubleshoot complex data pipeline, integration, Spark, SQL, performance, security, and connectivity issues.
• Collaborate directly with customers, architects, application developers, cybersecurity teams, cloud engineers, and business stakeholders.
• Provide technical leadership and mentoring to data engineers while remaining actively involved in design and implementation.

Qualifications

• 7+ years of professional experience in data engineering, data architecture, analytics engineering, cloud engineering, or related disciplines.
• 4+ years of hands-on experience designing or implementing cloud-based data platforms.
• Strong hands-on expertise with Microsoft Azure data services.
• Demonstrated experience with Azure Databricks and Apache Spark/PySpark.
• Strong understanding of Lakehouse and Data Lake architecture, including Delta Lake concepts.
• Strong SQL and Python/PySpark engineering skills.
• Experience designing enterprise data ingestion, ETL/ELT, transformation, orchestration, and integration pipelines.
• Experience with data modeling, schema design, performance optimization, partitioning, and large-scale dataset processing.
• Understanding of Databricks Unity Catalog, data governance, lineage, RBAC, and access-control concepts.
• Experience implementing secure cloud architectures for sensitive or regulated data.
• Experience integrating cloud data platforms with downstream analytics, applications, APIs, and AI/ML workloads.
• Strong analytical and troubleshooting skills.
• Ability to translate business requirements into scalable technical architectures.
• Excellent written and verbal communication skills, including the ability to explain architectural decisions and technical trade-offs.

Preferred Qualifications

• Experience with Azure Government and/or federal government environments.
• Previous experience supporting USDA or other federal civilian agencies.
• Experience with Microsoft Fabric and Power BI.
• Experience with Azure AI services, Azure Machine Learning, MLflow, Mosaic AI, Generative AI, RAG, vector search, or AI/ML pipelines.
• Experience with real-time/streaming architectures using Kafka, Event Hubs, or Spark Structured Streaming.
• Experience with Terraform, Bicep, GitHub, Azure DevOps, and CI/CD for data-platform deployment and automation.
• Experience with Microsoft Purview or other enterprise data governance/catalog platforms.
• Experience integrating Databricks with enterprise systems and APIs.
• Familiarity with federal security and governance frameworks including NIST 800-53, FedRAMP, and FISMA.
• Relevant certifications such as:
  • Databricks Certified Data Engineer Professional
  • Databricks Certified Data Engineer Associate
  • Microsoft Azure data/AI certifications
  • Azure architecture certifications
• Life Insurance
• Short, Long Term Disability Insurance
• 401(k) Plan (matching component included)
• Access to state-of-the-art gymnasium (at TechTrend HQ)

Travel Requirements

N/A

Work Location

Fully Remote (prefer a candidate that can be at our Reston, VA office once a week).

About TechTrend

TechTrend, Inc. is a veteran-friendly small business providing expert solutions, products, and services to the federal government. Founded in 2003, we continue to evolve with capabilities in cybersecurity, DevSecOps, cloud managed services, cloud migration, and application development. We are a Microsoft Gold Partner and leading provider of Azure cloud services. TechTrend is recognized as a trusted partner delivering knowledge and guidance for our client’s most critical and complex support and service needs.

As a liaison for positive organizational change, we form relationships and build bridges while ensuring quality across functions—gaining buy-in from both leaders and end-users and removing barriers to mission success. Our established processes ensure quality delivery of results by maximizing efficiency, productivity, and client satisfaction enterprise wide. TechTrend is a fast-growing company with a dynamic, inclusive corporate culture headquartered in a state-of-the-art facility located in the heart of Reston Town Center.