Data Engineer

SmallArc Inc; Washington (District of Columbia); On-site; USD 120,000 - 160,000; Full time

Job summary

SmallArc Inc. is seeking a hands-on Data Engineer with strong Azure experience to design, develop, and maintain scalable data pipelines and datasets that support analytics and operations.

You will work with ADF, Azure SQL, and blob storage, implement data quality and governance, and collaborate with analytics and product teams to deliver secure, well-documented data products while following CI/CD and security best practices.

Qualifications

• Bachelor's degree with 5 years of experience or 4 years with equivalent.
• U.S. citizenship and ability to obtain Public Trust clearance.
• Strong hands-on experience with Azure Data Factory, Blob Storage, and SQL.

Responsibilities

• Design, develop, schedule, and monitor batch and near-real-time data pipelines using ADF.
• Manage Azure Blob Storage, including containers, access tiers, lifecycle policies, and data partitioning.
• Design and optimize Azure SQL databases with schema design and performance tuning.
• Develop dimensional data models using star/snowflake schemas, 3NF, and Data Vault 2.0 concepts.
• Implement data quality, governance, lineage, and schema evolution standards.
• Collaborate with analytics, product, and operations teams to deliver secure data products.
• Apply security best practices, including RBAC, encryption, data masking, and key management.
• Develop reusable components and CI/CD practices.

Skills

Azure Data Factory

Azure Blob Storage

Azure SQL

SQL

Python

ETL/ELT design

Data modeling

Git-based workflows

Azure DevOps

CI/CD practices

Education

Bachelor's degree

Tools

Azure Data Factory

Azure Blob Storage

Azure SQL

Python

Git

Azure DevOps

CI/CD

Data Vault 2.0

Star/Snowflake Schemas

3NF

Job description

Job Overview

We are seeking a hands-on Data Engineer with strong Microsoft Azure experience to design, develop, and maintain reliable, scalable data pipelines and datasets that support analytics and operational insights. The ideal candidate will have extensive experience with Azure Data Factory (ADF), Azure Blob Storage, Azure SQL, data modeling, and ETL/ELT processes.

Key Responsibilities

• Design, develop, schedule, and monitor batch and near-real-time data pipelines using Azure Data Factory (ADF).
• Manage Azure Blob Storage, including containers, access tiers, lifecycle policies, and data partitioning.
• Design and optimize Azure SQL databases, including schema design, indexing, query optimization, and performance tuning.
• Develop dimensional data models using star and snowflake schemas, 3NF, and Data Vault 2.0 concepts.
• Implement data quality, validation, governance, lineage, and schema evolution standards.
• Optimize pipeline performance, implement monitoring and alerting, and troubleshoot production issues.
• Collaborate with analytics, product, and operations teams to deliver secure, reliable, and well-documented data products.
• Apply security best practices, including RBAC, encryption, data masking, and key management.
• Develop reusable components and follow best practices for data engineering and CI/CD.

Required Qualifications

• Bachelor's degree with 5 years of experience, or 4 years of experience in lieu of a degree.
• Ability to obtain and maintain a Public Trust clearance, requiring U.S. citizenship.
• Strong hands-on experience with Azure Data Factory, Azure Blob Storage, and Azure SQL.
• Knowledge of star/snowflake schemas, 3NF, Data Vault, and Kimball vs. Inmon data warehousing methodologies.
• Proficiency in SQL and data formats such as JSON, Parquet, CSV, and Avro.
• Experience with Python (preferred) or Scala for data transformations and automation.
• Experience with Git-based workflows and Azure DevOps or equivalent CI/CD tools.
• Experience monitoring, troubleshooting, and maintaining production data pipelines.

Preferred Qualifications

• Experience with Azure Synapse Analytics and large-scale ETL/ELT processing.
• Familiarity with Microsoft Fabric, Lakehouse, OneLake, and data pipelines.
• Power BI semantic modeling, DAX, and incremental refresh experience.
• Knowledge of Event Hubs, Azure Functions, Databricks, or Spark Structured Streaming.
• Experience with Terraform, Bicep, or ARM templates.
• Familiarity with Microsoft Purview or similar data cataloging and lineage tools.

I look forward to your response. Please feel free to reach out if you have any questions.