Data Engineer

Tech3pillars Technologies; Alaska; On-site; USD 120,000 - 180,000; Full time

Job summary

Tech3pillars Technologies in Atlanta, GA (onsite) seeks a Data Engineer with deep expertise in Azure Synapse Analytics and Apache Spark. You will design scalable data pipelines, build enterprise data solutions, and implement Data Lake architectures following the Medallion model.

The role requires 8+ years in data engineering, strong PySpark/Spark SQL, and solid SQL tuning. Collaborate with architects, analysts, and business stakeholders to deliver high-quality data solutions and support

Qualifications

• 8+ years of experience in Data Engineering and Data Platform development.
• Strong hands-on experience with Azure Synapse Analytics.
• Solid understanding of Microsoft Fabric and its data engineering capabilities.
• Deep knowledge of Apache Spark architecture and distributed computing concepts.
• Extensive experience with PySpark and Spark SQL.
• Strong understanding of Data Lake concepts and modern data architecture patterns.
• Experience implementing Medallion Architecture and large-scale data processing solutions.
• Strong SQL development and performance tuning skills.
• Excellent analytical, troubleshooting, and problem-solving abilities.
• Strong verbal and written communication skills.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure-based technologies.
• Build and maintain enterprise data solutions using Azure Synapse Analytics.
• Develop data transformation and processing workflows using PySpark and Spark SQL.
• Design and implement Data Lake solutions following Medallion Architecture (Bronze, Silver, Gold layers).
• Work with structured and unstructured data from multiple sources to support business analytics needs.
• Optimize Spark workloads and troubleshoot performance issues in distributed data processing environments.
• Collaborate with architects, analysts, developers, and business stakeholders to deliver high-quality data solutions.
• Support data quality, governance, and reliability initiatives across the data platform.
• Contribute to the adoption and implementation of Microsoft Fabric capabilities where applicable.

Skills

Azure Synapse Analytics

Apache Spark

PySpark

Spark SQL

Data Lake

SQL

Distributed computing

Analytics

Troubleshooting

Communication

Tools

Azure Data Factory

Azure Data Lake Storage

Microsoft Fabric

Job description

Job title: Data Engineer Azure Synapse & Microsoft Fabric

Location: Atlanta, GA(Onsite)

Job Summary
We are seeking a highly skilled Data Engineer with strong expertise in Azure Synapse Analytics, Apache Spark, and modern cloud-based data platforms. The ideal candidate will have hands-on experience designing and implementing scalable data solutions, building robust data pipelines, and leveraging Data Lake architectures to support enterprise analytics and reporting initiatives.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure-based technologies.
• Build and maintain enterprise data solutions using Azure Synapse Analytics.
• Develop data transformation and processing workflows using PySpark and Spark SQL.
• Design and implement Data Lake solutions following Medallion Architecture (Bronze, Silver, Gold layers).
• Work with structured and unstructured data from multiple sources to support business analytics needs.
• Optimize Spark workloads and troubleshoot performance issues in distributed data processing environments.
• Collaborate with architects, analysts, developers, and business stakeholders to deliver high-quality data solutions.
• Support data quality, governance, and reliability initiatives across the data platform.
• Contribute to the adoption and implementation of Microsoft Fabric capabilities where applicable.

Required Qualifications

• 8+ years of experience in Data Engineering and Data Platform development.
• Strong hands-on experience with Azure Synapse Analytics.
• Solid understanding of Microsoft Fabric and its data engineering capabilities.
• Deep knowledge of Apache Spark architecture and distributed computing concepts.
• Extensive experience with PySpark and Spark SQL.
• Strong understanding of Data Lake concepts and modern data architecture patterns.
• Experience implementing Medallion Architecture and large-scale data processing solutions.
• Strong SQL development and performance tuning skills.
• Excellent analytical, troubleshooting, and problem-solving abilities.
• Strong verbal and written communication skills.

Preferred Qualifications

• Experience with Azure Data Lake Storage (ADLS), Azure Data Factory, and other Azure data services.
• Exposure to data warehousing, ETL/ELT frameworks, and cloud-native analytics solutions.
• Experience working in Agile/Scrum environments.
• Knowledge of data governance, security, and performance optimization best practices.