Data Engineer

THE PRUITT GROUP LLC; Los Angeles (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

THE PRUITT GROUP LLC in Los Angeles seeks a Data Engineer to design and maintain Azure-based data pipelines and analytics solutions. You will collaborate with cross-functional teams to transform data into actionable insights, using Azure Data Factory, Synapse, and related Microsoft technologies.

Ideal candidates will have strong SQL, data modeling, and programming skills (Python/Java/C#), plus experience with data security, compliance, and documentation.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related discipline.
• Demonstrated experience as a Data Engineer with strong Microsoft tech focus (Azure Data Factory).
• Advanced SQL, data modeling, and experience with Azure data services (Azure SQL Database, Synapse).
• Hands-on experience designing and developing data pipelines using Azure Data Factory or similar ETL/ELT tools.

Responsibilities

• Design and develop data pipelines using Azure Data Factory and Microsoft tools with high data quality.
• Consolidate data from databases, APIs, and platforms into data lakes/warehouses using Azure services.
• Transform and cleanse data for analysis, reporting, and BI applications.
• Develop data models and schemas to enable efficient data access for users and apps.
• Monitor, optimize, and tune pipelines to improve performance and reduce latency.
• Enforce data security and access controls in line with privacy regulations.
• Troubleshoot and resolve data engineering challenges to maintain accuracy.
• Collaborate with data scientists and analysts; document pipelines and workflows.
• Oversee Azure cloud resources and apply cost-effective infrastructure practices.
• Keep up-to-date with Azure Data Factory and Microsoft tech for continuous improvement.

Skills

SQL

Data Modeling

Python

Java

C#

Data Security

Communication

Education

Bachelor's degree in Computer Science or IT

Tools

Azure Data Factory

Azure SQL Database

Azure Synapse Analytics

Cosmos DB

Microsoft Fabric

Job description

Job Description

We are looking for a talented and driven Data Engineer with strong expertise in Azure Data Factory and other Microsoft technologies to join our dynamic team. In this role, you will be responsible for designing, developing, and maintaining data pipelines, integration solutions, and data-centric applications. Your in-depth knowledge of Microsoft technologies and proficiency in Azure Data Factory will be key to optimizing data processing, storage, and analysis.

Responsibilities:

1. Design and Develop Data Pipelines: Partner with cross-functional teams to gather data requirements and create robust, scalable, and reliable data pipelines using Azure Data Factory and other Microsoft tools, ensuring high data quality.
2. Data Integration: Consolidate data from diverse sources—including databases, APIs, and third-party platforms—into centralized data lakes or warehouses utilizing Azure Data Factory, Azure SQL Database, Azure Synapse Analytics, and related technologies.
3. Data Transformation: Execute data transformation and cleansing processes to prepare data for analysis, reporting, and business intelligence applications.
4. Data Modeling: Develop and implement data models and schemas that align with business needs and industry standards, promoting efficient data access for end-users and applications.
5. Optimization and Performance Tuning: Regularly monitor and fine-tune data pipelines and processing tasks to enhance performance, reduce latency, and ensure accurate, timely data delivery.
6. Data Security and Compliance: Enforce data security protocols, including encryption and access controls, to protect sensitive information and maintain compliance with data privacy regulations.
7. Troubleshooting and Issue Resolution: Diagnose and resolve data engineering challenges, debugging pipelines and processes to uphold system performance and data accuracy.
8. Collaboration and Documentation: Collaborate with data scientists, analysts, and other stakeholders to address data needs, while maintaining thorough documentation of pipelines, models, and integration workflows.
9. Cloud Infrastructure Management: Oversee Azure cloud resources, monitor usage, and apply best practices for cost-effective infrastructure management.
10. Continuous Learning and Improvement: Keep abreast of the latest Azure Data Factory developments and Microsoft technologies, integrating best practices and innovative solutions to continually enhance data engineering processes.

Requirements:

1. A Bachelor’s degree in Computer Science, Information Technology, or a related discipline.
2. Demonstrated experience as a Data Engineer with a strong emphasis on Microsoft technologies, especially Azure Data Factory.
3. Advanced skills in SQL, data modeling, and working with database platforms such as Azure SQL Database, Azure Synapse Analytics, and Cosmos DB.
4. Hands-on experience designing and developing data pipelines using Azure Data Factory or comparable ETL/ELT tools.
5. Solid understanding of cloud-based data storage and processing solutions within the Microsoft Azure ecosystem.
6. Knowledge of data warehousing principles and data integration methodologies.
7. Proficiency in programming languages like Python, C#, or Java for data manipulation and automation purposes.
8. Familiarity with data security protocols and compliance standards related to data management.
9. Strong analytical and problem-solving abilities with a keen attention to detail.
10. Excellent communication and collaboration skills, capable of working effectively within cross-functional teams.
11. Relevant Microsoft certifications (e.g., Azure Data Engineer Associate) are preferred.
12. Experience with Microsoft Fabric is a plus but not required.