Azure Data Engineer - Remote & Growth Opportunities

Remote; San Francisco (CA); Remote; USD 135,000 - 185,000; Full time

Benefits offered by this job

Bonuses

PTO (Paid Time Off)

Flexible hours

Remote work

Education & certs

Job summary

Bluelight is a leading software consultancy seeking a Data Engineer to design and maintain ETL data pipelines using PySpark in Azure Synapse notebooks and pipelines. You will build scalable data warehouses, extract data from REST APIs, SQL databases, and CSV files, and ensure data quality and governance.

The role emphasizes collaboration with data architects and cross-functional teams, with opportunities for growth, remote work options, and ongoing education in a fast-paced environment.

Qualifications

• Bachelor's degree or equivalent; Azure Data Engineer cert is a plus.
• Proven ETL experience with Python (PySpark) extracting from REST APIs, SQL tables, and CSVs.
• Proficiency with Azure Synapse resources including Notebooks, Pipelines, Linked Services, and Key Vault.
• Ability to write complex SQL with SparkSQL and MS SQL; optimize queries.
• Knowledge of data integration best practices and tools.
• Experience with Git or Azure DevOps.
• Strong problem solving, analytical skills, and attention to detail.
• Excellent communication, teamwork, and adaptability.
• Familiarity with big data, ML, data analysis; BI tools a plus.

Responsibilities

• Develop and maintain ETL data pipelines using Python (PySpark) in Azure Synapse notebooks/pipelines.
• Design and build data warehouses with star schemas and appropriate dimensional models.
• Extract data from REST APIs, SQL tables, and CSV files for downstream analytics.
• Optimize performance of notebooks/pipelines to meet SLAs and handle scale.
• Monitor ETL jobs, troubleshoot data issues, and document data flows.
• Collaborate with data architects and cross functional teams on data requirements.
• Ensure data security and governance compliance.
• Provide data quality checks and validation across pipelines.
• Maintain comprehensive documentation of data engineering processes.
• Support data visualization and analytics initiatives with reliable data sources.

Skills

Python (PySpark)

ETL data engineering

SQL querying (SparkSQL/MS SQL)

Azure Synapse Analytics

Data modeling & warehousing

Data quality & governance

Version control (Git/Azure DevOps)

Communication skills

Big data & analytics familiarity

Data visualization (Power BI/Tableau)

Education

Bachelor's degree in CS/IT or related field; Azure Data Engineer cert a plus

Tools

Azure Synapse Analytics Notebooks

Azure Synapse Pipelines

Azure Key Vault

Git (Azure DevOps)

Power BI

Tableau

Job description

Bluelight is a leading software consultancy seeking a Data Engineer to design and maintain ETL data pipelines using PySpark in Azure Synapse notebooks and pipelines. You will build scalable data warehouses, extract data from REST APIs, SQL databases, and CSV files, and ensure data quality and governance.

The role emphasizes collaboration with data architects and cross-functional teams, with opportunities for growth, remote work options, and ongoing education in a fast-paced environment.