Cloud Data Engineer

Kavaliro; Napa (CA); On-site; USD 115,000 - 152,000; Full time

Job summary

Kavaliro is seeking a Cloud Data Engineer for a direct-hire opportunity in California. The role focuses on designing, implementing, and managing data solutions on Microsoft Azure and other cloud platforms, building scalable pipelines, storage, and data processing to enable data-driven decisions.

You will collaborate with analysts and stakeholders, ensure data quality and security, mentor IT staff, and support continuous learning in cybersecurity, while managing relationships with the System

Qualifications

• Bachelor’s degree in computer science or a closely related field.
• Minimum five+ years of Azure migration experience.
• Proven experience with Python and SQL for data processing.
• Experience with Azure components like Data Factory, Databricks, Data Lake Storage, SQL DW and PySpark.
• ETL in a data warehouse environment and big data pipeline optimization.
• Ability to multitask in a fast-paced setting and collaborate with vendors.

Responsibilities

• Design and implement scalable data processing pipelines and storage environments in cloud services.
• Collaborate with analysts and stakeholders to understand data requirements and deliver solutions.
• Ensure data is cleansed, transformed, and optimized for storage and analysis.
• Build data pipelines and workflows for data ingestion, processing, and distribution.
• Load transformed data into data warehouses and analytics applications.
• Monitor and troubleshoot the data environment for high availability and performance.
• Provide on-call support for critical production systems.
• Implement data security measures including encryption, access controls, and auditing.
• Use analytics services to provide actionable insights for decision-making.
• Maintain documentation on data procedures, systems, and architectures.
• Mentor IT staff on security best practices and emerging technologies.
• Collaborate on technology plans to enhance security posture and SI relationships.
• Review Statements of Work, deliverables, and invoices; measure SI performance against KPIs/SLAs.

Skills

Python

SQL

Azure Data Factory

Databricks

Data Lake Storage

SQL DW

Analysis Services

PySpark

ETL

Big data pipelines

JIRA

Education

Bachelor's degree in Computer Science or related field

Tools

JIRA

Job description

Job Description

Our client is seekding a Cloud Data Engineer for a direct hire opportunity. The Cloud Data Engineer’s responsibilities include designing, implementing, and managing data solutions on Microsoft Azure and other cloud platforms. Oversee the creation and maintenance of data pipelines, storage solutions, processing, analysis, and integration to facilitate data-driven decision-making. This role requires a combination of technical skills, analytical problem-solving, and collaboration with cross-functional teams to achieve data-driven goals.

Job Description

Our client is seekding a Cloud Data Engineer for a direct hire opportunity. The Cloud Data Engineer’s responsibilities include designing, implementing, and managing data solutions on Microsoft Azure and other cloud platforms. Oversee the creation and maintenance of data pipelines, storage solutions, processing, analysis, and integration to facilitate data-driven decision-making. This role requires a combination of technical skills, analytical problem-solving, and collaboration with cross-functional teams to achieve data-driven goals.

Pay Rate

$115,037 to $151,561 annually commensurate with experience

Duties

• Design and implement scalable and secure data processing pipelines and storage environments using cloud services and tools.
• Collaborate with technical analysts, business analysts, and line of business stakeholders to understand data requirements and implement appropriate data solutions.
• Ensure data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements.
• Build effective data pipelines and workflows to streamline data ingestion, processing, and distribution tasks.
• Load transformed data into storage and reporting structures in destinations including data warehouse, reporting systems and analytics applications.
• Monitor and troubleshoot issues with the data environment to maintain high availability and performance.
• Provide on-call support for critical production systems.
• Implement data security measures, including encryption, access controls, and auditing, to protect sensitive information.
• Utilize analytics services to provide actionable insights and support data-driven decision-making.
• Maintain effective documentation regarding data procedures, systems, and architectures to maintain clarity and ensure compliance with regulatory standards.
• Mentor IT staff on security best practices and emerging technologies.
• Participate in continuous learning to stay current with cybersecurity advancements.
• Collaborate with colleagues to draft technology plans that enhance security posture.
• Manage the relationship with the System Integrator.
• Review Statements of Work (SOWs), deliverables, and invoices.
• Measure SI performance against KPIs and SLAs.
• Resolve contractual or delivery issues.

Requirements

• The ideal candidate has a combination of education and experience equivalent to a bachelor’s degree in computer science or closely related field
• Minimum of five plus years of Azure migration experience.
• Proven experience using data processing languages such as Python and SQL.
• Experience with Azure: Data Factory, Databricks, Data Lake Storage, SQL DW, Analysis Services, and API – PySpark.
• ETL experience in a data warehouse environment using slowly changing dimensions.
• Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
• Ability to multi-task in a fast-paced environment.
• Ability to prioritize tasks and projects using a browser -based tracking system.
• Familiarity with JIRA preferred.
• Current knowledge of laws, rules, and regulations pertaining to information technology relating to financial institutions.
• Ability to work independently with limited supervision.
• Ability to effectively interact and communicate with outside vendors and internal clients.
• Ability to communicate effectively both verbally and in writing.
• Ability to establish and maintain effective working relationships with a diverse group of people.
• Expereince working with an “SI” (System Integrator)
• Experience managing of 500 – 800+ pipelines during a large-scale enterprise cloud migration

Kavaliro provides Equal Employment Opportunities to all employees and applicants. All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Kavaliro is committed to the full inclusion of all qualified individuals. In keeping with our commitment, Kavaliro will take the steps to assure that people with disabilities are provided reasonable accommodations. Accordingly, if reasonable accommodation is required to fully participate in the job application or interview process, to perform the essential functions of the position, and/or to receive all other benefits and privileges of employment, please respond to this posting to connect with a company representative.

Job Requirements

On-Site