Senior Data Platform Engineer

INSPYR; Los Angeles (CA); Remote; USD 140,000 - 160,000; Full time

Job summary

A technology solutions provider is seeking a Senior Data Platform Engineer for a remote role. The position requires strong Python/PySpark coding skills and involves enhancing data platform performance, developing scalable solutions, and ensuring data quality. Candidates should have experience with data lakes, cloud services, and workflow automation. Competitive salary range is $140-160K annually, along with a commitment to equal employment opportunities.

Qualifications

• Hands-on experience in a data platform role.
• Ensure data quality and compliance with regulations.
• Collaborate with cross-functional teams for data solutions.

Responsibilities

• Contribute to enhancements for the Data Platform Services.
• Design self-service onboarding capabilities.
• Monitor and troubleshoot data platform issues.

Skills

Python/PySpark coding skills

Understanding of Data Lakes and Data Warehouses

Experience with version control systems (Git)

ANSI SQL and Spark SQL

Data ingestion services design

Workflow automation using Airflow

Cloud services experience (AWS preferred)

Streaming and batch data sources

Tools

Databricks

Hadoop

Job description

Overview

Senior Data Platform Engineer - Direct-Hire/FTE - Remote (US)

Location: Remote (US)

Duration: Direct-Hire/FTE

Compensation: $140-160K Annual Base Salary

Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S.

The Role

This is a hands-on Senior Data Platform Engineer role that will require strong Python/PySpark coding skills. This is NOT a ETL/ELT development role.

What This Role Will Do

• Contribute to the enhancements and continuous improvement of performance and reliability of the existing Data Platform Services to meet the requirements from Data Engineering/Product team
• Design and build self servicing onboarding and automation capabilities in Core Data Platform services in a scalable, reliable and secure way.
• Design and build platform capabilities to support onboarding and operation effectiveness
• Design and develop scalable data platform and integration solutions for various data sources leveraging Databricks Unified Platform
• Ensure data quality and integrity across all data systems.
• Ensure data security and compliance with relevant regulations.
• Implement best practices for data management and governance.
• Develop and maintain documentation for data systems and processes.
• Monitor and troubleshoot data platform issues.
• Participate in on-call rotations/Pagerduty for data platform support.

Technical Skills/Competencies

• Good understanding of Data Lakes, Data Warehouses is MUST to have
• Software development, coding expertise in the Data engineering field using any of the Python/PySpark is MUST to have
• Hands on experience using version control systems such as Git and CI/CD workflows and practice
• Design and Develop data ingestion services that are highly performant, reliable and scalable
• Decent expertise using ANSI SQL and Spark SQL is key to have
• Workflow automation, orchestration using Airflow or equivalent tools stack
• Hands on working knowledge on at least in one of these : Databricks, Hadoop and related stacks
• Working experience
• in at least one of cloud services from any of the Amazon AWS, Google GCP or Microsoft Azure, preferably AWS
• streaming (Kafka) and batch based data sources
• diverse data sources and data formats (xml, json, yaml, parquet, avro, delta) and respective use cases
• Any visualization experience is an advantage so you can bring clarity to prod incidents by using them is a nice to have
• An excellent understanding of the nuances that add complexities around time zones, geo, various data formats, data types across different storage systems is nice to have
• Agile development methodologies using the Atlassian suite: Jira, Confluence
• Collaborate with cross-functional teams to deliver data solutions and provide technical support and guidance to team members.
• Stay up-to-date with the latest data engineering technologies and trends.

About INSPYR Solutions

Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients\' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.

INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.

Information collected and processed through your application with INSPYR Solutions (including any job applications you choose to submit) is subject to INSPYR Solutions’ Privacy Policy and INSPYR Solutions’ AI and Automated Employment Decision Tool Policy: https://www.inspyrsolutions.com/policies/. By submitting an application, you are consenting to being contacted by INSPYR Solutions through phone, email, or text.