Parallel Software Engineer Engineering Mountain View, CA / Hyderabad, IN / Remote

DataPelago, Inc.; Mountain View, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive compensation

Stock options

Comprehensive benefits package

Job summary

DataPelago, Inc. is seeking a Parallel Software Engineer to advance our data processing engine across GPUs and SIMD-capable CPUs. You will architect, design, and implement parallel operators and functions, driving performance and scalability for large-scale data workloads.

Join a team pushing the forefront of accelerated computing, collaborating closely with the execution engine group. This role offers remote flexibility while maintaining close collaboration with Mountain View and Hyderabad

Qualifications

• BS or MS in Computer Science or related field.
• 5+ years of relevant experience for this role.
• 3+ years of experience with SIMD/GPUs or accelerated computing.
• Strong knowledge of computer architecture and debugging/ profiling tools.

Responsibilities

• Architect data processing engine components to exploit GPU and SIMD parallelism.
• Lead design and optimization of operators and functions accelerated on hardware.
• Implement, test, and optimize parallel code across diverse acceleration hardware.
• Analyze advances in accelerated computing to maintain product leadership.
• Collaborate with execution engine teams to integrate parallel components.
• Uphold best practices in design reviews, testing, CI/CD, and issue resolution.

Skills

C/C++ programming

Linux

Parallel computing

Performance optimization

Education

Bachelor's degree in Computer Science

Master's degree in Computer Science

Tools

CUDA

ROCm

AVX

Job description

Parallel Software Engineer

Mountain View, CA / Hyderabad, IN / Remote

About DataPelago:

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Ourteam of engineers drive and adopt advances in hardware-accelerated computing, parallel processingof large-scale data, query optimization, distributed systems, compilers, machine learning, andcloud-native computing. We are looking for specialists to join our engineering team and shape thefuture of accelerated data processing.

The Opportunity:

As a Parallel Software Engineer, you will be a key individual contributor in developing advanced parallel software that unlocks the full potential of diverse hardware accelerators including GPUs and SIMD-capable CPUs. You will enhance functional breadth, performance, scale, and reliabilityof data processing operators that are integral to our data processing engine. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team ofengineers.

What You'll Do:

• Architect: Influence the architecture of how our data processing engine harnesses the par-allelism in GPUs and SIMD-capable CPUs efficiently in processing diverse large-scale data.
• Design: Lead design of functional and performance enhancements to operators and functions that are accelerated by our engine.
• Core Development: Individually design, implement, test, optimize, and maintain parallel implementations of operators and functions on diverse acceleration hardware.
• Innovation and Differentiation: Analyze advances in accelerated computing hardware, programming models, and related tools and ensure our engine extends technology and product leadership.
• Collaboration: Partner effectively with the execution engine engineering team in integrating parallel software components with the overall engine.
• Continuous Improvement: Foster best practices in design and code reviews, testing, CI/CD,and issue resolution to maintain highest product quality, security, efficiency, & productivity.

What You'll Bring:

• Bachelor's degree in Computer Science or a related field with 5+ years of relevant experience OR a Master's degree in Computer Science or a related field with 3+ years of relevant experience.
• 3+ years of deep technical experience in developing production applications that process large scale data using SIMD-extensions of CPUs (e.g., AVX), GPU programming models (e.g., CUDA, ROCm), or equivalent accelerated computing framework.
• Demonstrated experience working with software libraries, development tools, and profiling tools specific to parallel and accelerated computing.
• Demonstrated experience in troubleshooting and resolving functional and performance anomalies in both pre- and post-production scenarios.
• Strong knowledge of computer architecture.
• Exceptional programming skills in C, C++.
• Extensive development experience in Linux environments.
• Strong analytical and problem-solving skills with a passion for performance optimization.

Location Considerations:

We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and atremote locations.

Why Join DataPelago?:

• Technology Leadership: Shape the architecture and development of core parallel data processing functions of our engine.
• Cutting-Edge Innovation: Work on challenging problems at the forefront of accelerated computing and data processing.
• Significant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.
• Growth: Expand your technical expertise and scope of responsibilities working with other talented engineers and with a growing product.
• Competitive compensation, stock options, comprehensive benefits package, leadership development opportunities.