Senior Data Platform Engineer

NewVision Software; San Francisco (CA); Remote; USD 180,000 - 240,000; Full time

Job summary

NewVision Softcom & Consultancy Pvt. Ltd. in San Francisco offers a hybrid, US-based remote role for a senior data engineering expert.

The candidate will design and manage enterprise-grade data ecosystems, including cloud-native platforms, big data pipelines, and analytics integrations. Collaboration with architects, DevOps, and business teams is essential to deliver scalable, secure, high-performing solutions.

Qualifications

• Advanced SQL, query optimization, partitioning, and transaction handling.

Responsibilities

• Design and implement robust, scalable database architectures for transactional and analytical workloads.
• Define partitioning strategies, indexing, and normalization for optimal performance and maintainability.
• Deploy and manage microservices-based applications on Azure Service Fabric clusters.
• Ensure high availability, fault tolerance, and scalability of distributed services.
• Design and orchestrate ETL/ELT pipelines for batch and streaming data.
• Build and optimize big data processing workflows using Apache Spark on Databricks.
• Collaborate with BI teams to design semantic models and optimize data for reporting.
• Provide mentorship and technical guidance to junior engineers and developers.

Skills

Platform Development

Platform Architecture

Platform Integration

Database Architecting

Performance Tuning

Big Data

Tools

Azure Service Fabric

Azure Data Factory

Azure Databricks

Azure Storage

Spark

Delta Lake

Power BI

Job description

Company Name New Vision Softcom & Consultancy Pvt. Ltd

Function Business Units (BU)

Department/Practice AI & Analytics

Region USA

Country US

Base office location San Francisco

Working model Hybrid

Weekly Off US standard

State California

Skills

Skill

Platform Development Platform Architecture Platform Integration Database Architecting Performance Tuning Big Data

CERTIFICATION

Working Language

Job Description

Location - US Remote. Candidate to be based out of US.

Mandatory Skills

-designing, optimizing, and managing enterprise-grade data ecosystems.

-mastery of database architecture, performance tuning, and transaction management,

- hands-on experience in cloud-native platforms, big data frameworks, and advanced analytics pipelines.

The ideal candidate will be comfortable working across distributed systems, modern data engineering tools, and collaborating with architects, DevOps, and business teams to deliver scalable, secure, and high-performing solutions.

KEY RESPONSIBILITIES

• Design and implement robust, scalable, and secure database architectures for transactional and analytical workloads.
• Define partitioning strategies, indexing, and normalization for optimal performance and maintainability.

Performance Tuning & Optimization

• Conduct advanced query optimization and execution plan analysis.
• Implement proactive monitoring and troubleshooting for high-volume, mission-critical systems.
• Deploy and manage microservices-based applications on Azure Service Fabric clusters.
• Ensure high availability, fault tolerance, and scalability of distributed services.
• Optimize communication patterns and stateful/stateless service design for data-intensive workloads.
• Design and orchestrate complex ETL/ELT pipelines for batch and streaming data.
• Build and optimize big data processing workflows using Apache Spark on Databricks.
• Implement Delta Lake for ACID transactions on large-scale data lakes.
• Develop scalable solutions for machine learning and advanced analytics.
• Manage versioned data storage for reliability and reproducibility.
• Optimize schema evolution and data compaction strategies.
• Collaborate with BI teams to design semantic models and optimize data for reporting.
• Ensure data pipelines deliver clean, consistent, and performant datasets for visualization.
• Work closely with architects, DevOps, and business stakeholders to align technical solutions with strategic objectives.
• Provide mentorship and technical guidance to junior engineers and developers.

REQUIRED SKILLS & QUALIFICATIONS

Database Mastery

• Advanced SQL, query optimization, partitioning, and transaction handling.

Performance Tuning

• Proven ability to diagnose and resolve complex performance bottlenecks.
• Hands-on experience with Azure Service Fabric, Azure Data Factory, Azure Databricks, and Azure Storage.
• Strong knowledge of Spark, Delta Lake, and distributed data processing.
• Familiarity with Power BI and data modeling best practices.

Troubleshooting & Optimization

• Ability to resolve issues across multi-tiered systems under pressure.

Cross-Functional Collaboration

• Excellent communication and stakeholder management skills.

PREFERRED QUALIFICATIONS

• Experience with CI/CD pipelines for data solutions.
• Familiarity with data governance, security, and compliance frameworks.
• Exposure to machine learning workflows and real-time streaming architectures.