Principal Data Engineer

Curate Partners; Boston (MA); On-site; USD 120,000 - 150,000; Full time

Job summary

Curate Partners is seeking a seasoned Data Engineer in Boston, Massachusetts, to design and enhance enterprise-grade data platforms. This role requires at least 8 years of experience in data engineering along with strong proficiency in SQL and Python, and hands-on experience with major cloud platforms like AWS and Azure.

The successful candidate will build scalable data pipelines, optimize solutions for performance, and implement data governance frameworks. Collaboration with cross-functional teams and mentorship of engineering teams will be key parts of the job.

Qualifications

• 8+ years of experience in data engineering or platform development.
• Strong proficiency in large-scale data processing frameworks.
• Hands-on experience with cloud platforms and analytics tools.

Responsibilities

• Design and enhance enterprise-grade data platforms.
• Build and maintain scalable data pipelines and frameworks.
• Optimize data solutions for cost efficiency and performance.

Skills

SQL

Python

Apache Spark

Data Governance

Cloud Platforms

Apache Kafka

Data Modeling

Education

Bachelor’s degree in computer science or related field

Tools

AWS

Azure

Google Cloud

Databricks

Snowflake

Apache Airflow

Job description

• This role does not offer sponsorship and is open to candidates authorized to work in the U.S. without sponsorship (USC/GC).

Responsibilities

• Design and enhance enterprise-grade data platforms, including ingestion, transformation, storage, orchestration, and data serving layers for both batch and streaming use cases
• Build and maintain scalable data pipelines, reusable frameworks, and enterprise data models to support analytics, artificial intelligence, and operational reporting
• Define and manage semantic data layers while implementing governance controls such as data quality validation, lineage tracking, metadata management, and secure access
• Establish engineering standards for development, testing, version control, documentation, and continuous integration and delivery practices
• Optimize data solutions for cost efficiency, scalability, and performance using modern engineering and operational practices
• Lead technical design reviews, incident response activities, and root cause analysis to improve platform stability and reliability
• Collaborate with data science teams to deploy and operationalize machine learning models for batch and real-time use cases
• Partner with cross-functional teams including analytics, security, and architecture to deliver compliant, high-quality data solutions
• Evaluate new technologies and guide architectural decisions, including build-versus-buy considerations
• Mentor engineering teams through technical guidance, code reviews, and knowledge sharing to raise overall engineering standards
• Promote consistency and reuse across distributed teams by sharing best practices and standardized components

Required Experience and Skills

• Bachelor’s degree in computer science, statistics, applied mathematics, or a related quantitative field
• At least 8 years of experience in data engineering or data platform development, including significant experience in senior or principal-level roles
• Strong proficiency in SQL, Python, and large-scale data processing frameworks such as Apache Spark or PySpark
• Hands-on experience with major cloud platforms such as AWS, Azure, or Google Cloud, along with modern data platforms such as Databricks or Snowflake
• Experience with streaming technologies such as Apache Kafka or similar tools and orchestration frameworks such as Apache Airflow
• Strong background in data modeling, including dimensional, data vault, and domain-oriented approaches
• Experience implementing data governance frameworks, including quality controls, lineage tracking, metadata management, and access controls
• Knowledge of software engineering practices including CI/CD, infrastructure as code, and automated testing
• Proven ability to lead complex technical initiatives, influence stakeholders, and guide engineering teams
• Strong communication skills with the ability to present technical concepts clearly and effectively
• Ability to manage multiple priorities and work effectively in fast-paced environments