Data Engineer (Salesforce Data Cloud)

CareCone Group; Sydney; On-site; AUD 120,000 - 160,000; Full time

Job summary

CareCone Group is seeking a Data Engineer to own end-to-end data pipelines across Salesforce Data Cloud and Databricks in Sydney. You will design DMOs/DSOs, ingest data, and manage identity resolution for personalised experiences at scale.

You will collaborate with DevOps and Data Architects to ensure reliable, observable pipelines across Dev to Production, with strong documentation and governance practices.

Qualifications

• 3–5+ years of hands-on data engineering experience in a complex enterprise environment.
• Demonstrated experience with Salesforce Data Cloud - including DMO/DSO development, data ingestion, identity resolution, and end-to-end deployment lifecycle management.
• Strong working knowledge of Databricks, data lake, or similar UDP environment, including batch and streaming ingestion patterns and pipeline orchestration.
• Proficiency in SQL and at least one scripting language (Python preferred) for data transformation and pipeline development.
• Experience working across multiple delivery environments (Dev / SIT / UAT / Pre-Production / Production) with formal change management and release governance processes.
• Ability to write clear technical documentation - implementation plans, data dictionaries, and solution design notes.

Responsibilities

• Design and build Data Model Objects (DMOs), Data Stream Objects (DSOs), and source-to-model mapping layers within Salesforce Data Cloud.
• Develop and maintain data offload solutions between the Data Cloud and upstream or downstream platforms.
• Configure and validate Identity Resolution rulesets and unified contact profiles within the Data Cloud environment.
• Support deployments across development, SIT, UAT, pre-production, and production environments with plans, change management docs, and post-go-live validation.
• Manage and govern data onboarding processes including data dictionaries, attribute mapping reviews, and build validation.
• Ingest and transform data from multiple sources into Databricks with batch and streaming patterns and orchestrate pipelines.
• Collaborate with DevOps, Data Architects, and operations teams for end-to-end pipeline reliability and observability.
• Provide SIT, UAT, and data validation support across environments; ensure data availability for personalisation and customer engagement use cases.

Skills

Salesforce Data Cloud

Databricks

SQL

Python

Technical documentation

Tools

Azure Databricks

Delta Lake

Airflow

Azure DevOps

Git

Job description

Location- Sydney

Description

We are looking for a skilled and versatile Data Engineer to join our team - a role that spans two critical platforms: Salesforce Data Cloud, and Databricks (Unified Data Platform).

You will be responsible for designing, building, testing, and deploying data pipelines, data model objects, and ingestion solutions that power personalisation capabilities at scale across batch and real-time channels.

Key Responsibilities

Salesforce Data Cloud

• Design and build Data Model Objects (DMOs), Data Stream Objects (DSOs), and source-to-model mapping layers within Salesforce Data Cloud (SFDC360).
• Develop and maintain data offload solutions between the Data Cloud and upstream or downstream platforms.
• Configure and validate Identity Resolution (IR) rulesets and unified contact profiles within the Data Cloud environment.
• Support deployments across development, SIT, UAT, pre-production, and production environments - including preparation of implementation plans, change management documentation, and post-go-live validation.
• Manage and govern data onboarding processes including data dictionaries, attribute mapping reviews, and build validation.

Databricks (Unified Data Platform)

Demonstrate a sound understanding of:

• Ingestion and transformation of data from multiple source domains (e.g. CRM feeds, third-party data, digital signals) into the platform following approved design principles and data governance standards.
• Building and maintaining batch and streaming transform pipelines and workflow orchestration.
• Data dictionary documentation and cloud object storage design for data offload scenarios.
• Collaboration with extended team like DevOps, Data Architects, and operations teams to ensure end-to-end pipeline reliability and observability.
• Demonstrate a sound understanding of how real-time decisioning platforms (such as Pega) consume and depend on data engineering outputs.
• Work with decisioning and analytics teams to ensure accurate and timely data availability for personalisation and customer engagement use cases.
• Participate in cross-team planning and technical reviews as they relate to data dependencies for decisioning workflows.

Cross-Platform & Delivery

• Deliver across multiple use cases simultaneously in an Agile squad model, managing dependencies across platforms and teams.
• Author and maintain technical documentation, including solution design notes, implementation plans, and deployment runbooks.
• Actively participate in sprint ceremonies, backlog refinement, and cross-team dependency management.
• Provide SIT, UAT, and data validation support across environments; work with QA and operations teams to validate data integrity pre- and post-release.

Skills & Experience

• 3–5+ years of hands-on data engineering experience in a complex enterprise environment.
• Demonstrated experience with Salesforce Data Cloud - including DMO/DSO development, data ingestion, identity resolution, and end-to-end deployment lifecycle management.
• Strong working knowledge of Databricks, data lake, or similar UDP environment, including batch and streaming ingestion patterns and pipeline orchestration.
• Proficiency in SQL and at least one scripting language (Python preferred) for data transformation and pipeline development.
• Experience working across multiple delivery environments (Dev / SIT / UAT / Pre-Production / Production) with formal change management and release governance processes.
• Ability to write clear technical documentation - implementation plans, data dictionaries, and solution design notes.

Highly Desirable

• Familiarity with cloud-native data engineering patterns and object storage-based data offload architectures (e.g. AWS S3, Azure Data Lake).
• Understanding of real-time decisioning platforms such as Pega, and how they integrate with data engineering layers.
• Exposure to MarTech or Transformation data programs in a large enterprise or telco context.
• Experience with workflow orchestration tools such as Apache Airflow or equivalent.
• Understanding of data governance frameworks, data lineage, and data quality processes.
• Knowledge of Agile or SAFe delivery methodology and working in cross-functional squads.

What We're Looking For in You

You are someone who thrives in a hybrid engineering role - comfortable getting your hands dirty with pipelines and data modelling one moment and engaging stakeholders on a deployment plan the next. You take pride in data quality, can navigate multiple platforms without losing sight of the bigger picture, and are energised by the challenge of delivering personalisation capability that directly impacts customer experience at scale.

Primary or Mandatory skills

2. Scala Spark

3. SQL

4. Python

Good to have skills

• Databricks, SQL, Python, Data warehousing concepts
• Detailed Job description (Role & Responsibility, Skill and Experience range)
• We‿re looking for a hands on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL
• Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
• Proven expertise in designing real time and near real time integrations
• Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.
• Expertise in cloud platforms and related technologies, specifically:

o Data Platforms ‿ (e.g. Databricks) and big data technologies.

o Real time data processing frameworks Spark Streaming, Apache/Confluent Kafka

• Proficiency in SQL, Python, and data integration tools.
• Familiarity with Master Data Management solutions
• Excellent communication and stakeholder management skills.