Data Engineer/Senior Data Engineer

Salesforce, Inc.

Hyderabad, Bengaluru

On-site

INR 2,000,000 - 3,200,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Salesforce in Hyderabad is seeking an experienced Data Engineer to lead the design and delivery of data acquisition and integration projects across batch and real-time streams.

You will architect layered data stacks ensuring data quality, collaborate with product owners, and mentor ETL design while leveraging Python, SQL, Airflow, Spark, and cloud platforms like AWS/Snowflake.

Qualifications

  • 4–12 years experience in data engineering.
  • Build programmatic ETL pipelines with SQL based technologies and platforms.
  • Solid understanding of databases and working with sophisticated datasets.
  • Data governance, verification and data documentation using current tools and future adopted tools and platform.
  • Work with different technologies (Python, shell scripts) and translate logic into well-performing SQL.
  • Perform tasks such as writing scripts, web scraping, getting data from APIs etc.
  • Automate data pipelines using scheduling tools like Airflow.
  • Experience with CI/CD technologies and tools like Jenkins, Ant or Gradle, Github.
  • Be prepared for changes in business direction and understand when to adjust designs.
  • Experience writing production level SQL code and good understanding of Data Engineering pipelines.
  • Experience with Hadoop ecosystem and similar frameworks.
  • Previous projects should display technical leadership with an emphasis on data lake, data warehouse solutions, BI, big data analytics.
  • Knowledge of data modeling techniques and high-volume ETL/ELT design.
  • Experience with version control systems (Github, Subversion) and deployment tools (e.g. CI) required.
  • Experience working with Public Cloud platforms like GCP, AWS, or Snowflake.
  • Ability to work effectively in unstructured and fast-paced environment with self-management.
  • A related technical degree required.

Responsibilities

  • Be responsible for the technical solution design and lead architecture of data acquisition and integration projects.
  • Define the overall solution architecture for layered data stack ensuring data quality and timely insights.
  • Communicate with product owners and analysts to clarify requirements.
  • Craft technical solutions and assemble design artifacts (functional design docs, data flow diagrams, data models).
  • Build data pipelines and data processing tools in open source and proprietary products.
  • Serve as domain expert and mentor for ETL design and related big data technologies.
  • Identify incomplete data, improve quality, and integrate data from multiple sources.
  • Address performance and data quality problems and drive remediation.
  • Design and develop tailored data structures and production-ready data flows.
  • Support Data Science by designing and maintaining big data pipelines for reporting and ML needs.
  • Perform data profiling, sampling, statistical testing, and reliability testing.
  • Evaluate technologies and implement proofs of concept to guide tool selection.
  • Strong SQL optimization and performance tuning in high-volume environments using parallel processing.
  • Familiarity with SQL, Python, Airflow, AWS, Spark, Tableau, AWS EMR, Snowflake.
  • Participate in on-call rotation to address real-time incidents.

Skills

Data engineering
SQL
Python
Airflow
CI/CD
Jenkins
Spark
Hadoop
Data modeling
Data governance
APIs
Web scraping

Education

Related technical degree

Tools

Airflow
Jenkins
Ant
Gradle
GitHub
Subversion
Tableau
Snowflake
AWS
GCP
Spark
Hadoop

Job description

Your Impact:

  • Be responsible for the technical solution design, lead the technical architecture and implementation of data acquisition and integration projects, both batch and real time
  • Define the overall solution architecture needed to implement a layered data stack that ensures a high level of data quality and timely insights
  • Communicate with product owners and analysts to clarify requirements
  • Craft technical solutions and assemble design artifacts (functional design documents, data flow diagrams, data models, etc.)
  • Build data pipelines data processing tools and technologies in open source and proprietary products
  • Serve the team as a domain expert & mentor for ETL design, and other related big data and programming technologies
  • Identify incomplete data, improve quality of data, and integrate data from several data sources
  • Proactively identify performance & data quality problems and drive the team to remediate them. Advocate architectural and code improvements to the team to improve execution speed and reliability
  • Design and develop tailored data structures
  • Reinvent prototypes to create production-ready data flows
  • Support Data Science research by designing, developing, and maintaining all parts of the Big Data pipeline for reporting, statistical and machine learning, and computational requirements
  • Perform data profiling, sophisticated sampling, statistical testing, and testing of reliability on data
  • Clearly articulate pros and cons of various technologies and platforms in open source and proprietary products Implement proof of concept on new technology and tools to help the organization pick the best tools and solutions
  • Strong SQL optimization and performance tuning experience in a high volume data environment that uses parallel processing
  • Teams are using the following: SQL, Python, Airflow, AWS, Spark, Tableau, AWS EMR, Snowflake
  • Participate in the team’s on-call rotation to address sophisticated problems in real-time and keep services operational and highly available

Required Skills:

  • 4 – 12 years experience in data engineering
  • Build programmatic ETL pipelines with SQL based technologies and platforms
  • Solid understanding of databases, and working with sophisticated datasets
  • Data governance, verification and data documentation using current tools and future adopted tools and platform
  • Work with different technologies (Python, shell scripts) and translate logic into well-performing SQL
  • Perform tasks such as writing scripts, web scraping, getting data from APIs etc.
  • Automate data pipelines using scheduling tools like Airflow
  • Experience with CI/CD technologies and tools like Jenkins, Ant or Gradle, Github
  • Be prepared for changes in business direction and understand when to adjust designs
  • Experience writing production level SQL code and good understanding of Data Engineering pipelines
  • Experience with Hadoop ecosystem and similar frameworks
  • Previous projects should display technical leadership with an emphasis on data lake, data warehouse solutions, business intelligence, big data analytics, enterprise-scale custom data products
  • Knowledge of data modeling techniques and high-volume ETL/ELT design
  • Experience with version control systems (Github, Subversion) and deployment tools (e.g. continuous integration) required
  • Experience working with Public Cloud platforms like GPC, AWS, or Snowflake
  • Ability to work effectively in an unstructured and fast-paced environment both independently and in a team setting, with a high degree of self-management with clear communication and commitment to delivery timelines
  • A related technical degree required
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

PW (PhysicsWallah) • Dadri

On-site
INR 900,000 - 1,300,000
Data Engineer
Data Engineer

Staples India • Chennai District

On-site
INR 800,000 - 1,200,000
Data Engineer
Data Engineer

Bacancy Technology Inc • Ahmedabad District

On-site
INR 1,500,000 - 3,000,000
Senior Data Engineer
Senior Data Engineer

Proclink • Gandhamguda

On-site
INR 800,000 - 1,500,000
Data Engineer
Data Engineer

Bayer • Bengaluru

On-site
INR 1,800,000 - 3,200,000
Senior Data Engineer - Data & Analytics
Senior Data Engineer - Data & Analytics

Stryker India Private Limited • Gurugram District

Hybrid
INR 1,800,000 - 3,200,000
Hybrid work model
Flexible work arrangements
Data Engineer
Data Engineer

DataPhi • Pune District

On-site
INR 1,200,000 - 1,800,000
Data Engineer
Data Engineer

Valuenode Private Limited • India

Remote
INR 1,200,000 - 1,800,000
Data Engineer - Data Warehouse, BigData, Python and SQL
Data Engineer - Data Warehouse, BigData, Python and SQL

Tech ECS Limited • Kolkata District

On-site
INR 1,400,000 - 2,100,000
Lead Data Engineer
Lead Data Engineer

Stryker India Private Limited • Gurugram District

Hybrid
INR 3,000,000 - 6,000,000