NEWPosted 2 hours ago

Job ID: JOB_ID_6324

Job Summary:

We are seeking a skilled Databricks Engineer with strong experience in Databricks and Healthcare data processing. This role involves developing and maintaining ETL pipelines, ensuring data availability for business intelligence tools, and working with various data transfer and ingestion technologies.

Key Responsibilities:

  • Develop, implement, and maintain robust ETL pipelines using Databricks and PySpark.
  • Ensure timely ingestion of source data and transformation for consumption by Power BI and Tableau.
  • Experience with API ingestion, including connecting to external APIs, extracting data, and performing validation.
  • Utilize Delta Sharing (Deltashare) for secure, real-time data sharing across systems and teams.
  • Collaborate with data analysts and business intelligence teams to meet reporting and analytical needs.
  • Monitor ETL pipelines and workflows using AWS CloudWatch.
  • Understand and implement structural changes in the Data Model as requirements evolve.
  • Work with data replication and Change Data Capture (CDC) processes, potentially using tools like Qlik.
  • Troubleshoot and resolve issues related to data pipelines and data quality.
  • Ensure data security and compliance with relevant regulations.

Technical Skills Required:

  • Proficiency in Databricks and PySpark for large-scale data processing.
  • Strong SQL proficiency.
  • Experience with data transfer and ETL tools.
  • Familiarity with networking concepts and data storage interfaces.
  • Experience with API ingestion and data validation.
  • Knowledge of Delta Sharing (Deltashare).
  • Familiarity with AWS cloud services, particularly S3 and CloudWatch.
  • Understanding of data models and optimization strategies (partitioning, indexing).
  • Knowledge of data replication and Change Data Capture (CDC) processes.

Qualifications:

  • Proven experience as a Databricks Engineer or similar role.
  • Experience in the Healthcare industry is highly preferred.
  • Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Additional Information:

This is a remote contract position. Candidates must be authorized to work in the US.

Special Requirements

Remote

Compensation & Location

Salary: $100,000 – $150,000 per year (Estimated)

Location: Remote

Recruiter / Company – Contact Information

Email: harsan.s@vysystems.com

Interested in this position?
Apply via Email

Recruiter Notice:
To remove this job posting, please send an email from
harsan.s@vysystems.com with the subject:

DELETE_JOB_ID_6324

to delete@join-this.com.