NEWPosted 1 hour ago

Job ID: JOB_ID_6356

Job Role: AWS Databricks Engineer

We are seeking an experienced AWS Databricks Engineer for a long-term contract position. This role is 100% remote and requires significant experience in Databricks and the healthcare domain.

Key Responsibilities:

  • Develop and maintain ETL pipelines and workflows in Databricks to ensure timely data transfer and availability for consumption by Power BI and Tableau.
  • Utilize Python, PySpark, Databricks, and SQL for large-scale data processing.
  • Implement data transfer and ETL processes, ensuring data accuracy and completeness.
  • Manage API ingestion, including connecting to external APIs, extracting data, and validating for accuracy.
  • Utilize Delta Sharing (Deltashare) for secure, real-time data sharing across systems and teams.
  • Leverage AWS cloud services such as S3 for data storage and CloudWatch for monitoring ETL pipelines.
  • Understand and implement structural changes in the Data Model as requirements evolve.
  • Work with data replication and Change Data Capture (CDC) processes, including tools like Qlik.
  • Ensure source data is received on schedule and transformed data is available for downstream consumption.

Technical Skills Required:

  • Strong proficiency in Databricks and PySpark.
  • Extensive experience with Python for data processing.
  • Advanced SQL skills.
  • Familiarity with AWS services (S3, CloudWatch).
  • Experience with API ingestion and data validation.
  • Knowledge of Delta Sharing (Deltashare).
  • Understanding of data models and optimization strategies (partitioning, indexing, query performance).
  • Experience with data replication and Change Data Capture (CDC) tools like Qlik.
  • Familiarity with networking concepts and data storage interfaces.
  • Experience in the Healthcare domain is a strong plus.

Additional Requirements:

  • Ability to create and maintain ETL pipelines and workflows.
  • Ensure source data is received on schedule and transformed data is available for Power BI and Tableau consumption.
  • Good understanding of system Entities and Attributes.

Compensation & Location

Salary: $50 – $70 per year (Estimated)

Location: Remote

Recruiter / Company – Contact Information

Recruiter / Employer: GENPACT

Email: maraiselvan.m@smartitframe.com

Interested in this position?
Apply via Email

Recruiter Notice:
To remove this job posting, please send an email from
maraiselvan.m@smartitframe.com with the subject:

DELETE_JOB_ID_6356

to delete@join-this.com.