NEWPosted 2 hours ago

Job ID: JOB_ID_10735

Role Overview

Client is undergoing a major Data Modernization initiative, transitioning from a legacy on-premise footprint to a high-performance Azure Cloud Lakehouse. We are seeking a Lead Data Engineer with a deep background in Informatica and a proven pivot into Databricks.

You will be the technical linchpin responsible for re-architecting complex insurance data workflows, moving away from GUI-based ETL toward code-driven, scalable PySpark frameworks.

Key Responsibilities

  • Architecture & Modernization: Lead the design and migration of legacy Informatica (PowerCenter/IICS) mapping logic into Databricks (PySpark) and Delta Lake.
  • Pipeline Engineering: Build and optimize end-to-end data pipelines using Azure Data Factory (ADF) and Databricks Workflows to support Claims, Underwriting, and Actuarial analytics.
  • Performance Tuning: Solve complex data engineering challenges, including Spark performance tuning, data skew mitigation, and Z-Ordering within Delta Lake.
  • Governance & Security: Implement and manage data governance policies using Unity Catalog to ensure compliance with insurance industry regulations.
  • Technical Leadership: Act as a subject matter expert (SME) for a team of developers, conducting code reviews and defining the “Golden Standards” for PySpark development within the organization.
  • Stakeholder Collaboration: Partner with Business Analysts and Data Scientists to translate Insurance domain requirements into robust technical architectures.

Technical Qualifications

  • Core ETL: 12+ years of experience in Data Engineering, with at least 8 years focused on Informatica (PowerCenter/IICS).
  • Modern Stack: 4+ years of hands-on experience with Databricks (PySpark, SQL, Delta Lake, Unity Catalog).
  • Azure Ecosystem: Strong proficiency in ADLS Gen2, Azure Data Factory (ADF), and Azure DevOps for CI/CD.
  • Programming: Advanced Python skills and expert-level SQL for complex data transformation and analysis.
  • Domain Expertise: Prior experience in P&C Insurance (Policy, Claims, or Billing data) is highly preferred.
  • Education: Bachelors or Masters degree in Computer Science, Data Analytics, or a related field.

Compensation & Location

Salary: $60 – $80 per hour

Location: Charlotte, NC

Recruiter / Company – Contact Information

Recruiter / Employer: Valzo Soft Solutions

Email: manish@valzosoft.com

Interested in this position?
Apply via Email

Recruiter Notice:
To remove this job posting, please send an email from
manish@valzosoft.com with the subject:

DELETE_JOB_ID_10735

to delete@join-this.com.