Job ID: JOB_ID_4017
Job Summary:
We are looking for an experienced Databricks Lead to architect and implement scalable Lakehouse data platforms. This role requires strong expertise in Apache Spark, Delta Lake, Databricks SQL, Python, SQL, and GCP platform services. The ideal candidate will lead migration efforts, implement data governance, and optimize performance for data engineering teams.
Responsibilities:
- Architect and implement scalable Lakehouse data platforms using Databricks and Delta Lake.
- Design robust batch and streaming data pipelines leveraging Apache Spark, structured streaming, and modern ELT patterns.
- Lead migration of Jobs from other cloud data platforms to Databricks.
- Implement secure data governance, access control, and lineage using Unity Catalog.
- Architect integrations with cloud platforms such as Google Cloud Platform.
- Optimize performance and manage compute costs through efficient cluster configuration and Spark workload tuning.
- Collaborate with data engineers, analytics teams, and ML engineers to enable scalable data products and analytics.
- Define best practices for data quality, reliability, and observability across the data platform.
- Provide technical leadership, architecture guidance, and mentorship to data engineering teams.
Mandatory Skills:
- Strong expertise in Apache Spark, Delta Lake, Databricks SQL, Python, SQL and GCP Platform services.
Job Description:
- 10+ years of experience in data engineering or data architecture in Big Data platforms.
- 3+ years hands-on experience with Databricks platform architecture.
- Strong expertise in Apache Spark, Delta Lake, Databricks SQL, Python, SQL and GCP Platform services.
Keywords:
machine learning, golang, green card, Idaho
Special Requirements
USC or GC only. Onsite role, 3 days biweekly.
Compensation & Location
Salary: $140,000 – $180,000 per year (Estimated)
Location: Bolingbrook, IL
Recruiter / Company – Contact Information
Recruiter / Employer: Sunray Enterprise, Inc.
Email: manoj@sunraycorp.com
Recruiter Notice:
To remove this job posting, please send an email from
manoj@sunraycorp.com with the subject:
DELETE_JOB_ID_4017