Job ID: 1868
Position Overview: Senior DataBricks Engineering
We are seeking a highly skilled and experienced DataBricks Developer for a high-impact, onsite role in Palm Beach, Florida. This position is designed for a seasoned professional with at least 10 years of experience in the data engineering space, specifically focusing on big data technologies and cloud-based analytics platforms. As a DataBricks Developer, you will be at the forefront of our data strategy, building and maintaining the pipelines that power our advanced analytics and machine learning initiatives. This is a 12-month contract with a high probability of extension, offering a stable yet dynamic environment for a senior engineer to showcase their expertise in performance optimization and data governance.
Core Technical Responsibilities
- Develop, test, and deploy highly scalable data processing pipelines within the Databricks environment using Spark, Python, or Scala.
- Implement advanced performance optimization techniques to ensure that data processing is both cost-effective and time-efficient.
- Collaborate extensively with data science and analytics teams to integrate complex data solutions into broader business intelligence frameworks.
- Support and enhance data architecture initiatives that leverage the latest Databricks features, including Delta Lake and Unity Catalog.
- Provide expert-level troubleshooting for operational issues, ensuring the reliability and integrity of production data systems.
- Establish and enforce rigorous data governance and security protocols to maintain compliance with industry regulations and internal standards.
- Create comprehensive technical documentation and lead knowledge transfer sessions to elevate the technical capabilities of the broader engineering team.
Key Deliverables and Performance Metrics
The successful candidate will be responsible for delivering production-ready data pipelines that meet strict performance SLAs. You will be expected to produce detailed optimization reports that demonstrate improvements in processing speed and resource utilization. Furthermore, you will develop and maintain data visualization dashboards that provide stakeholders with clear insights into pipeline health and data quality. Documentation of security best practices and governance frameworks is a critical deliverable, ensuring that our data assets are protected and managed according to the highest industry standards.
Qualifications and Professional Experience
- Minimum of 10 years of professional experience in data engineering or software development.
- Proven track record of success in developing and maintaining Databricks-based solutions in a production environment.
- Deep understanding of big data technologies and the ability to optimize complex data analysis workflows.
- Experience with advanced querying tools and data visualization services.
- Strong background in implementing data security and governance best practices.
- Excellent communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.
This onsite role in Palm Beach offers a unique opportunity to work in a beautiful location while tackling some of the most challenging problems in modern data engineering.
Special Requirements
Onsite in Palm Beach, FL; 10 Years Experience Required; 12-month contract with extension.
Compensation & Location
Salary: $145,000 – $198,000 per year (Estimated)
Location: Palm Beach, FL
Recruiter / Company – Contact Information
Email: kumarp.staffing1803@gmail.com
Recruiter Notice:
To remove this job posting, please send an email from
kumarp.staffing1803@gmail.com with the subject:
DELETE_1868