Job ID: JOB_ID_1783
Role Overview
We are seeking a highly skilled DataBricks Developer for a critical onsite role in Palm Beach, Florida. This position is focused on building and optimizing sophisticated data processing pipelines that drive business intelligence and advanced analytics. As a key member of the data engineering team, you will leverage the power of the Databricks platform to transform raw data into actionable insights, ensuring that our data infrastructure is scalable, secure, and high-performing. This is a contract position for 12 months with a high possibility of extension based on performance and project needs.
Key Responsibilities
- Pipeline Engineering: Design, develop, and maintain end-to-end data pipelines using Databricks and Spark.
- Performance Tuning: Implement advanced optimization techniques to reduce latency and improve the efficiency of data processing jobs.
- Cross-Functional Collaboration: Work closely with data scientists and analysts to integrate data solutions into their workflows.
- System Reliability: Provide ongoing support and troubleshooting for Databricks-based tools, resolving operational issues promptly.
- Data Governance: Establish and enforce strict data governance and security protocols to ensure compliance with regulatory standards.
- Documentation: Create comprehensive technical documentation for all processes and system behaviors to facilitate knowledge sharing.
- Knowledge Transfer: Lead knowledge transfer sessions to train team members on data processing techniques and tools.
Technical Skills
- Expertise in Databricks, Apache Spark, and Big Data technologies.
- Strong proficiency in SQL, Python, or Scala for data manipulation.
- Experience with data visualization tools and dashboard creation.
- Knowledge of cloud-based data warehousing solutions.
- Familiarity with Agile development practices and DevOps integration.
Qualifications
- At least 10 years of professional experience in data engineering or software development.
- Proven track record of delivering complex data projects in a corporate environment.
- Strong understanding of data architecture principles and security best practices.
- Excellent communication skills and the ability to lead knowledge transfer sessions.
- Local candidates with H1B visa status are preferred for this onsite engagement.
The project involves working with massive datasets that require sophisticated partitioning and indexing strategies within the Delta Lake architecture. You will be responsible for implementing automated testing for data pipelines to ensure data quality and integrity. Furthermore, you will play a lead role in migrating legacy ETL processes to modern Databricks workflows, significantly reducing processing time and costs. The onsite nature of this role in Palm Beach allows for direct collaboration with business stakeholders, enabling a deeper understanding of the data’s impact on the organization’s strategic goals. You will be expected to stay current with the latest Databricks releases and features, such as Unity Catalog and Photon, to continuously enhance the platform’s capabilities. This role requires a developer who can work independently while also contributing to the broader team goals through advanced querying tools and data visualization services.
Special Requirements
Local H1b Visa only; Onsite requirement.
Compensation & Location
Salary: $124,800 – $135,000 per year
Location: Palm Beach, FL
Recruiter / Company – Contact Information
Recruiter / Employer: Arbor Teksystems
Email: chetnad@arborteksys.com
Recruiter Notice:
To remove this job posting, please send an email from
chetnad@arborteksys.com with the subject:
DELETE_JOB_ID_1783