Job ID: JOB_ID_164

Role Overview

We are seeking a highly skilled and visionary Foundry Lead Data Engineer to join our innovative technology team. In this pivotal role, you will be at the forefront of designing and implementing sophisticated data solutions using PySpark and the Palantir Foundry platform. This position is designed for a technical leader who thrives in a hybrid work environment and is passionate about leveraging data to drive societal progress and operational excellence. As a Lead Engineer, you will not only be responsible for high-level architectural decisions but also for the hands-on development of scalable data pipelines that power our business intelligence and analytics initiatives.

Key Responsibilities

  • Architect and implement highly scalable data processing workflows using PySpark to handle massive datasets with efficiency and precision.
  • Lead the integration of Palantir Foundry solutions into existing enterprise systems, ensuring seamless data flow and interoperability across platforms.
  • Develop, maintain, and optimize robust data models that serve as the foundation for advanced business analytics and strategic decision-making.
  • Provide expert technical guidance and mentorship to junior and mid-level data engineers, fostering a collaborative environment of continuous learning and professional growth.
  • Analyze complex and disparate data sets to uncover hidden trends, patterns, and insights that inform and drive the company’s strategic roadmap.
  • Enforce rigorous data security protocols and ensure full compliance with industry standards and regulations to safeguard sensitive information.
  • Oversee the end-to-end deployment of Palantir Work Model solutions, ensuring that all technical components are functional and aligned with business requirements.
  • Evaluate and recommend emerging technologies, tools, and methodologies to enhance existing data architectures and maintain a competitive edge.
  • Optimize data processing pipelines to significantly reduce latency and improve overall system performance and reliability.
  • Collaborate closely with cross-functional stakeholders to define project scopes, requirements, and deliverables, ensuring technical solutions meet business goals.
  • Maintain comprehensive documentation of architectural designs, data flows, and processes to facilitate knowledge sharing and future system maintenance.

Technical Requirements and Qualifications

  • Minimum of 10 years of professional experience in data architecture, solution design, and large-scale data engineering.
  • Proven expertise in PySpark for complex data processing, transformation, and analysis tasks.
  • Extensive hands-on experience with the Palantir Foundry ecosystem, including data integration, ontology management, and application building.
  • Strong background in developing scalable data models and architectures within cloud or hybrid environments.
  • Solid understanding of data security, governance, and compliance best practices within the technology sector.
  • Exceptional problem-solving skills with the ability to navigate and resolve complex technical challenges in a fast-paced setting.
  • Excellent communication and interpersonal skills, with a demonstrated ability to lead cross-functional teams and present technical concepts to non-technical audiences.
  • Bachelor’s degree in Computer Science, Information Technology, or a related field is required; a Master’s degree is highly preferred.
  • Proactive approach to innovation, with a track record of identifying and implementing cutting-edge solutions that improve business outcomes.

Work Environment and Culture

This role offers a flexible hybrid work model, allowing for remote collaboration while maintaining a strong connection to the team. We value adaptability and the ability to use modern remote collaboration tools effectively. Our culture is built on the principles of continuous improvement, technical excellence, and a commitment to delivering impactful technology solutions that make a difference.

Special Requirements

Remote, Contract

Compensation & Location

Salary: $165,000 – $215,000 per year (Estimated)

Location: Remote

Recruiter / Company – Contact Information

Recruiter / Employer: Smart IT Frame LLC

Email: gayathri.s@smartitframe.com

Interested in this position?
Apply via Email

Recruiter Notice:
To remove this job posting, please send an email from
gayathri.s@smartitframe.com with the subject:

DELETE_JOB_ID_164

to delete@join-this.com.