Job ID: JOB_ID_645
Job Summary
New York Technology Partners is seeking a visionary Foundry Lead Data Engineer to spearhead the design and deployment of high-scale data architectures within the Palantir Foundry ecosystem. This 100% remote role is critical for optimizing data processing workflows for our clients, Cognizant and PG&E. You will be responsible for leveraging PySpark and integrating complex data models to support utility-scale business intelligence. As a technical lead, you will collaborate with cross-functional teams to build robust pipelines, ensure data security, and mentor junior engineers while driving innovation in grid and operational analytics. This is a high-priority opening requiring a seasoned professional with over a decade of experience in data architecture.
Key Responsibilities
- Design and implement end-to-end data solutions within Palantir Foundry, ensuring seamless integration with existing utility enterprise systems.
- Develop and maintain robust data models and ontologies that serve as the foundation for strategic decision-making and advanced analytics.
- Oversee the deployment of Palantir Work Model solutions, maintaining high availability and reliability for mission-critical grid data.
- Design scalable data processing architectures using PySpark to handle massive datasets and reduce processing latency.
- Continuously monitor and optimize data pipelines to improve performance, ensuring efficient data flow from raw ingestion to final insights.
- Implement rigorous data security protocols and industry compliance standards to safeguard sensitive utility information.
- Provide technical mentorship to junior team members, conducting code reviews and fostering a culture of technical excellence.
- Partner with business stakeholders to define project requirements and deliver technology solutions that align with operational goals.
Mandatory Technical Requirements
- Extensive hands-on experience with Palantir Foundry and Palantir platforms is mandatory.
- Expert proficiency in PySpark for complex data transformation and analysis.
- 10-14 years of professional experience in data architecture and solution design.
- Proven expertise in developing scalable data models and enterprise-level data architectures.
- Strong understanding of utility-scale data challenges and grid analytics.
- Ability to evaluate new tools and features within the Palantir and Spark ecosystems to recommend architectural enhancements.
Education and Leadership
Candidates must possess a Bachelor’s degree in Computer Science or Information Technology, though a Master’s degree is highly preferred. This role requires a leader who can navigate complex stakeholder environments and provide clear technical direction. You will be expected to drive innovation and maintain a high standard of engineering excellence across the team. The position offers the flexibility of a 100% remote work environment while working on high-impact projects for major utility providers. If you are a data engineering expert with a passion for Palantir Foundry and PySpark, we encourage you to apply for this strategic leadership position.
Special Requirements
100% Remote. Client: Cognizant / PG&E.
Compensation & Location
Salary: $165,000 – $220,000 per year (Estimated)
Location: Remote
Recruiter / Company – Contact Information
Recruiter / Employer: New York Technology Partners
Email: aditya.jain@nytpcorp.com
Recruiter Notice:
To remove this job posting, please send an email from
aditya.jain@nytpcorp.com with the subject:
DELETE_JOB_ID_645