Job ID: JOB_ID_3772
About the Role:
We are seeking a highly skilled and experienced Knowledge Engineer to join our Enterprise AI Platform Team. This role focuses on building and managing the foundation of our AI systems, transforming diverse data sources into AI-consumable knowledge assets. You will be instrumental in designing scalable knowledge structures, implementing robust data pipelines, and ensuring the governance and security of our AI knowledge base.
Key Responsibilities:
- Enterprise Knowledge Foundation: Transform structured data (SQL, Delta tables) and unstructured sources (documents, contracts, emails, repositories) into AI-consumable knowledge assets.
- Build semantic layers, metadata enrichment frameworks, and ingestion/embedding pipelines for LLM and Agentic AI consumption.
- Design scalable entity models, ontologies, and knowledge graphs supporting multiple AI use cases.
- Databricks-Centric Knowledge Engineering: Build and manage end-to-end knowledge pipelines using Databricks.
- Implement lifecycle pipelines: ingestion, transformation, chunking, embedding, indexing, retrieval.
- Deliver high-quality knowledge layers optimized for AI retrieval and reasoning.
- Governance, Catalog & Lineage: Implement enterprise Data & AI governance including classification, RBAC/ABAC controls, PII detection, masking, tagging, and lineage tracking.
- Ensure all AI knowledge assets comply with security, legal, and regulatory standards.
- Knowledge Architecture for Agentic AI: Design knowledge structures optimized for RAG pipelines and AI agents.
- Implement hybrid retrieval strategies (vector + keyword + metadata filtering).
- Build reusable entity relationships and contextual orchestration patterns for agent systems.
- AI Platform Collaboration: Work with AI Platform Engineers to expose low-latency, governed knowledge endpoints.
- Contribute to prompt-context design, chunking strategies, and retrieval quality improvements.
- Support AI teams in debugging retrieval failures and hallucination issues.
Required Skills:
- Data & Knowledge Engineering: Hands-on experience with Databricks. Strong Python and Advanced SQL skills. Experience with semantic modeling and data pipelines. Experience managing unstructured document repositories.
- AI & Semantic Layer: Strong understanding of RAG (Retrieval Augmented Generation) architectures. Experience with LLM orchestration frameworks such as LangChain or LlamaIndex. Hands-on experience with vector databases like Pinecone, Weaviate, pgvector, or Azure AI Search.
- Governance & Security: Data lineage tracking and governance frameworks. Data masking, classification pipelines, and PII management at scale. Enterprise IAM integration with RBAC / ABAC models.
This is a senior-level role for experienced Data / AI Engineers looking to make a significant impact on our enterprise AI initiatives.
Special Requirements
GC/USC Candidates only. Focus on GenAI, RAG, Knowledge Engineering, Databricks. Location: Open (Enterprise AI Platform Team). Databricks, Python, SQL, RAG, LangChain/LlamaIndex, Vector Databases (Pinecone, Weaviate, pgvector, Azure AI Search), Data Lineage, RBAC/ABAC, PII Management.
Compensation & Location
Salary: $120,000 – $180,000 per year
Location: Open, UT
Recruiter / Company – Contact Information
Email: pramodm@techdestin.com
Recruiter Notice:
To remove this job posting, please send an email from
pramodm@techdestin.com with the subject:
DELETE_JOB_ID_3772