made a pixelated interface with catppuccin styling.
This commit is contained in:
442
README.md
442
README.md
@@ -1,240 +1,330 @@
|
||||
## 📦 **Naam: "DevDen"** (of alternatieven: "CodeBurrow", "TerminalNest", "AskDen")
|
||||
## 🦊 DevDen
|
||||
|
||||
**Tagline:** *"Your cozy AI-powered development terminal"*
|
||||
**Your AI assistant, powered by your knowledge**
|
||||
|
||||
---
|
||||
|
||||
## 🎯 **Concept Beschrijving**
|
||||
## What is DevDen?
|
||||
|
||||
DevDen is een self-hosted, terminal-stijl webinterface die developers een unified interface geeft om met verschillende AI providers te praten, gekoppeld aan hun eigen codebases en kennisbanken. Het combineert de vertrouwdheid van een terminal met de kracht van moderne AI, waarbij je volledige controle houdt over je data en AI provider keuzes.
|
||||
DevDen is a self-hosted AI chat platform that lets people ask questions and get answers from AI providers like Claude, OpenAI, Gemini, and others. The key difference? As an administrator, you control what knowledge the AI has access to by connecting your own documentation, manuals, wikis, or any other information sources.
|
||||
|
||||
Think of it as giving your users a smart assistant that knows exactly what's in your company documentation, support guides, or knowledge bases - without them having to search through files themselves.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ **Architectuur Overzicht**
|
||||
## Who is it for?
|
||||
|
||||
### **Core Components**
|
||||
**Companies** - Employees can ask questions about HR policies, procedures, or internal processes
|
||||
|
||||
1. **Frontend: Terminal Interface**
|
||||
- Clean, terminal-achtige UI (denk: groen-op-zwart, of customizable themes)
|
||||
- Command-line style interacties met autocomplete
|
||||
- Split-pane view: terminal + context viewer
|
||||
- Syntax highlighting voor code responses
|
||||
- Session management (bewaar conversatie geschiedenis)
|
||||
**Support Teams** - Customers get instant answers based on your product documentation
|
||||
|
||||
2. **Backend: API Gateway**
|
||||
- FastAPI (past bij je Python stack)
|
||||
- Provider abstraction layer
|
||||
- Request routing en rate limiting
|
||||
- WebSocket support voor streaming responses
|
||||
**Organizations** - Members can query policies, regulations, or guidelines
|
||||
|
||||
3. **Knowledge Base Engine**
|
||||
- Git repository indexing
|
||||
- Docker volume mounting
|
||||
- Vector embeddings voor semantic search
|
||||
- File watching voor auto-updates
|
||||
- Support voor multiple knowledge bases per user/project
|
||||
|
||||
4. **Provider Manager**
|
||||
- Unified interface voor alle AI providers
|
||||
- API key management per provider
|
||||
- Fallback logic (als één provider down is)
|
||||
- Cost tracking per provider/model
|
||||
- Model capability mapping
|
||||
**Communities** - Users can get answers about rules, resources, or FAQs
|
||||
|
||||
---
|
||||
|
||||
## 🔐 **Authenticatie: Microsoft Entra ID**
|
||||
## How does it work?
|
||||
|
||||
**Setup:**
|
||||
- OAuth 2.0 flow met Entra ID
|
||||
- Role-based access control (RBAC)
|
||||
- Multi-tenant support (voor meerdere orgs)
|
||||
- API token generation voor programmatic access
|
||||
**For Users (the simple part):**
|
||||
|
||||
**User Management:**
|
||||
- Per-user provider API keys (encrypted)
|
||||
- Per-user knowledge base access
|
||||
- Usage quotas en billing (optioneel)
|
||||
1. User visits DevDen and logs in with their Microsoft account
|
||||
2. User types a question in the chat interface
|
||||
3. AI reads the question, searches your knowledge base, and provides an answer
|
||||
4. User can see where the answer came from (which document or page)
|
||||
5. Conversation is saved so they can refer back to it later
|
||||
|
||||
That's it. No training needed, no complicated setup for end users.
|
||||
|
||||
**For You (the admin):**
|
||||
|
||||
1. You set up DevDen with Docker Compose on your server
|
||||
2. You add your API keys for the AI providers you want to use
|
||||
3. You connect your knowledge bases (either from Git repositories or local folders)
|
||||
4. You configure who can access DevDen via Microsoft Entra ID
|
||||
5. You monitor usage, costs, and manage everything through the admin dashboard
|
||||
|
||||
DevDen automatically indexes your documentation, and when users ask questions, it finds the relevant information and gives it to the AI as context.
|
||||
|
||||
---
|
||||
|
||||
## 📚 **Knowledge Base Integratie**
|
||||
## Two Different Interfaces
|
||||
|
||||
### **Opties:**
|
||||
**Clean Chat Interface (for regular users)**
|
||||
|
||||
1. **Git Integration**
|
||||
- Clone repos via URL + credentials
|
||||
- Auto-sync op interval of webhook
|
||||
- Branch/tag selection
|
||||
- .devdenignore voor exclusions
|
||||
A simple, friendly chat interface that looks like WhatsApp or any modern messaging app. Users just type questions and get answers. No technical knowledge required. Perfect for employees, customers, or community members.
|
||||
|
||||
2. **Volume Mounting**
|
||||
- Docker volumes voor local codebases
|
||||
- Read-only of read-write access
|
||||
- Hot-reload bij file changes
|
||||
**Terminal-Style Dashboard (for you, the admin)**
|
||||
|
||||
3. **Future Options:**
|
||||
- Confluence/Notion integration
|
||||
- S3/Azure Blob storage
|
||||
- Database schema introspection
|
||||
- API documentation scraping
|
||||
|
||||
**Indexing:**
|
||||
- Chunk documents voor context
|
||||
- Vector embeddings (gebruik Ollama local, of OpenAI)
|
||||
- Metadata extraction (file types, authors, dates)
|
||||
- Full-text search + semantic search
|
||||
A more technical, terminal-inspired interface where you manage everything: knowledge bases, AI providers, user access, usage statistics, and costs. This gives you full control over the platform.
|
||||
|
||||
---
|
||||
|
||||
## 🎨 **UI/UX Features**
|
||||
## Knowledge Base Options
|
||||
|
||||
### **Terminal Commands**
|
||||
```
|
||||
/model claude-sonnet-4 # Switch AI model
|
||||
/kb add my-project # Add knowledge base
|
||||
/kb search "authentication" # Search in KB
|
||||
/provider openai # Switch provider
|
||||
/session save my-analysis # Save session
|
||||
/export markdown # Export conversation
|
||||
/clear # Clear terminal
|
||||
/help # Show commands
|
||||
```
|
||||
You can connect your documentation in two ways:
|
||||
|
||||
### **Context Awareness**
|
||||
- Toon welke knowledge base chunks gebruikt worden
|
||||
- Highlight relevante files in sidebar
|
||||
- Token usage indicator
|
||||
- Model capabilities badge
|
||||
**Git Repositories**
|
||||
|
||||
Connect any Git repository (GitHub, GitLab, etc.) and DevDen will automatically clone and index it. Whenever you update the documentation in Git, DevDen can automatically sync and re-index the new content.
|
||||
|
||||
Great for: Documentation that multiple people maintain, version-controlled content, automated workflows
|
||||
|
||||
**Docker Volumes**
|
||||
|
||||
Mount a local folder with your documents and DevDen will index everything inside. Any changes to files are detected and the index updates automatically.
|
||||
|
||||
Great for: Local files, quick setup, direct file system access
|
||||
|
||||
DevDen can index markdown files, text files, PDFs, Word documents, and more.
|
||||
|
||||
---
|
||||
|
||||
## 🐳 **Docker Setup**
|
||||
## AI Provider Flexibility
|
||||
|
||||
### **docker-compose.yml Structure**
|
||||
You're not locked into a single AI provider. DevDen supports:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
devden-web: # Frontend + Backend
|
||||
devden-db: # PostgreSQL voor sessions/users
|
||||
devden-redis: # Caching + session storage
|
||||
devden-vector: # Vector DB (qdrant/chroma)
|
||||
ollama: # Optional local AI
|
||||
```
|
||||
- **Claude** (Anthropic) - Sonnet, Opus, Haiku models
|
||||
- **OpenAI** - GPT-4 and other models
|
||||
- **OpenRouter** - Access to many different AI models through one API
|
||||
- **Google Gemini** - Google's AI models
|
||||
- **Ollama** - Run AI models locally on your own hardware (no API costs)
|
||||
|
||||
**Volumes:**
|
||||
- `/data/knowledge-bases` - Mounted KB directories
|
||||
- `/data/git-repos` - Cloned repositories
|
||||
- `/data/configs` - User configs + API keys
|
||||
You can set a default provider, allow users to choose, or even set up automatic fallbacks if one provider is having issues.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 **Provider Configuration**
|
||||
## Authentication & Security
|
||||
|
||||
### **providers.yaml**
|
||||
```yaml
|
||||
providers:
|
||||
claude:
|
||||
models: [sonnet-4, opus-4, haiku-4]
|
||||
capabilities: [code, vision, long-context]
|
||||
|
||||
openai:
|
||||
models: [gpt-4, gpt-4-turbo]
|
||||
capabilities: [code, vision]
|
||||
|
||||
openrouter:
|
||||
proxy: true
|
||||
models: [all-available]
|
||||
|
||||
gemini:
|
||||
models: [gemini-pro, gemini-ultra]
|
||||
|
||||
ollama:
|
||||
local: true
|
||||
endpoint: http://ollama:11434
|
||||
```
|
||||
**Microsoft Entra ID Integration**
|
||||
|
||||
Users log in with their existing Microsoft/Azure AD accounts. No need to create separate accounts or remember new passwords.
|
||||
|
||||
You control:
|
||||
- Which users or groups can access DevDen
|
||||
- How many questions users can ask per day
|
||||
- Which knowledge bases different users can access
|
||||
|
||||
**Data Privacy**
|
||||
|
||||
Everything runs on your server. Conversations are stored in your database, not with the AI providers. Knowledge bases never leave your infrastructure. Only the actual questions and selected context are sent to the AI provider's API.
|
||||
|
||||
---
|
||||
|
||||
## 🌟 **Killer Features**
|
||||
## What Makes DevDen Special?
|
||||
|
||||
1. **Smart Context Selection**
|
||||
- AI suggests relevant files from KB
|
||||
- Automatic dependency detection
|
||||
- "What's changed" git diff integration
|
||||
**Context-Aware Answers**
|
||||
|
||||
2. **Collaborative Sessions**
|
||||
- Share session URLs
|
||||
- Real-time collaboration (optional)
|
||||
- Export sessions als gists
|
||||
Instead of generic AI responses, users get answers specific to your organization because the AI has access to your documentation.
|
||||
|
||||
3. **Cost Optimization**
|
||||
- Use cheap models voor simple queries
|
||||
- Automatic switching based op task complexity
|
||||
- Monthly budget alerts
|
||||
**Source Transparency**
|
||||
|
||||
4. **Privacy First**
|
||||
- All data self-hosted
|
||||
- Optional E2E encryption voor KB
|
||||
- Audit logs voor compliance
|
||||
- No telemetry naar providers (alleen noodzakelijke API calls)
|
||||
Every answer shows which documents or pages were used, so users can verify information or read more if they want.
|
||||
|
||||
**Full Control**
|
||||
|
||||
You decide which AI providers to use, what documentation to include, who has access, and you can see all usage statistics and costs.
|
||||
|
||||
**Self-Hosted**
|
||||
|
||||
Your data stays on your server. No external service has access to your knowledge base or user conversations.
|
||||
|
||||
**Cost Effective**
|
||||
|
||||
Only pay for what you use. AI providers charge per token/request, so you only pay when users ask questions. Running DevDen itself just requires a server (or even just a computer with Docker).
|
||||
|
||||
---
|
||||
|
||||
## 📊 **Tech Stack**
|
||||
## Example Use Cases
|
||||
|
||||
**Frontend:**
|
||||
- Svelte/Alpine.js (lightweight)
|
||||
- Xterm.js voor terminal emulation
|
||||
- TailwindCSS voor styling
|
||||
**HR Support Bot**
|
||||
|
||||
**Backend:**
|
||||
- FastAPI (Python)
|
||||
- PostgreSQL (users, sessions, configs)
|
||||
- Redis (caching, rate limiting)
|
||||
- Qdrant/ChromaDB (vector search)
|
||||
Knowledge Base: Employee handbook, vacation policies, benefits documentation
|
||||
|
||||
**Infrastructure:**
|
||||
- Docker + Docker Compose
|
||||
- Nginx reverse proxy
|
||||
- Traefik voor routing (optioneel)
|
||||
Questions users ask:
|
||||
- "How many vacation days do I have?"
|
||||
- "What's the parental leave policy?"
|
||||
- "How do I submit an expense report?"
|
||||
|
||||
**IT Helpdesk**
|
||||
|
||||
Knowledge Base: IT documentation, troubleshooting guides, software manuals
|
||||
|
||||
Questions users ask:
|
||||
- "How do I reset my password?"
|
||||
- "My laptop won't connect to WiFi, what should I do?"
|
||||
- "Which VPN should I use for remote work?"
|
||||
|
||||
**Customer Support**
|
||||
|
||||
Knowledge Base: Product documentation, FAQs, return policies
|
||||
|
||||
Questions customers ask:
|
||||
- "How do I set up my new device?"
|
||||
- "What's your return policy?"
|
||||
- "Is this product compatible with X?"
|
||||
|
||||
**Internal Wiki Alternative**
|
||||
|
||||
Instead of employees searching through wiki pages, they just ask DevDen and get instant answers with sources.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 **MVP Feature Set**
|
||||
## Technical Overview
|
||||
|
||||
**Phase 1: Core**
|
||||
- [ ] Terminal UI met basic commands
|
||||
- [ ] Claude + OpenAI provider support
|
||||
- [ ] Volume-based KB (single directory)
|
||||
- [ ] Basic auth (admin/password)
|
||||
- [ ] Session persistence
|
||||
**What You Need**
|
||||
|
||||
**Phase 2: Enhanced**
|
||||
- [ ] Microsoft Entra ID integration
|
||||
- [ ] Git repository integration
|
||||
- [ ] Vector search in KB
|
||||
- [ ] OpenRouter + Gemini providers
|
||||
- [ ] Cost tracking
|
||||
- A server (VPS, cloud instance, or even a local machine)
|
||||
- Docker and Docker Compose installed
|
||||
- API keys for the AI providers you want to use
|
||||
- Microsoft Entra ID tenant for authentication
|
||||
- Your documentation/knowledge base ready
|
||||
|
||||
**Phase 3: Advanced**
|
||||
- [ ] Ollama local models
|
||||
- [ ] Multi-KB support
|
||||
- [ ] Collaborative features
|
||||
- [ ] Advanced prompt templates
|
||||
- [ ] API voor external integrations
|
||||
**What DevDen Includes**
|
||||
|
||||
- Web-based chat interface (frontend)
|
||||
- API backend (FastAPI with Python)
|
||||
- PostgreSQL database (stores users, conversations, settings)
|
||||
- Redis (caching and session management)
|
||||
- Vector database (Qdrant or ChromaDB - for semantic search in your docs)
|
||||
- Optional: Ollama container (for local AI models)
|
||||
|
||||
Everything runs in Docker containers, making it easy to deploy and manage.
|
||||
|
||||
---
|
||||
|
||||
## 🎨 **Visual Identity**
|
||||
## Admin Features
|
||||
|
||||
Gezien je voorliefde voor cozy aesthetics:
|
||||
**Knowledge Base Management**
|
||||
|
||||
**Color Palette:**
|
||||
- Primary: Warm terminal green (#7CFC00)
|
||||
- Background: Deep charcoal (#1e1e1e)
|
||||
- Accent: Cappuccino brown (#8B7355)
|
||||
- Highlights: Soft amber (#FFB347)
|
||||
- Add/remove knowledge bases
|
||||
- Enable/disable specific knowledge bases
|
||||
- See sync status and last update time
|
||||
- View how many documents are indexed
|
||||
- Configure auto-sync intervals
|
||||
|
||||
**Mascot Concept:**
|
||||
Een cozy fox in een burrow met een terminal, omringd door code snippets en boeken (knowledge bases). Warm, friendly, tech-savvy.
|
||||
**User Management**
|
||||
|
||||
- See all active users
|
||||
- View per-user usage statistics
|
||||
- Set usage limits (questions per day/month)
|
||||
- Control access to specific knowledge bases
|
||||
- Export user data (GDPR compliance)
|
||||
|
||||
**Provider Settings**
|
||||
|
||||
- Configure multiple AI providers
|
||||
- Set default provider and model
|
||||
- Add/rotate API keys
|
||||
- Monitor API usage and costs
|
||||
- Set up fallback providers
|
||||
|
||||
**Analytics Dashboard**
|
||||
|
||||
- Total questions asked (today/week/month)
|
||||
- Most active users
|
||||
- Most asked topics
|
||||
- Response times
|
||||
- Cost tracking per provider
|
||||
- Most used knowledge bases
|
||||
|
||||
**System Monitoring**
|
||||
|
||||
- Health status of all services
|
||||
- Database connection status
|
||||
- Vector database status
|
||||
- API provider connectivity
|
||||
- Error logs and alerts
|
||||
|
||||
---
|
||||
|
||||
## Development Phases
|
||||
|
||||
**Phase 1: Basic Chat**
|
||||
|
||||
Get a working chat interface where users can ask questions and get answers from Claude or OpenAI.
|
||||
|
||||
**Phase 2: Knowledge Base**
|
||||
|
||||
Add document indexing and semantic search so the AI can use your documentation.
|
||||
|
||||
**Phase 3: Authentication**
|
||||
|
||||
Integrate Microsoft Entra ID so users can securely log in.
|
||||
|
||||
**Phase 4: Admin Dashboard**
|
||||
|
||||
Build the admin interface so you can manage everything.
|
||||
|
||||
**Phase 5: Multi-Provider**
|
||||
|
||||
Add support for multiple AI providers and switching between them.
|
||||
|
||||
**Phase 6: Git Integration**
|
||||
|
||||
Add automatic syncing from Git repositories.
|
||||
|
||||
**Phase 7: Polish**
|
||||
|
||||
Add all the nice-to-have features like conversation export, mobile responsiveness, advanced search, etc.
|
||||
|
||||
---
|
||||
|
||||
## Why "DevDen"?
|
||||
|
||||
The name combines "Dev" (short for development or device - it's a tool for building better workflows) with "Den" (a cozy, safe space - like a fox's burrow).
|
||||
|
||||
The fox mascot represents intelligence, adaptability, and resourcefulness - just like the platform itself. The "den" is where knowledge lives, protected and organized, ready to be accessed when needed.
|
||||
|
||||
Plus, it fits nicely with your other project names like Burrow and Bean Den, keeping that warm, cozy aesthetic.
|
||||
|
||||
---
|
||||
|
||||
## Cost Estimation
|
||||
|
||||
For a small organization (50 users, 1000 questions per month):
|
||||
|
||||
**AI Costs:**
|
||||
- Claude Sonnet: around 10-15 dollars per month
|
||||
- OpenAI GPT-4: around 25-35 dollars per month
|
||||
- Ollama (local): free, just electricity
|
||||
|
||||
**Hosting:**
|
||||
- Small VPS: 20-40 dollars per month
|
||||
- Or self-host: just electricity
|
||||
|
||||
**Total: Roughly 30-75 dollars per month depending on usage and provider choice**
|
||||
|
||||
Much cheaper than enterprise chat platforms or support desk solutions, and you have full control.
|
||||
|
||||
---
|
||||
|
||||
## Privacy & Compliance
|
||||
|
||||
**Data Residency**
|
||||
|
||||
All data stays on your server. Conversations, user data, and knowledge bases are stored in your database, not on external servers.
|
||||
|
||||
**GDPR Compliant**
|
||||
|
||||
Users can request to see their data or have it deleted. Admin tools make this easy to manage.
|
||||
|
||||
**Audit Logs**
|
||||
|
||||
All admin actions are logged. You can see who did what and when.
|
||||
|
||||
**Encryption**
|
||||
|
||||
API keys are encrypted in the database. Conversations can be encrypted at rest. HTTPS for all traffic.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
If you want to build DevDen, the next step would be to create:
|
||||
|
||||
1. A detailed technical architecture document
|
||||
2. Database schema design
|
||||
3. API endpoint specifications
|
||||
4. UI/UX wireframes and mockups
|
||||
5. A development roadmap with specific tasks
|
||||
|
||||
Reference in New Issue
Block a user