loyal_companion/docs/database.md

Database Schema Guide

This document describes the database schema used by Daemon Boyfriend.

Table of Contents

• Overview
• Core Tables
• Living AI Tables
• Indexes
• Relationships
• Schema Diagram

---

Overview

Database Support

• Primary: PostgreSQL with async support (asyncpg)
• Fallback: In-memory (no persistence)
• Testing: SQLite (via aiosqlite)

Key Features

• PortableJSON: Custom type for PostgreSQL JSONB / SQLite JSON compatibility
• UTC Timestamps: All timestamps are stored as timezone-aware UTC
• Soft Deletes: Many tables use is_active flag instead of hard deletes
• Cascade Deletes: Foreign keys cascade on user/conversation deletion

Connection Configuration

# PostgreSQL connection string
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/loyal_companion

# Optional settings
DATABASE_ECHO=false        # Log SQL queries
DATABASE_POOL_SIZE=5       # Connection pool size
DATABASE_MAX_OVERFLOW=10   # Max connections beyond pool

---

Core Tables

users

Stores Discord user information.

Column	Type	Description
id	BIGSERIAL	Primary key
discord_id	BIGINT	Discord user ID (unique)
discord_username	VARCHAR(255)	Discord username
discord_display_name	VARCHAR(255)	Discord display name
custom_name	VARCHAR(255)	Custom name set by user/admin
first_seen_at	TIMESTAMPTZ	When user was first seen
last_seen_at	TIMESTAMPTZ	When user was last seen
is_active	BOOLEAN	Whether user is active
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Usage:

user = await user_service.get_or_create_user(
    discord_id=123456789,
    username="john_doe",
    display_name="John"
)
print(user.display_name)  # Returns custom_name or discord_display_name

---

user_preferences

Key-value storage for user preferences.

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users
preference_key	VARCHAR(100)	Preference name
preference_value	TEXT	Preference value
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Unique Constraint: (user_id, preference_key)

---

user_facts

Facts the bot knows about users.

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users
fact_type	VARCHAR(50)	Type (hobby, work, family, etc.)
fact_content	TEXT	The fact itself
confidence	FLOAT	0.0-1.0 confidence level
source	VARCHAR(50)	How learned (conversation, auto_extraction, manual)
is_active	BOOLEAN	Whether fact is active
learned_at	TIMESTAMPTZ	When fact was learned
last_referenced_at	TIMESTAMPTZ	When fact was last used
category	VARCHAR(50)	Category (same as fact_type)
importance	FLOAT	0.0-1.0 importance level
temporal_relevance	VARCHAR(20)	past/present/future/timeless
expiry_date	TIMESTAMPTZ	When fact expires (optional)
extracted_from_message_id	BIGINT	Source Discord message ID
extraction_context	TEXT	Context of extraction
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Fact Types:

• hobby - Activities, interests
• work - Job, career
• family - Family members
• preference - Likes, dislikes
• location - Places
• event - Life events
• relationship - Personal relationships
• general - Other facts

---

guilds

Discord server (guild) information.

Column	Type	Description
id	BIGSERIAL	Primary key
discord_id	BIGINT	Discord guild ID (unique)
name	VARCHAR(255)	Guild name
joined_at	TIMESTAMPTZ	When bot joined
is_active	BOOLEAN	Whether bot is active in guild
settings	JSONB	Guild-specific settings
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

guild_members

User membership in guilds.

Column	Type	Description
id	BIGSERIAL	Primary key
guild_id	BIGINT	Foreign key to guilds
user_id	BIGINT	Foreign key to users
guild_nickname	VARCHAR(255)	Nickname in this guild
roles	TEXT[]	Array of role names
joined_guild_at	TIMESTAMPTZ	When user joined guild
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Unique Constraint: (guild_id, user_id)

---

conversations

Conversation sessions between users and the bot.

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users
guild_id	BIGINT	Guild ID (nullable for DMs)
channel_id	BIGINT	Discord channel ID
started_at	TIMESTAMPTZ	When conversation started
last_message_at	TIMESTAMPTZ	Last message timestamp
message_count	INTEGER	Number of messages
is_active	BOOLEAN	Whether conversation is active
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Conversation Timeout: A new conversation starts after 60 minutes of inactivity.

---

messages

Individual messages in conversations.

Column	Type	Description
id	BIGSERIAL	Primary key
conversation_id	BIGINT	Foreign key to conversations
user_id	BIGINT	Foreign key to users
discord_message_id	BIGINT	Discord message ID
role	VARCHAR(20)	user/assistant/system
content	TEXT	Message content
has_images	BOOLEAN	Whether message has images
image_urls	JSONB	Array of image URLs
token_count	INTEGER	Token count (estimated)
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

Living AI Tables

bot_states

Per-guild bot state (mood, statistics).

Column	Type	Description
id	BIGSERIAL	Primary key
guild_id	BIGINT	Guild ID (unique, NULL = global)
mood_valence	FLOAT	-1.0 to 1.0 (sad to happy)
mood_arousal	FLOAT	-1.0 to 1.0 (calm to excited)
mood_updated_at	TIMESTAMPTZ	When mood was last updated
total_messages_sent	INTEGER	Lifetime message count
total_facts_learned	INTEGER	Facts extracted count
total_users_known	INTEGER	Unique users count
first_activated_at	TIMESTAMPTZ	Bot "birth date"
preferences	JSONB	Bot preferences
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

bot_opinions

Bot's opinions on topics.

Column	Type	Description
id	BIGSERIAL	Primary key
guild_id	BIGINT	Guild ID (NULL = global)
topic	VARCHAR(255)	Topic name (lowercase)
sentiment	FLOAT	-1.0 to 1.0
interest_level	FLOAT	0.0 to 1.0
discussion_count	INTEGER	Times discussed
reasoning	TEXT	AI explanation (optional)
formed_at	TIMESTAMPTZ	When opinion formed
last_reinforced_at	TIMESTAMPTZ	When last discussed
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Unique Constraint: (guild_id, topic)

---

user_relationships

Relationship depth with each user.

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users
guild_id	BIGINT	Guild ID (NULL = global)
relationship_score	FLOAT	0-100 score
total_interactions	INTEGER	Total interaction count
positive_interactions	INTEGER	Positive interaction count
negative_interactions	INTEGER	Negative interaction count
avg_message_length	FLOAT	Average message length
conversation_depth_avg	FLOAT	Average conversation turns
shared_references	JSONB	Inside jokes, nicknames, etc.
first_interaction_at	TIMESTAMPTZ	First interaction
last_interaction_at	TIMESTAMPTZ	Last interaction
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

Unique Constraint: (user_id, guild_id)

Relationship Score Levels:

• 0-20: Stranger
• 21-40: Acquaintance
• 41-60: Friend
• 61-80: Good Friend
• 81-100: Close Friend

---

user_communication_styles

Learned communication preferences per user.

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users (unique)
preferred_length	VARCHAR(20)	short/medium/long
preferred_formality	FLOAT	0.0 (casual) to 1.0 (formal)
emoji_affinity	FLOAT	0.0 (none) to 1.0 (lots)
humor_affinity	FLOAT	0.0 (serious) to 1.0 (playful)
detail_preference	FLOAT	0.0 (concise) to 1.0 (detailed)
engagement_signals	JSONB	Engagement signal data
samples_collected	INTEGER	Number of samples analyzed
confidence	FLOAT	0.0-1.0 confidence level
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

scheduled_events

Proactive events (birthdays, follow-ups).

Column	Type	Description
id	BIGSERIAL	Primary key
user_id	BIGINT	Foreign key to users
guild_id	BIGINT	Target guild ID
channel_id	BIGINT	Target channel ID
event_type	VARCHAR(50)	birthday/follow_up/reminder
trigger_at	TIMESTAMPTZ	When to trigger
title	VARCHAR(255)	Event title
context	JSONB	Additional context
is_recurring	BOOLEAN	Whether event recurs
recurrence_rule	VARCHAR(100)	yearly/monthly/weekly/etc.
status	VARCHAR(20)	pending/triggered/cancelled
triggered_at	TIMESTAMPTZ	When event was triggered
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

fact_associations

Cross-user fact associations.

Column	Type	Description
id	BIGSERIAL	Primary key
fact_id_1	BIGINT	Foreign key to user_facts
fact_id_2	BIGINT	Foreign key to user_facts
association_type	VARCHAR(50)	shared_interest/same_location/etc.
strength	FLOAT	0.0-1.0 association strength
discovered_at	TIMESTAMPTZ	When discovered
created_at	TIMESTAMPTZ	Record creation time

Unique Constraint: (fact_id_1, fact_id_2)

---

mood_history

Historical mood changes.

Column	Type	Description
id	BIGSERIAL	Primary key
guild_id	BIGINT	Guild ID
valence	FLOAT	Valence at this point
arousal	FLOAT	Arousal at this point
trigger_type	VARCHAR(50)	conversation/time_decay/event
trigger_user_id	BIGINT	User who triggered (optional)
trigger_description	TEXT	Description of trigger
recorded_at	TIMESTAMPTZ	When recorded
created_at	TIMESTAMPTZ	Record creation time
updated_at	TIMESTAMPTZ	Record update time

---

Indexes

All indexes are created with IF NOT EXISTS for idempotency.

Core Table Indexes

Table	Index	Columns
users	ix_users_discord_id	discord_id
users	ix_users_last_seen_at	last_seen_at
user_preferences	ix_user_preferences_user_id	user_id
user_facts	ix_user_facts_user_id	user_id
user_facts	ix_user_facts_fact_type	fact_type
user_facts	ix_user_facts_is_active	is_active
guilds	ix_guilds_discord_id	discord_id
guild_members	ix_guild_members_guild_id	guild_id
guild_members	ix_guild_members_user_id	user_id
conversations	ix_conversations_user_id	user_id
conversations	ix_conversations_channel_id	channel_id
conversations	ix_conversations_last_message_at	last_message_at
conversations	ix_conversations_is_active	is_active
messages	ix_messages_conversation_id	conversation_id
messages	ix_messages_user_id	user_id
messages	ix_messages_created_at	created_at

Living AI Table Indexes

Table	Index	Columns
bot_states	ix_bot_states_guild_id	guild_id
bot_opinions	ix_bot_opinions_guild_id	guild_id
bot_opinions	ix_bot_opinions_topic	topic
user_relationships	ix_user_relationships_user_id	user_id
user_relationships	ix_user_relationships_guild_id	guild_id
user_communication_styles	ix_user_communication_styles_user_id	user_id
scheduled_events	ix_scheduled_events_user_id	user_id
scheduled_events	ix_scheduled_events_trigger_at	trigger_at
scheduled_events	ix_scheduled_events_status	status
fact_associations	ix_fact_associations_fact_id_1	fact_id_1
fact_associations	ix_fact_associations_fact_id_2	fact_id_2
mood_history	ix_mood_history_guild_id	guild_id
mood_history	ix_mood_history_recorded_at	recorded_at

---

Relationships

Entity Relationship Diagram

┌──────────────────┐         ┌──────────────────┐
│      users       │         │      guilds      │
├──────────────────┤         ├──────────────────┤
│ id (PK)          │◄────┐   │ id (PK)          │◄────┐
│ discord_id       │     │   │ discord_id       │     │
│ discord_username │     │   │ name             │     │
│ custom_name      │     │   │ settings         │     │
└──────────────────┘     │   └──────────────────┘     │
         │               │            │               │
         │               │            │               │
         ▼               │            ▼               │
┌──────────────────┐     │   ┌──────────────────┐     │
│  user_facts      │     │   │  guild_members   │     │
├──────────────────┤     │   ├──────────────────┤     │
│ id (PK)          │     │   │ id (PK)          │     │
│ user_id (FK) ────┼─────┘   │ guild_id (FK) ───┼─────┘
│ fact_type        │         │ user_id (FK) ────┼─────┐
│ fact_content     │         │ guild_nickname   │     │
└──────────────────┘         └──────────────────┘     │
         ▲                                            │
         │                                            │
         │                                            │
┌────────┴─────────┐         ┌──────────────────┐     │
│fact_associations │         │  conversations   │     │
├──────────────────┤         ├──────────────────┤     │
│ id (PK)          │         │ id (PK)          │◄────┤
│ fact_id_1 (FK)   │         │ user_id (FK) ────┼─────┘
│ fact_id_2 (FK)   │         │ guild_id         │
│ association_type │         │ channel_id       │
└──────────────────┘         └──────────────────┘
                                     │
                                     │
                                     ▼
                             ┌──────────────────┐
                             │    messages      │
                             ├──────────────────┤
                             │ id (PK)          │
                             │ conversation_id  │
                             │ user_id (FK)     │
                             │ role             │
                             │ content          │
                             └──────────────────┘

Living AI Relationships

┌──────────────────┐
│      users       │
├──────────────────┤
│ id (PK)          │◄─────────────────────────────────────┐
└──────────────────┘                                      │
         │                                                │
         │                                                │
    ┌────┴────────┬───────────────┬───────────────┐      │
    │             │               │               │      │
    ▼             ▼               ▼               ▼      │
┌────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐  │
│user_   │ │user_comm   │ │scheduled   │ │mood_       │  │
│relation│ │styles      │ │events      │ │history     │  │
│ships   │ │            │ │            │ │            │  │
├────────┤ ├────────────┤ ├────────────┤ ├────────────┤  │
│user_id │ │user_id     │ │user_id     │ │trigger_    │  │
│guild_id│ │preferred_  │ │guild_id    │ │user_id ────┼──┘
│score   │ │length      │ │event_type  │ │valence     │
│        │ │emoji       │ │trigger_at  │ │arousal     │
└────────┘ └────────────┘ └────────────┘ └────────────┘


┌──────────────────┐         ┌──────────────────┐
│   bot_states     │         │   bot_opinions   │
├──────────────────┤         ├──────────────────┤
│ guild_id (unique)│         │ guild_id         │
│ mood_valence     │         │ topic            │
│ mood_arousal     │         │ sentiment        │
│ total_messages   │         │ interest_level   │
│ total_facts      │         │ discussion_count │
└──────────────────┘         └──────────────────┘

---

Schema Initialization

From Code

from loyal_companion.services.database import db

# Initialize connection
await db.init()

# Create tables from schema.sql
await db.create_tables()

From SQL

# Create database
createdb loyal_companion

# Run schema
psql -U postgres -d loyal_companion -f schema.sql

Docker

The docker-compose.yml handles database setup automatically:

docker-compose up -d

---

Migrations

The project uses Alembic for migrations, but currently relies on schema.sql for initial setup.

# Generate migration
alembic revision --autogenerate -m "description"

# Run migrations
alembic upgrade head

# Rollback
alembic downgrade -1

---

Type Compatibility

PortableJSON

For JSONB (PostgreSQL) and JSON (SQLite) compatibility:

from loyal_companion.models.base import PortableJSON

class MyModel(Base):
    settings = Column(PortableJSON, default={})

ensure_utc()

Handles timezone-naive datetimes from SQLite:

from loyal_companion.models.base import ensure_utc

# Safe for both PostgreSQL (already UTC) and SQLite (naive)
utc_time = ensure_utc(model.created_at)