Latte
f318a6bdaf
Move and restructure BaseLayout to src/layouts with SEO meta tags Add 404 page with accessibility improvements Restructure pages to src/pages
17 lines
301 B
Plaintext
17 lines
301 B
Plaintext
# Hidden Den Cafe - Robots.txt
|
|
User-agent: *
|
|
Allow: /
|
|
|
|
# Sitemap
|
|
Sitemap: https://hiddenden.cafe/sitemap-index.xml
|
|
|
|
# Crawl-delay for polite crawlers
|
|
Crawl-delay: 1
|
|
|
|
# Block aggressive bots (optional - uncomment if needed)
|
|
# User-agent: AhrefsBot
|
|
# Disallow: /
|
|
|
|
# User-agent: SemrushBot
|
|
# Disallow: /
|