Title Intelligence
This skill orchestrates data collection from multiple platforms to gather metrics, metadata, and fan engagement data for titles.
When to Use This Skill
- •Collecting platform metrics for a new title
- •Updating metrics for existing titles
- •Batch collection for titles missing data
- •Gathering fan engagement data (Reddit, AO3)
- •Debugging data collection issues
Supported Platforms
Official Platforms (Korean)
| Platform | Domain | Data Collected |
|---|---|---|
| Naver Webtoon | comic.naver.com | Views, subscribers, rating, episodes, synopsis |
| Naver Series | series.naver.com | Views, subscribers, rating, episodes |
| Kakao Page | page.kakao.com | Views, likes, comments, episodes |
| Kakao Webtoon | webtoon.kakao.com | Views, likes, rating |
| Ridibooks | ridibooks.com | Rating, reviews |
| Bomtoon | bomtoon.com | Views, rating |
Official Platforms (English)
| Platform | Domain | Data Collected |
|---|---|---|
| Manta | manta.net | Views, likes, rating |
| Webtoons | webtoons.com | Views, subscribers, rating |
Fan Engagement Sources
| Source | Domain | Data Collected |
|---|---|---|
| reddit.com | Subreddit mentions, posts, comments | |
| AO3 | archiveofourown.org | Fanfic count, kudos, bookmarks |
| Comick | comick.live | Unofficial aggregator stats |
Commands
/title-intelligence --title="Title Name" # Collect by title name /title-intelligence --url="https://..." # Collect from specific URL /title-intelligence --platform=kakao --batch=20 # Batch by platform /title-intelligence --missing-views # Titles without view data /title-intelligence --fan-engagement="Title Name" # Reddit + AO3 only
Data Collection Workflows
Single Title Collection (by URL)
The most accurate method - provide direct platform URL:
# Via edge function
curl -X POST "$SUPABASE_URL/functions/v1/title-intelligence" \
-H "Authorization: Bearer $SUPABASE_ANON_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": [{
"platform": "naver_webtoon",
"platformId": "783052",
"originalUrl": "https://comic.naver.com/webtoon/list?titleId=783052"
}],
"collectedBy": "admin@example.com",
"contentType": "webtoon"
}'
Single Title Collection (by Name)
Search-based collection (less reliable):
curl -X POST "$SUPABASE_URL/functions/v1/title-intelligence" \
-H "Authorization: Bearer $SUPABASE_ANON_KEY" \
-H "Content-Type: application/json" \
-d '{
"titleNameInput": "재벌집 막내아들",
"titleNameEn": "Reborn Rich",
"sources": ["naver", "kakao", "reddit"],
"collectedBy": "admin@example.com",
"contentType": "webnovel"
}'
Batch Collection Script
For collecting multiple titles:
# Use the batch collection script node scripts/batch-collect-kakao-metrics.js
Fan Engagement Only
Collect Reddit and AO3 data without platform metrics:
curl -X POST "$SUPABASE_URL/functions/v1/title-intelligence" \
-H "Authorization: Bearer $SUPABASE_ANON_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": [],
"collectedBy": "admin@example.com",
"fanEngagement": {
"titleName": "Solo Leveling",
"sources": ["reddit", "ao3"]
}
}'
Database Schema
intelligence_titles
Central record for each title being tracked:
CREATE TABLE intelligence_titles ( id UUID PRIMARY KEY, title_ko TEXT, title_en TEXT, slug TEXT UNIQUE, content_type TEXT, -- webtoon, webnovel, light_novel, manga created_at TIMESTAMPTZ, updated_at TIMESTAMPTZ );
intelligence_sources
Platform connections for each title:
CREATE TABLE intelligence_sources ( id UUID PRIMARY KEY, intelligence_title_id UUID REFERENCES intelligence_titles, platform TEXT, -- naver_webtoon, kakao, manta, etc. platform_id TEXT, -- Platform-specific ID platform_url TEXT, -- Original URL source_category TEXT, -- official_platform, fandom_forum, fanfiction is_primary BOOLEAN, last_scraped_at TIMESTAMPTZ );
intelligence_metrics
Time-series snapshots:
CREATE TABLE intelligence_metrics ( id UUID PRIMARY KEY, source_id UUID REFERENCES intelligence_sources, scraped_at TIMESTAMPTZ, views BIGINT, subscribers BIGINT, likes BIGINT, rating_score NUMERIC, rating_count INTEGER, episode_count INTEGER, comment_count INTEGER, -- Platform-specific JSON raw_data JSONB );
Field Mapping to Titles Table
When data is collected, it can be mapped to the main titles table:
| Intelligence Field | Titles Field |
|---|---|
| views | views |
| subscribers | likes |
| rating_score | rating |
| episode_count | chapters |
| synopsis_kr | synopsis_kr |
| genre | genre |
| author | story_author |
| thumbnail | title_image |
| tags | keywords |
Mapping via Admin UI
In the Dashboard admin panel, use the "Collect Data" button in TitleEditModal to:
- •Trigger intelligence collection
- •Preview collected data
- •Map fields to title record
Mapping via SQL
-- Update title with intelligence data
UPDATE titles t
SET
views = m.views,
likes = m.subscribers,
rating = m.rating_score,
chapters = m.episode_count,
updated_at = NOW()
FROM intelligence_metrics m
JOIN intelligence_sources s ON m.source_id = s.id
JOIN intelligence_titles it ON s.intelligence_title_id = it.id
WHERE it.title_en = t.title_name_en
AND s.is_primary = true
AND m.scraped_at = (
SELECT MAX(scraped_at)
FROM intelligence_metrics
WHERE source_id = s.id
);
Rate Limiting
To avoid detection and blocking:
| Platform | Rate Limit | Notes |
|---|---|---|
| Naver | 1 req/3s | Strict bot detection |
| Kakao | 1 req/3s | Strict bot detection |
| Manta | 1 req/2s | Moderate detection |
| No limit | Uses official API | |
| AO3 | 1 req/5s | Community respect |
Error Handling
Platform Blocks
If a platform blocks requests:
- •Wait 5-10 minutes before retrying
- •Use different request headers
- •Consider using platform-specific cookies
Missing Data
Some fields may not be available:
{
"views": 1234567,
"rating": null, // Not available on this platform
"subscribers": 45000,
"error_fields": ["rating"]
}
Scraper Failures
Check edge function logs:
npx supabase functions logs title-intelligence --scroll
Verification
Check Collection Results
-- Recent collections SELECT it.title_en, s.platform, m.views, m.scraped_at FROM intelligence_metrics m JOIN intelligence_sources s ON m.source_id = s.id JOIN intelligence_titles it ON s.intelligence_title_id = it.id ORDER BY m.scraped_at DESC LIMIT 20;
Find Titles Missing Data
-- Titles without any intelligence data SELECT t.title_name_en, t.views FROM titles t LEFT JOIN intelligence_titles it ON t.title_name_en = it.title_en WHERE it.id IS NULL ORDER BY t.views DESC NULLS LAST LIMIT 50;
Console Output
Collecting title intelligence...
[1/4] Parsing request
Title: "재벌집 막내아들"
Platforms: naver, kakao
[2/4] Scraping Naver Webtoon
URL: https://comic.naver.com/...
Views: 12,345,678
Subscribers: 890,123
Rating: 9.8
Episodes: 156
[3/4] Scraping Kakao Page
URL: https://page.kakao.com/...
Views: 8,901,234
Likes: 234,567
Episodes: 156
[4/4] Storing results
Intelligence title: created
Sources: 2 created
Metrics: 2 snapshots saved
Collection complete!
Total platforms: 2
Total views: 21,246,912
Slack Notification
{
"text": "Title Intelligence Collected",
"attachments": [{
"color": "good",
"fields": [
{"title": "Title", "value": "Reborn Rich", "short": true},
{"title": "Platforms", "value": "2", "short": true},
{"title": "Total Views", "value": "21.2M", "short": true},
{"title": "Rating", "value": "9.8", "short": true}
]
}]
}
Tips
- •Use URLs when possible - More reliable than title name search
- •Collect during off-peak hours - Less likely to be rate-limited
- •Verify data accuracy - Cross-reference with platform
- •Monitor for platform changes - Scrapers may need updates
- •Batch responsibly - Don't overwhelm platforms
Related Documentation
- •Creator App: Full TitleInvestigator tool at
/tools/title-investigator - •Dashboard Admin: "Collect Data" button in TitleEditModal
- •Edge Function:
supabase/functions/title-intelligence/
Related Skills
- •
/regenerate-embeddings- Regenerate after data collection - •
/health-check- Verify intelligence system health - •
/cost-report- Track collection operations