docs/legacy/SITEMAP.md

# Complete Site Map - Crypto Monitor ULTIMATE

## 📋 Table of Contents
1. [Frontend Pages & Routes](#frontend-pages--routes)
2. [Backend API Endpoints](#backend-api-endpoints)
3. [Static Assets](#static-assets)
4. [Backend Services](#backend-services)
5. [Database Files](#database-files)
6. [Configuration Files](#configuration-files)
7. [System Monitor Components](#system-monitor-components)

---

## 🌐 Frontend Pages & Routes

### Main Application Pages

| Route | File Path | Description | Access URL |
|-------|-----------|-------------|------------|
| `/` | `static/pages/dashboard/index.html` | Main Dashboard | `http://localhost:7860/` |
| `/dashboard` | `static/pages/dashboard/index.html` | Dashboard Page | `http://localhost:7860/dashboard` |
| `/market` | `static/pages/market/index.html` | Market Data Page | `http://localhost:7860/market` |
| `/models` | `static/pages/models/index.html` | AI Models Page | `http://localhost:7860/models` |
| `/sentiment` | `static/pages/sentiment/index.html` | Sentiment Analysis | `http://localhost:7860/sentiment` |
| `/ai-analyst` | `static/pages/ai-analyst/index.html` | AI Analyst Tool | `http://localhost:7860/ai-analyst` |
| `/technical-analysis` | `static/pages/technical-analysis/index.html` | Technical Analysis | `http://localhost:7860/technical-analysis` |
| `/trading-assistant` | `static/pages/trading-assistant/index.html` | Trading Assistant | `http://localhost:7860/trading-assistant` |
| `/news` | `static/pages/news/index.html` | Crypto News | `http://localhost:7860/news` |
| `/providers` | `static/pages/providers/index.html` | Data Providers | `http://localhost:7860/providers` |
| `/system-monitor` | `static/pages/system-monitor/index.html` | **System Monitor** | `http://localhost:7860/system-monitor` |
| `/help` | `static/pages/help/index.html` | Help & Documentation | `http://localhost:7860/help` |
| `/api-explorer` | `static/pages/api-explorer/index.html` | API Explorer | `http://localhost:7860/api-explorer` |
| `/crypto-api-hub` | `static/pages/crypto-api-hub/index.html` | Crypto API Hub | `http://localhost:7860/crypto-api-hub` |
| `/diagnostics` | `static/pages/diagnostics/index.html` | System Diagnostics | `http://localhost:7860/diagnostics` |

### Static File Structure

```
static/
├── pages/
│   ├── dashboard/
│   │   ├── index.html
│   │   ├── dashboard.js
│   │   └── dashboard.css
│   ├── system-monitor/          ⭐ System Monitor
│   │   ├── index.html           → Main page HTML
│   │   ├── system-monitor.js   → JavaScript logic
│   │   ├── system-monitor.css  → Styling
│   │   └── README.md           → Documentation
│   ├── market/
│   ├── models/
│   ├── sentiment/
│   ├── ai-analyst/
│   ├── technical-analysis/
│   ├── trading-assistant/
│   ├── news/
│   ├── providers/
│   ├── help/
│   ├── api-explorer/
│   └── crypto-api-hub/
├── shared/
│   ├── layouts/
│   │   ├── sidebar.html         → Main sidebar (includes System Monitor link)
│   │   └── sidebar-modern.html  → Modern sidebar variant
│   ├── js/
│   │   ├── core/
│   │   │   ├── layout-manager.js → Loads sidebar/header
│   │   │   ├── api-client.js     → API client
│   │   │   └── models-client.js  → Models API client
│   │   └── sidebar-manager.js
│   └── css/
│       ├── design-system.css
│       ├── global.css
│       ├── components.css
│       └── layout.css
└── assets/
    └── icons/
        └── crypto-icons.js      → Crypto SVG icons
```

---

## 🔌 Backend API Endpoints

### System Monitor API Endpoints

| Endpoint | Method | File Location | Description |
|----------|--------|---------------|-------------|
| `/api/monitoring/status` | GET | `backend/routers/realtime_monitoring_api.py:40` | Get comprehensive system status |
| `/api/monitoring/ws` | WebSocket | `backend/routers/realtime_monitoring_api.py:188` | Real-time WebSocket updates |
| `/api/monitoring/sources/detailed` | GET | `backend/routers/realtime_monitoring_api.py:138` | Get detailed source information |
| `/api/monitoring/requests/recent` | GET | `backend/routers/realtime_monitoring_api.py:171` | Get recent API requests |
| `/api/monitoring/requests/log` | POST | `backend/routers/realtime_monitoring_api.py:181` | Log an API request |

### Core API Endpoints

| Endpoint | Method | File Location | Description |
|----------|--------|---------------|-------------|
| `/api/health` | GET | `hf_unified_server.py` | Health check |
| `/api/status` | GET | `hf_unified_server.py` | System status |
| `/api/models/summary` | GET | `hf_unified_server.py:1226` | Models summary with categories |
| `/api/models/status` | GET | `hf_unified_server.py:814` | Models status |
| `/api/models/list` | GET | `hf_unified_server.py:786` | List all models |
| `/api/resources` | GET | `hf_unified_server.py` | Resources statistics |
| `/api/resources/summary` | GET | `hf_unified_server.py` | Resources summary |
| `/api/resources/categories` | GET | `hf_unified_server.py` | Resources by category |

### Router Endpoints

All routers are included in `hf_unified_server.py`:

1. **Unified Service API** (`backend/routers/unified_service_api.py`)
   - `/api/service/rate`
   - `/api/service/rate/batch`
   - `/api/service/pair/{pair}`
   - `/api/service/sentiment`
   - `/api/service/history`
   - `/api/service/market-status`

2. **Real Data API** (`backend/routers/real_data_api.py`)
   - `/api/models/list`
   - `/api/models/initialize`
   - `/api/sentiment/analyze`
   - `/api/providers`

3. **Direct API** (`backend/routers/direct_api.py`)
   - `/api/v1/coingecko/price`
   - `/api/v1/binance/klines`
   - `/api/v1/hf/sentiment`
   - `/api/v1/hf/models`

4. **Crypto API Hub** (`backend/routers/crypto_api_hub_router.py`)
   - `/api/crypto-hub/*`

5. **AI API** (`backend/routers/ai_api.py`)
   - `/api/ai/*`

6. **Market API** (`backend/routers/market_api.py`)
   - `/api/market/*`

7. **Technical Analysis API** (`backend/routers/technical_analysis_api.py`)
   - `/api/technical/*`

8. **Real-Time Monitoring API** (`backend/routers/realtime_monitoring_api.py`) ⭐
   - `/api/monitoring/*` - **System Monitor endpoints**

---

## 🎨 Static Assets

### CSS Files

| File | Path | Used By |
|------|------|---------|
| Design System | `static/shared/css/design-system.css` | All pages |
| Global Styles | `static/shared/css/global.css` | All pages |
| Components | `static/shared/css/components.css` | All pages |
| Layout | `static/shared/css/layout.css` | All pages |
| Dashboard | `static/pages/dashboard/dashboard.css` | Dashboard page |
| **System Monitor** | `static/pages/system-monitor/system-monitor.css` | **System Monitor page** |

### JavaScript Files

| File | Path | Purpose |
|------|------|---------|
| Layout Manager | `static/shared/js/core/layout-manager.js` | Loads sidebar/header |
| API Client | `static/shared/js/core/api-client.js` | API communication |
| Models Client | `static/shared/js/core/models-client.js` | Models API client |
| **System Monitor** | `static/pages/system-monitor/system-monitor.js` | **System Monitor logic** |
| Crypto Icons | `static/assets/icons/crypto-icons.js` | SVG icons library |

---

## ⚙️ Backend Services

### Service Files

| Service | File Path | Used By |
|---------|-----------|---------|
| AI Models Monitor | `backend/services/ai_models_monitor.py` | System Monitor, Models API |
| Source Pool Manager | `monitoring/source_pool_manager.py` | System Monitor |
| Database Manager | `database/db_manager.py` | All services |
| Backtesting Service | `backend/services/backtesting_service.py` | Trading API |
| ML Training Service | `backend/services/ml_training_service.py` | AI API |

### Main Application File

| File | Path | Purpose |
|------|------|---------|
| FastAPI Server | `hf_unified_server.py` | Main application entry point |
| Server Runner | `main.py` | Start server with uvicorn |
| AI Models Registry | `ai_models.py` | Model management |

---

## 💾 Database Files

| Database | Path | Purpose |
|----------|------|---------|
| AI Models DB | `data/ai_models.db` | AI models monitoring data |
| Main Database | SQLite via `database/db_manager.py` | Providers, sources, pools |

### Database Models

| Model | File Path | Description |
|-------|-----------|-------------|
| Provider | `database/models.py` | Data provider information |
| SourcePool | `database/models.py` | Source pool management |
| PoolMember | `database/models.py` | Pool member details |

---

## 📁 Configuration Files

| File | Path | Purpose |
|------|------|---------|
| Environment | `.env` | Environment variables |
| Config | `config.py` | Application configuration |
| Requirements | `requirements.txt` | Python dependencies |
| Package | `package.json` | Node.js dependencies (if any) |

---

## 🎯 System Monitor Components

### Frontend Components

#### HTML Structure
```
static/pages/system-monitor/index.html
├── <head>
│   ├── Meta tags
│   ├── Theme CSS (design-system, global, components, layout)
│   └── System Monitor CSS
├── <body>
│   ├── app-container
│   │   ├── sidebar-container (injected by LayoutManager)
│   │   └── main-content
│   │       ├── header-container (injected by LayoutManager)
│   │       └── page-content
│   │           ├── page-header (title, status badge, refresh button)
│   │           ├── stats-grid (4 stat cards)
│   │           │   ├── Database Status Card
│   │           │   ├── AI Models Card
│   │           │   ├── Data Sources Card
│   │           │   └── Active Requests Card
│   │           └── network-section
│   │               ├── section-header (title + legend)
│   │               └── network-canvas-container
│   │                   └── #network-canvas
│   ├── connection-status (fixed bottom-right)
│   └── toast-container
└── <script>
    └── LayoutManager.init('system-monitor')
    └── SystemMonitor class initialization
```

#### JavaScript Class Structure
```
static/pages/system-monitor/system-monitor.js
└── SystemMonitor class
    ├── constructor()
    ├── init()
    ├── setupCanvas()
    ├── connectWebSocket() → /api/monitoring/ws
    ├── startPolling() → /api/monitoring/status
    ├── fetchSystemStatus()
    ├── updateSystemStatus(data)
    ├── updateHeader()
    ├── updateDatabaseStatus()
    ├── updateAIModels()
    ├── updateDataSources()
    ├── updateRequests()
    ├── updateNetworkNodes()
    ├── createPacket()
    ├── startAnimation()
    ├── draw() (canvas rendering)
    └── destroy()
```

#### CSS Structure
```
static/pages/system-monitor/system-monitor.css
├── Page Header Styles
├── Stats Grid Layout
├── Stat Cards
├── Status Indicators
├── Network Section
├── Canvas Container
├── Connection Status
└── Responsive Media Queries
```

### Backend Components

#### API Router
```
backend/routers/realtime_monitoring_api.py
├── Router: APIRouter(prefix="/api/monitoring")
├── Endpoints:
│   ├── GET /status → get_system_status()
│   ├── GET /sources/detailed → get_detailed_sources()
│   ├── GET /requests/recent → get_recent_requests()
│   ├── POST /requests/log → log_request()
│   └── WebSocket /ws → websocket_endpoint()
└── Dependencies:
    ├── ai_models_db (AI models database)
    ├── db_manager (Main database)
    └── SourcePoolManager (Source pool management)
```

#### Data Flow
```
Frontend (system-monitor.js)
    ↓
    ├─→ WebSocket: /api/monitoring/ws
    │   └─→ Real-time updates every 2 seconds
    │
    └─→ HTTP Polling: /api/monitoring/status
        └─→ Fallback if WebSocket fails
        
Backend (realtime_monitoring_api.py)
    ↓
    ├─→ AI Models Monitor Service
    │   └─→ Get models status, health, metrics
    │
    ├─→ Database Manager
    │   └─→ Get providers, sources, pools
    │
    └─→ Request Log (in-memory)
        └─→ Recent API requests tracking
```

---

## 🔄 Execution Flow

### Server Startup

1. **Entry Point**: `main.py`
   ```bash
   python main.py
   ```

2. **Server File**: `hf_unified_server.py`
   - Loads all routers
   - Includes `realtime_monitoring_router`
   - Sets up middleware
   - Starts uvicorn server on port 7860

3. **Routes Registered**:
   - All page routes (`/system-monitor`, `/dashboard`, etc.)
   - All API routes (`/api/*`)
   - WebSocket routes (`/api/monitoring/ws`)

### System Monitor Page Load

1. **User navigates to**: `http://localhost:7860/system-monitor`

2. **FastAPI serves**: `static/pages/system-monitor/index.html`

3. **HTML loads**:
   - LayoutManager initializes
   - Sidebar injected from `static/shared/layouts/sidebar.html`
   - Header injected
   - System Monitor CSS loaded

4. **JavaScript executes**:
   - `SystemMonitor` class instantiated
   - Canvas setup
   - WebSocket connection to `/api/monitoring/ws`
   - HTTP polling to `/api/monitoring/status` (every 2s)

5. **Data updates**:
   - Backend gathers status from:
     - AI Models Monitor DB
     - Main Database (providers, sources)
     - Request log
   - Returns JSON to frontend
   - Frontend updates UI and canvas animation

---

## 📂 Complete File Tree

```
crypto-dt-source-main/
├── hf_unified_server.py          → Main FastAPI application
├── main.py                        → Server entry point
├── ai_models.py                   → AI models registry
├── config.py                      → Configuration
├── requirements.txt               → Dependencies
│
├── static/
│   ├── pages/
│   │   ├── system-monitor/       ⭐ System Monitor
│   │   │   ├── index.html
│   │   │   ├── system-monitor.js
│   │   │   ├── system-monitor.css
│   │   │   └── README.md
│   │   ├── dashboard/
│   │   ├── market/
│   │   ├── models/
│   │   └── ... (other pages)
│   │
│   ├── shared/
│   │   ├── layouts/
│   │   │   ├── sidebar.html       → Includes System Monitor link
│   │   │   └── sidebar-modern.html
│   │   ├── js/
│   │   │   ├── core/
│   │   │   │   ├── layout-manager.js
│   │   │   │   ├── api-client.js
│   │   │   │   └── models-client.js
│   │   │   └── sidebar-manager.js
│   │   └── css/
│   │       ├── design-system.css
│   │       ├── global.css
│   │       ├── components.css
│   │       └── layout.css
│   │
│   └── assets/
│       └── icons/
│           └── crypto-icons.js
│
├── backend/
│   ├── routers/
│   │   ├── realtime_monitoring_api.py  ⭐ System Monitor API
│   │   ├── unified_service_api.py
│   │   ├── real_data_api.py
│   │   ├── direct_api.py
│   │   ├── ai_api.py
│   │   ├── market_api.py
│   │   └── ... (other routers)
│   │
│   └── services/
│       ├── ai_models_monitor.py   → Used by System Monitor
│       ├── backtesting_service.py
│       └── ml_training_service.py
│
├── database/
│   ├── db_manager.py             → Used by System Monitor
│   └── models.py                  → Provider, SourcePool, etc.
│
├── monitoring/
│   └── source_pool_manager.py    → Used by System Monitor
│
└── data/
    └── ai_models.db               → AI models monitoring database
```

---

## 🚀 Quick Reference

### Access System Monitor
- **URL**: `http://localhost:7860/system-monitor`
- **Route Handler**: `hf_unified_server.py:409` → `system_monitor_page()`
- **HTML File**: `static/pages/system-monitor/index.html`

### API Endpoints
- **Status**: `GET http://localhost:7860/api/monitoring/status`
- **WebSocket**: `WS ws://localhost:7860/api/monitoring/ws`
- **Recent Requests**: `GET http://localhost:7860/api/monitoring/requests/recent`

### Key Files
- **Frontend**: `static/pages/system-monitor/index.html`
- **JavaScript**: `static/pages/system-monitor/system-monitor.js`
- **CSS**: `static/pages/system-monitor/system-monitor.css`
- **Backend API**: `backend/routers/realtime_monitoring_api.py`
- **Sidebar**: `static/shared/layouts/sidebar.html` (line ~157)

---

## 📝 Notes

- All paths are relative to project root: `crypto-dt-source-main/`
- Server runs on port **7860** by default (configurable via `PORT` env var)
- System Monitor uses both WebSocket and HTTP polling for reliability
- Sidebar is injected by `LayoutManager` on page load
- Theme is applied via CSS variables defined in `design-system.css`

---

**Last Updated**: 2025-12-08  
**Version**: 2.0