Web_BLS_ProjectConsole/openspec/specs/logging/design.md

# Logging Capability Design

## Context
This design document describes the technical implementation of the logging capability for the BLS Project Console, which allows the system to read log records from Redis queues and display them in the console interface.

## Goals / Non-Goals

### Goals
- Implement real-time log reading from Redis queues
- Provide a user-friendly log display interface
- Support log filtering by level and time range
- Ensure high performance and low latency
- Implement proper error handling and reconnection mechanisms

### Non-Goals
- Log storage or persistence beyond memory
- Log analysis or visualization (charts, graphs)
- Advanced log search capabilities

## Decisions

### Decision: Redis Queue Implementation
- **What**: Use Redis List as the queue data structure
- **Why**: Redis Lists provide efficient push/pop operations with O(1) time complexity, making them ideal for message queues
- **Alternatives considered**: 
  - Redis Streams: More advanced but overkill for our use case
  - Redis Pub/Sub: No persistence, so logs would be lost if the server is down

### Decision: Real-time Updates
- **What**: Use Server-Sent Events (SSE) for real-time log updates
- **Why**: SSE is simpler than WebSockets for one-way communication, has better browser support, and is easier to implement
- **Alternatives considered**: 
  - WebSockets: More complex for one-way communication
  - Polling: Higher latency and more resource-intensive

### Decision: Log Storage
- **What**: Store logs in memory with a configurable maximum size
- **Why**: In-memory storage provides fast access times and avoids the complexity of database management
- **Alternatives considered**: 
  - Database storage: Adds complexity and latency
  - File system: Not suitable for real-time access

## Architecture

### Frontend Architecture
```
LogView Component
├── LogList Component
├── LogFilter Component
└── LogService
```

### Backend Architecture
```
Log Routes
├── Log Service
│   ├── Redis Client
│   └── Log Manager
└── SSE Controller
```

## Implementation Details

### Redis Connection
- Use the `redis` npm package to connect to Redis
- Implement automatic reconnection with exponential backoff
- Handle connection errors gracefully

### Log Reading
1. Server establishes connection to Redis
2. Server listens for new log records using `BLPOP` command (blocking pop)
3. When a log record is received, it's added to the in-memory log store
4. The log is then sent to all connected SSE clients

### Log Storage
- Use an array to store log records in memory
- Implement a circular buffer to limit memory usage
- Default maximum log count: 10,000
- Configurable via environment variable

### Log Display
- Use a scrollable list to display logs
- Implement virtual scrolling for large log sets to improve performance
- Color-code logs by level (INFO: gray, WARN: yellow, ERROR: red, DEBUG: blue)

### Log Filtering
- Implement client-side filtering for performance
- Allow filtering by log level (INFO, WARN, ERROR, DEBUG)
- Allow filtering by time range using a date picker

## Risks / Trade-offs

### Risk: Redis Connection Failure
- **Risk**: If Redis connection is lost, logs won't be received
- **Mitigation**: Implement automatic reconnection with exponential backoff, and notify users when connection is lost

### Risk: High Log Volume
- **Risk**: Large number of logs could cause performance issues
- **Mitigation**: Implement a circular buffer to limit memory usage, and use virtual scrolling in the frontend

### Risk: Browser Performance
- **Risk**: Displaying thousands of logs could slow down the browser
- **Mitigation**: Use virtual scrolling and limit the number of logs displayed at once

## Migration Plan
No migration is required as this is a new feature.

## Open Questions
- What is the expected maximum log volume per minute?
- Should we add support for log persistence to disk?
- Should we implement log search functionality?