feat: 初始化 bls-onoffline-backend 项目基础结构
添加 Kafka 消费者、数据库写入、Redis 集成等核心模块，实现设备上下线事件处理 - 创建项目基础目录结构与配置文件 - 实现 Kafka 消费逻辑与手动提交偏移量 - 添加 PostgreSQL 数据库连接与分区表管理 - 集成 Redis 用于错误队列和项目心跳 - 包含数据处理逻辑，区分重启与非重启数据 - 提供数据库初始化脚本与分区创建工具 - 添加单元测试与代码校验脚本
This commit is contained in:
456
bls-onoffline-backend/openspec/AGENTS.md
Normal file
456
bls-onoffline-backend/openspec/AGENTS.md
Normal file
@@ -0,0 +1,456 @@
|
||||
# OpenSpec Instructions
|
||||
|
||||
Instructions for AI coding assistants using OpenSpec for spec-driven development.
|
||||
|
||||
## TL;DR Quick Checklist
|
||||
|
||||
- Search existing work: `openspec spec list --long`, `openspec list` (use `rg` only for full-text search)
|
||||
- Decide scope: new capability vs modify existing capability
|
||||
- Pick a unique `change-id`: kebab-case, verb-led (`add-`, `update-`, `remove-`, `refactor-`)
|
||||
- Scaffold: `proposal.md`, `tasks.md`, `design.md` (only if needed), and delta specs per affected capability
|
||||
- Write deltas: use `## ADDED|MODIFIED|REMOVED|RENAMED Requirements`; include at least one `#### Scenario:` per requirement
|
||||
- Validate: `openspec validate [change-id] --strict` and fix issues
|
||||
- Request approval: Do not start implementation until proposal is approved
|
||||
|
||||
## Three-Stage Workflow
|
||||
|
||||
### Stage 1: Creating Changes
|
||||
Create proposal when you need to:
|
||||
- Add features or functionality
|
||||
- Make breaking changes (API, schema)
|
||||
- Change architecture or patterns
|
||||
- Optimize performance (changes behavior)
|
||||
- Update security patterns
|
||||
|
||||
Triggers (examples):
|
||||
- "Help me create a change proposal"
|
||||
- "Help me plan a change"
|
||||
- "Help me create a proposal"
|
||||
- "I want to create a spec proposal"
|
||||
- "I want to create a spec"
|
||||
|
||||
Loose matching guidance:
|
||||
- Contains one of: `proposal`, `change`, `spec`
|
||||
- With one of: `create`, `plan`, `make`, `start`, `help`
|
||||
|
||||
Skip proposal for:
|
||||
- Bug fixes (restore intended behavior)
|
||||
- Typos, formatting, comments
|
||||
- Dependency updates (non-breaking)
|
||||
- Configuration changes
|
||||
- Tests for existing behavior
|
||||
|
||||
**Workflow**
|
||||
1. Review `openspec/project.md`, `openspec list`, and `openspec list --specs` to understand current context.
|
||||
2. Choose a unique verb-led `change-id` and scaffold `proposal.md`, `tasks.md`, optional `design.md`, and spec deltas under `openspec/changes/<id>/`.
|
||||
3. Draft spec deltas using `## ADDED|MODIFIED|REMOVED Requirements` with at least one `#### Scenario:` per requirement.
|
||||
4. Run `openspec validate <id> --strict` and resolve any issues before sharing the proposal.
|
||||
|
||||
### Stage 2: Implementing Changes
|
||||
Track these steps as TODOs and complete them one by one.
|
||||
1. **Read proposal.md** - Understand what's being built
|
||||
2. **Read design.md** (if exists) - Review technical decisions
|
||||
3. **Read tasks.md** - Get implementation checklist
|
||||
4. **Implement tasks sequentially** - Complete in order
|
||||
5. **Confirm completion** - Ensure every item in `tasks.md` is finished before updating statuses
|
||||
6. **Update checklist** - After all work is done, set every task to `- [x]` so the list reflects reality
|
||||
7. **Approval gate** - Do not start implementation until the proposal is reviewed and approved
|
||||
|
||||
### Stage 3: Archiving Changes
|
||||
After deployment, create separate PR to:
|
||||
- Move `changes/[name]/` → `changes/archive/YYYY-MM-DD-[name]/`
|
||||
- Update `specs/` if capabilities changed
|
||||
- Use `openspec archive <change-id> --skip-specs --yes` for tooling-only changes (always pass the change ID explicitly)
|
||||
- Run `openspec validate --strict` to confirm the archived change passes checks
|
||||
|
||||
## Before Any Task
|
||||
|
||||
**Context Checklist:**
|
||||
- [ ] Read relevant specs in `specs/[capability]/spec.md`
|
||||
- [ ] Check pending changes in `changes/` for conflicts
|
||||
- [ ] Read `openspec/project.md` for conventions
|
||||
- [ ] Run `openspec list` to see active changes
|
||||
- [ ] Run `openspec list --specs` to see existing capabilities
|
||||
|
||||
**Before Creating Specs:**
|
||||
- Always check if capability already exists
|
||||
- Prefer modifying existing specs over creating duplicates
|
||||
- Use `openspec show [spec]` to review current state
|
||||
- If request is ambiguous, ask 1–2 clarifying questions before scaffolding
|
||||
|
||||
### Search Guidance
|
||||
- Enumerate specs: `openspec spec list --long` (or `--json` for scripts)
|
||||
- Enumerate changes: `openspec list` (or `openspec change list --json` - deprecated but available)
|
||||
- Show details:
|
||||
- Spec: `openspec show <spec-id> --type spec` (use `--json` for filters)
|
||||
- Change: `openspec show <change-id> --json --deltas-only`
|
||||
- Full-text search (use ripgrep): `rg -n "Requirement:|Scenario:" openspec/specs`
|
||||
|
||||
## Quick Start
|
||||
|
||||
### CLI Commands
|
||||
|
||||
```bash
|
||||
# Essential commands
|
||||
openspec list # List active changes
|
||||
openspec list --specs # List specifications
|
||||
openspec show [item] # Display change or spec
|
||||
openspec validate [item] # Validate changes or specs
|
||||
openspec archive <change-id> [--yes|-y] # Archive after deployment (add --yes for non-interactive runs)
|
||||
|
||||
# Project management
|
||||
openspec init [path] # Initialize OpenSpec
|
||||
openspec update [path] # Update instruction files
|
||||
|
||||
# Interactive mode
|
||||
openspec show # Prompts for selection
|
||||
openspec validate # Bulk validation mode
|
||||
|
||||
# Debugging
|
||||
openspec show [change] --json --deltas-only
|
||||
openspec validate [change] --strict
|
||||
```
|
||||
|
||||
### Command Flags
|
||||
|
||||
- `--json` - Machine-readable output
|
||||
- `--type change|spec` - Disambiguate items
|
||||
- `--strict` - Comprehensive validation
|
||||
- `--no-interactive` - Disable prompts
|
||||
- `--skip-specs` - Archive without spec updates
|
||||
- `--yes`/`-y` - Skip confirmation prompts (non-interactive archive)
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
openspec/
|
||||
├── project.md # Project conventions
|
||||
├── specs/ # Current truth - what IS built
|
||||
│ └── [capability]/ # Single focused capability
|
||||
│ ├── spec.md # Requirements and scenarios
|
||||
│ └── design.md # Technical patterns
|
||||
├── changes/ # Proposals - what SHOULD change
|
||||
│ ├── [change-name]/
|
||||
│ │ ├── proposal.md # Why, what, impact
|
||||
│ │ ├── tasks.md # Implementation checklist
|
||||
│ │ ├── design.md # Technical decisions (optional; see criteria)
|
||||
│ │ └── specs/ # Delta changes
|
||||
│ │ └── [capability]/
|
||||
│ │ └── spec.md # ADDED/MODIFIED/REMOVED
|
||||
│ └── archive/ # Completed changes
|
||||
```
|
||||
|
||||
## Creating Change Proposals
|
||||
|
||||
### Decision Tree
|
||||
|
||||
```
|
||||
New request?
|
||||
├─ Bug fix restoring spec behavior? → Fix directly
|
||||
├─ Typo/format/comment? → Fix directly
|
||||
├─ New feature/capability? → Create proposal
|
||||
├─ Breaking change? → Create proposal
|
||||
├─ Architecture change? → Create proposal
|
||||
└─ Unclear? → Create proposal (safer)
|
||||
```
|
||||
|
||||
### Proposal Structure
|
||||
|
||||
1. **Create directory:** `changes/[change-id]/` (kebab-case, verb-led, unique)
|
||||
|
||||
2. **Write proposal.md:**
|
||||
```markdown
|
||||
# Change: [Brief description of change]
|
||||
|
||||
## Why
|
||||
[1-2 sentences on problem/opportunity]
|
||||
|
||||
## What Changes
|
||||
- [Bullet list of changes]
|
||||
- [Mark breaking changes with **BREAKING**]
|
||||
|
||||
## Impact
|
||||
- Affected specs: [list capabilities]
|
||||
- Affected code: [key files/systems]
|
||||
```
|
||||
|
||||
3. **Create spec deltas:** `specs/[capability]/spec.md`
|
||||
```markdown
|
||||
## ADDED Requirements
|
||||
### Requirement: New Feature
|
||||
The system SHALL provide...
|
||||
|
||||
#### Scenario: Success case
|
||||
- **WHEN** user performs action
|
||||
- **THEN** expected result
|
||||
|
||||
## MODIFIED Requirements
|
||||
### Requirement: Existing Feature
|
||||
[Complete modified requirement]
|
||||
|
||||
## REMOVED Requirements
|
||||
### Requirement: Old Feature
|
||||
**Reason**: [Why removing]
|
||||
**Migration**: [How to handle]
|
||||
```
|
||||
If multiple capabilities are affected, create multiple delta files under `changes/[change-id]/specs/<capability>/spec.md`—one per capability.
|
||||
|
||||
4. **Create tasks.md:**
|
||||
```markdown
|
||||
## 1. Implementation
|
||||
- [ ] 1.1 Create database schema
|
||||
- [ ] 1.2 Implement API endpoint
|
||||
- [ ] 1.3 Add frontend component
|
||||
- [ ] 1.4 Write tests
|
||||
```
|
||||
|
||||
5. **Create design.md when needed:**
|
||||
Create `design.md` if any of the following apply; otherwise omit it:
|
||||
- Cross-cutting change (multiple services/modules) or a new architectural pattern
|
||||
- New external dependency or significant data model changes
|
||||
- Security, performance, or migration complexity
|
||||
- Ambiguity that benefits from technical decisions before coding
|
||||
|
||||
Minimal `design.md` skeleton:
|
||||
```markdown
|
||||
## Context
|
||||
[Background, constraints, stakeholders]
|
||||
|
||||
## Goals / Non-Goals
|
||||
- Goals: [...]
|
||||
- Non-Goals: [...]
|
||||
|
||||
## Decisions
|
||||
- Decision: [What and why]
|
||||
- Alternatives considered: [Options + rationale]
|
||||
|
||||
## Risks / Trade-offs
|
||||
- [Risk] → Mitigation
|
||||
|
||||
## Migration Plan
|
||||
[Steps, rollback]
|
||||
|
||||
## Open Questions
|
||||
- [...]
|
||||
```
|
||||
|
||||
## Spec File Format
|
||||
|
||||
### Critical: Scenario Formatting
|
||||
|
||||
**CORRECT** (use #### headers):
|
||||
```markdown
|
||||
#### Scenario: User login success
|
||||
- **WHEN** valid credentials provided
|
||||
- **THEN** return JWT token
|
||||
```
|
||||
|
||||
**WRONG** (don't use bullets or bold):
|
||||
```markdown
|
||||
- **Scenario: User login** ❌
|
||||
**Scenario**: User login ❌
|
||||
### Scenario: User login ❌
|
||||
```
|
||||
|
||||
Every requirement MUST have at least one scenario.
|
||||
|
||||
### Requirement Wording
|
||||
- Use SHALL/MUST for normative requirements (avoid should/may unless intentionally non-normative)
|
||||
|
||||
### Delta Operations
|
||||
|
||||
- `## ADDED Requirements` - New capabilities
|
||||
- `## MODIFIED Requirements` - Changed behavior
|
||||
- `## REMOVED Requirements` - Deprecated features
|
||||
- `## RENAMED Requirements` - Name changes
|
||||
|
||||
Headers matched with `trim(header)` - whitespace ignored.
|
||||
|
||||
#### When to use ADDED vs MODIFIED
|
||||
- ADDED: Introduces a new capability or sub-capability that can stand alone as a requirement. Prefer ADDED when the change is orthogonal (e.g., adding "Slash Command Configuration") rather than altering the semantics of an existing requirement.
|
||||
- MODIFIED: Changes the behavior, scope, or acceptance criteria of an existing requirement. Always paste the full, updated requirement content (header + all scenarios). The archiver will replace the entire requirement with what you provide here; partial deltas will drop previous details.
|
||||
- RENAMED: Use when only the name changes. If you also change behavior, use RENAMED (name) plus MODIFIED (content) referencing the new name.
|
||||
|
||||
Common pitfall: Using MODIFIED to add a new concern without including the previous text. This causes loss of detail at archive time. If you aren’t explicitly changing the existing requirement, add a new requirement under ADDED instead.
|
||||
|
||||
Authoring a MODIFIED requirement correctly:
|
||||
1) Locate the existing requirement in `openspec/specs/<capability>/spec.md`.
|
||||
2) Copy the entire requirement block (from `### Requirement: ...` through its scenarios).
|
||||
3) Paste it under `## MODIFIED Requirements` and edit to reflect the new behavior.
|
||||
4) Ensure the header text matches exactly (whitespace-insensitive) and keep at least one `#### Scenario:`.
|
||||
|
||||
Example for RENAMED:
|
||||
```markdown
|
||||
## RENAMED Requirements
|
||||
- FROM: `### Requirement: Login`
|
||||
- TO: `### Requirement: User Authentication`
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Errors
|
||||
|
||||
**"Change must have at least one delta"**
|
||||
- Check `changes/[name]/specs/` exists with .md files
|
||||
- Verify files have operation prefixes (## ADDED Requirements)
|
||||
|
||||
**"Requirement must have at least one scenario"**
|
||||
- Check scenarios use `#### Scenario:` format (4 hashtags)
|
||||
- Don't use bullet points or bold for scenario headers
|
||||
|
||||
**Silent scenario parsing failures**
|
||||
- Exact format required: `#### Scenario: Name`
|
||||
- Debug with: `openspec show [change] --json --deltas-only`
|
||||
|
||||
### Validation Tips
|
||||
|
||||
```bash
|
||||
# Always use strict mode for comprehensive checks
|
||||
openspec validate [change] --strict
|
||||
|
||||
# Debug delta parsing
|
||||
openspec show [change] --json | jq '.deltas'
|
||||
|
||||
# Check specific requirement
|
||||
openspec show [spec] --json -r 1
|
||||
```
|
||||
|
||||
## Happy Path Script
|
||||
|
||||
```bash
|
||||
# 1) Explore current state
|
||||
openspec spec list --long
|
||||
openspec list
|
||||
# Optional full-text search:
|
||||
# rg -n "Requirement:|Scenario:" openspec/specs
|
||||
# rg -n "^#|Requirement:" openspec/changes
|
||||
|
||||
# 2) Choose change id and scaffold
|
||||
CHANGE=add-two-factor-auth
|
||||
mkdir -p openspec/changes/$CHANGE/{specs/auth}
|
||||
printf "## Why\n...\n\n## What Changes\n- ...\n\n## Impact\n- ...\n" > openspec/changes/$CHANGE/proposal.md
|
||||
printf "## 1. Implementation\n- [ ] 1.1 ...\n" > openspec/changes/$CHANGE/tasks.md
|
||||
|
||||
# 3) Add deltas (example)
|
||||
cat > openspec/changes/$CHANGE/specs/auth/spec.md << 'EOF'
|
||||
## ADDED Requirements
|
||||
### Requirement: Two-Factor Authentication
|
||||
Users MUST provide a second factor during login.
|
||||
|
||||
#### Scenario: OTP required
|
||||
- **WHEN** valid credentials are provided
|
||||
- **THEN** an OTP challenge is required
|
||||
EOF
|
||||
|
||||
# 4) Validate
|
||||
openspec validate $CHANGE --strict
|
||||
```
|
||||
|
||||
## Multi-Capability Example
|
||||
|
||||
```
|
||||
openspec/changes/add-2fa-notify/
|
||||
├── proposal.md
|
||||
├── tasks.md
|
||||
└── specs/
|
||||
├── auth/
|
||||
│ └── spec.md # ADDED: Two-Factor Authentication
|
||||
└── notifications/
|
||||
└── spec.md # ADDED: OTP email notification
|
||||
```
|
||||
|
||||
auth/spec.md
|
||||
```markdown
|
||||
## ADDED Requirements
|
||||
### Requirement: Two-Factor Authentication
|
||||
...
|
||||
```
|
||||
|
||||
notifications/spec.md
|
||||
```markdown
|
||||
## ADDED Requirements
|
||||
### Requirement: OTP Email Notification
|
||||
...
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Simplicity First
|
||||
- Default to <100 lines of new code
|
||||
- Single-file implementations until proven insufficient
|
||||
- Avoid frameworks without clear justification
|
||||
- Choose boring, proven patterns
|
||||
|
||||
### Complexity Triggers
|
||||
Only add complexity with:
|
||||
- Performance data showing current solution too slow
|
||||
- Concrete scale requirements (>1000 users, >100MB data)
|
||||
- Multiple proven use cases requiring abstraction
|
||||
|
||||
### Clear References
|
||||
- Use `file.ts:42` format for code locations
|
||||
- Reference specs as `specs/auth/spec.md`
|
||||
- Link related changes and PRs
|
||||
|
||||
### Capability Naming
|
||||
- Use verb-noun: `user-auth`, `payment-capture`
|
||||
- Single purpose per capability
|
||||
- 10-minute understandability rule
|
||||
- Split if description needs "AND"
|
||||
|
||||
### Change ID Naming
|
||||
- Use kebab-case, short and descriptive: `add-two-factor-auth`
|
||||
- Prefer verb-led prefixes: `add-`, `update-`, `remove-`, `refactor-`
|
||||
- Ensure uniqueness; if taken, append `-2`, `-3`, etc.
|
||||
|
||||
## Tool Selection Guide
|
||||
|
||||
| Task | Tool | Why |
|
||||
|------|------|-----|
|
||||
| Find files by pattern | Glob | Fast pattern matching |
|
||||
| Search code content | Grep | Optimized regex search |
|
||||
| Read specific files | Read | Direct file access |
|
||||
| Explore unknown scope | Task | Multi-step investigation |
|
||||
|
||||
## Error Recovery
|
||||
|
||||
### Change Conflicts
|
||||
1. Run `openspec list` to see active changes
|
||||
2. Check for overlapping specs
|
||||
3. Coordinate with change owners
|
||||
4. Consider combining proposals
|
||||
|
||||
### Validation Failures
|
||||
1. Run with `--strict` flag
|
||||
2. Check JSON output for details
|
||||
3. Verify spec file format
|
||||
4. Ensure scenarios properly formatted
|
||||
|
||||
### Missing Context
|
||||
1. Read project.md first
|
||||
2. Check related specs
|
||||
3. Review recent archives
|
||||
4. Ask for clarification
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Stage Indicators
|
||||
- `changes/` - Proposed, not yet built
|
||||
- `specs/` - Built and deployed
|
||||
- `archive/` - Completed changes
|
||||
|
||||
### File Purposes
|
||||
- `proposal.md` - Why and what
|
||||
- `tasks.md` - Implementation steps
|
||||
- `design.md` - Technical decisions
|
||||
- `spec.md` - Requirements and behavior
|
||||
|
||||
### CLI Essentials
|
||||
```bash
|
||||
openspec list # What's in progress?
|
||||
openspec show [item] # View details
|
||||
openspec validate --strict # Is it correct?
|
||||
openspec archive <change-id> [--yes|-y] # Mark complete (add --yes for automation)
|
||||
```
|
||||
|
||||
Remember: Specs are truth. Changes are proposals. Keep them in sync.
|
||||
@@ -0,0 +1,17 @@
|
||||
# Change: Fix Kafka Partitioning and Schema Issues
|
||||
|
||||
## Why
|
||||
Production deployment revealed issues with data ingestion:
|
||||
1. Kafka Topic name changed to include partition suffix.
|
||||
2. Legacy data contains second-level timestamps (1970s) causing partition lookup failures in PostgreSQL (which expects ms).
|
||||
3. Variable-length fields (reboot reason, status) exceeded VARCHAR(10) limits, causing crashes.
|
||||
|
||||
## What Changes
|
||||
- **Modified Requirement**: Update Kafka Topic to `blwlog4Nodejs-rcu-onoffline-topic-0`.
|
||||
- **New Requirement**: Implement heuristic timestamp conversion (Sec -> MS) for values < 100B.
|
||||
- **New Requirement**: Truncate specific fields to VARCHAR(255) to prevent DB rejection.
|
||||
- **Modified Requirement**: Update DB Schema to VARCHAR(255) for robustness.
|
||||
|
||||
## Impact
|
||||
- Affected specs: `onoffline`
|
||||
- Affected code: `src/processor/index.js`, `scripts/init_db.sql`
|
||||
@@ -0,0 +1,25 @@
|
||||
## MODIFIED Requirements
|
||||
### Requirement: 消费并落库
|
||||
系统 SHALL 从 blwlog4Nodejs-rcu-onoffline-topic-0 消费消息，并写入 log_platform.onoffline.onoffline_record。
|
||||
|
||||
#### Scenario: 非重启数据写入
|
||||
- **GIVEN** RebootReason 为空或不存在
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** current_status 等于 CurrentStatus (截断至 255 字符)
|
||||
|
||||
## ADDED Requirements
|
||||
### Requirement: 字段长度限制与截断
|
||||
系统 SHALL 将部分变长字段截断至数据库允许的最大长度 (VARCHAR(255))，防止写入失败。
|
||||
|
||||
#### Scenario: 超长字段处理
|
||||
- **GIVEN** LauncherVersion, CurrentStatus 或 RebootReason 超过 255 字符
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** 字段被截断为前 255 个字符并入库
|
||||
|
||||
### Requirement: 时间戳单位自动识别
|
||||
系统 SHALL 自动识别 UnixTime 字段是秒还是毫秒，并统一转换为毫秒。
|
||||
|
||||
#### Scenario: 秒级时间戳转换
|
||||
- **GIVEN** UnixTime < 100000000000 (约 1973 年前)
|
||||
- **WHEN** 解析时间戳
|
||||
- **THEN** 自动乘以 1000 转换为毫秒
|
||||
@@ -0,0 +1,6 @@
|
||||
## 1. Implementation
|
||||
- [x] Update Kafka Topic in .env and config
|
||||
- [x] Implement timestamp unit detection and conversion in processor
|
||||
- [x] Implement field truncation logic in processor
|
||||
- [x] Update database schema definition (init_db.sql) to VARCHAR(255)
|
||||
- [x] Verify data ingestion with production stream
|
||||
31
bls-onoffline-backend/openspec/project.md
Normal file
31
bls-onoffline-backend/openspec/project.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Project Context
|
||||
|
||||
## Purpose
|
||||
[Describe your project's purpose and goals]
|
||||
|
||||
## Tech Stack
|
||||
- [List your primary technologies]
|
||||
- [e.g., TypeScript, React, Node.js]
|
||||
|
||||
## Project Conventions
|
||||
|
||||
### Code Style
|
||||
[Describe your code style preferences, formatting rules, and naming conventions]
|
||||
|
||||
### Architecture Patterns
|
||||
[Document your architectural decisions and patterns]
|
||||
|
||||
### Testing Strategy
|
||||
[Explain your testing approach and requirements]
|
||||
|
||||
### Git Workflow
|
||||
[Describe your branching strategy and commit conventions]
|
||||
|
||||
## Domain Context
|
||||
[Add domain-specific knowledge that AI assistants need to understand]
|
||||
|
||||
## Important Constraints
|
||||
[List any technical, business, or regulatory constraints]
|
||||
|
||||
## External Dependencies
|
||||
[Document key external services, APIs, or systems]
|
||||
85
bls-onoffline-backend/openspec/specs/onoffline/spec.md
Normal file
85
bls-onoffline-backend/openspec/specs/onoffline/spec.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Spec: onoffline-backend
|
||||
|
||||
## Purpose
|
||||
从 Kafka 消费设备上下线事件并按规则写入 PostgreSQL 分区表，确保高可靠性、幂等写入和错误恢复能力。
|
||||
## Requirements
|
||||
### Requirement: 消费并落库
|
||||
系统 SHALL 从 blwlog4Nodejs-rcu-onoffline-topic-0 消费消息，并写入 log_platform.onoffline.onoffline_record。
|
||||
|
||||
#### Scenario: 非重启数据写入
|
||||
- **GIVEN** RebootReason 为空或不存在
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** current_status 等于 CurrentStatus (截断至 255 字符)
|
||||
|
||||
### Requirement: 重启数据处理
|
||||
系统 SHALL 在 RebootReason 非空时强制 current_status 为 on。
|
||||
|
||||
#### Scenario: 重启数据写入
|
||||
- **GIVEN** RebootReason 为非空值
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** current_status 等于 on
|
||||
|
||||
### Requirement: 空值保留
|
||||
系统 SHALL 保留上游空值，不对字段进行补 0。
|
||||
|
||||
#### Scenario: 空值写入
|
||||
- **GIVEN** LauncherVersion 或 RebootReason 为空字符串
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** 数据库存储值为对应的空字符串
|
||||
|
||||
### Requirement: 数据库分区策略
|
||||
系统 SHALL 使用 Range Partitioning 按天分区，并自动维护未来 30 天的分区表。
|
||||
|
||||
#### Scenario: 分区预创建
|
||||
- **GIVEN** 系统启动或每日凌晨
|
||||
- **WHEN** 运行分区维护任务
|
||||
- **THEN** 确保数据库中存在未来 30 天的分区表
|
||||
|
||||
### Requirement: 消费可靠性 (At-Least-Once)
|
||||
系统 SHALL 仅在数据成功写入数据库后，才向 Kafka 提交消费位点。
|
||||
|
||||
#### Scenario: 逐条确认与顺序提交
|
||||
- **GIVEN** 并发处理多条消息 (Offset 1, 2, 3)
|
||||
- **WHEN** Offset 2 先完成，Offset 1 尚未完成
|
||||
- **THEN** 系统不提交 Offset 2，直到 Offset 1 也完成，才提交 Offset 3 (即 1, 2, 3 都完成)
|
||||
|
||||
### Requirement: 数据库离线保护
|
||||
系统 SHALL 在数据库连接丢失时暂停消费，防止数据堆积或丢失。
|
||||
|
||||
#### Scenario: 数据库断连
|
||||
- **GIVEN** 数据库连接失败 (ECONNREFUSED 等)
|
||||
- **WHEN** 消费者尝试写入
|
||||
- **THEN** 暂停 Kafka 消费 1 分钟，并进入轮询检测模式，直到数据库恢复
|
||||
|
||||
### Requirement: 幂等写入
|
||||
系统 SHALL 处理重复消费的数据，防止主键冲突。
|
||||
|
||||
#### Scenario: 重复数据处理
|
||||
- **GIVEN** Kafka 重新投递已处理过的消息
|
||||
- **WHEN** 尝试写入数据库
|
||||
- **THEN** 使用 `ON CONFLICT DO NOTHING` 忽略冲突，视为处理成功
|
||||
|
||||
### Requirement: 性能与日志
|
||||
系统 SHALL 最小化正常运行时的日志输出。
|
||||
|
||||
#### Scenario: 正常运行日志
|
||||
- **GIVEN** 数据正常处理
|
||||
- **WHEN** 写入成功
|
||||
- **THEN** 不输出单条日志，仅每分钟输出聚合统计 (Pulled/Inserted)
|
||||
|
||||
### Requirement: 字段长度限制与截断
|
||||
系统 SHALL 将部分变长字段截断至数据库允许的最大长度 (VARCHAR(255))，防止写入失败。
|
||||
|
||||
#### Scenario: 超长字段处理
|
||||
- **GIVEN** LauncherVersion, CurrentStatus 或 RebootReason 超过 255 字符
|
||||
- **WHEN** 消息被处理
|
||||
- **THEN** 字段被截断为前 255 个字符并入库
|
||||
|
||||
### Requirement: 时间戳单位自动识别
|
||||
系统 SHALL 自动识别 UnixTime 字段是秒还是毫秒，并统一转换为毫秒。
|
||||
|
||||
#### Scenario: 秒级时间戳转换
|
||||
- **GIVEN** UnixTime < 100000000000 (约 1973 年前)
|
||||
- **WHEN** 解析时间戳
|
||||
- **THEN** 自动乘以 1000 转换为毫秒
|
||||
|
||||
11
bls-onoffline-backend/openspec/specs/onoffline/status.md
Normal file
11
bls-onoffline-backend/openspec/specs/onoffline/status.md
Normal file
@@ -0,0 +1,11 @@
|
||||
|
||||
## Implementation Status
|
||||
- **Date**: 2026-02-04
|
||||
- **Status**: Completed
|
||||
- **Notes**:
|
||||
- 已完成核心消费逻辑、分区管理、数据库幂等写入。
|
||||
- 已处理数据库连接泄露 (EADDRINUSE) 问题，增加了离线保护机制。
|
||||
- 已修复时间戳单位问题 (Seconds -> MS)。
|
||||
- 已将关键字段长度扩展至 VARCHAR(255) 并增加了代码层截断保护。
|
||||
- 验证了数据积压消费能力。
|
||||
- 本阶段开发任务已归档。
|
||||
Reference in New Issue
Block a user