Start AI knowledge base v2 workspace

ai_knowledge_base_v2/reports/legacy_reuse_evaluation.md

@@ -0,0 +1,133 @@
+# Legacy Reuse Evaluation
+
+Date: 2026-07-06
+
+## Conclusion
+
+The existing project can provide useful engineering references, but it should not be directly extended as-is for the new requirement.
+
+The old project is a **大本营答疑资产后台系统 MVP**. Its core flow is:
+
+`Feishu session/transcript -> AI cleaning -> raw QA -> manual review -> standard QA -> manually marked callable QA`
+
+The new project is an **AI 企业知识库问答系统**. Its core flow is:
+
+`User login -> authorized knowledge bases -> real-time Feishu retrieval -> Prompt assembly -> model streaming answer -> chat/history/log persistence`
+
+These two systems both touch "QA" and "Feishu", but their domain models, permissions, APIs, database tables, and user workflows are different. For long-term maintainability, V2 should use a new domain model and only reuse selected implementation patterns or small infrastructure pieces.
+
+## New Requirement Snapshot
+
+Source documents define the V2 target as:
+
+- Feishu knowledge base is the only phase-one knowledge source.
+- Phase one does not build an independent local knowledge base, does not manually sync content, and does not maintain a vector database sync.
+- User login uses phone number + SMS verification code.
+- Admin login uses username + password.
+- AI chat supports sessions, history, streaming output, stop generation, Markdown rendering, auto title, and daily chat quota.
+- AI can only answer from the user's authorized knowledge bases and retrieved snippets.
+- No-hit answer must be: `当前知识库中未检索到相关内容，请联系管理员补充相关知识。`
+- Feishu/model failures need unified fallback messages and logs.
+- Admin backend includes dashboard, user management, admin management, knowledge base management, Prompt management, model management, chat records, system config, and operation logs.
+- Core database tables include `sys_user`, `sys_admin`, `sys_role`, `sys_knowledge`, `sys_user_kb`, `sys_chat_session`, `sys_chat_message`, `sys_prompt`, `sys_model`, `sys_system_config`, `sys_ai_request_log`, and `sys_operation_log`.
+
+## Existing Code Overview
+
+The old project currently uses:
+
+- Backend: FastAPI, SQLAlchemy, Pydantic, APScheduler, HTTPX.
+- Frontend: Next.js, React, TypeScript, TailwindCSS.
+- Database: PostgreSQL preferred, SQLite fallback.
+- AI: OpenAI-compatible chat completions with mock fallback.
+- Feishu: Bitable/document adapter with mock fallback.
+- Existing H5 page: mobile-first chat surface connected to `/api/standard-qa/callable`.
+
+The old database model centers on:
+
+- `users`
+- `feishu_sessions`
+- `raw_qa_items`
+- `standard_qa_items`
+- `task_runs`
+- `audit_logs`
+- `system_settings`
+
+## Reuse Assessment
+
+| Area | Reuse Level | Recommendation |
+| --- | --- | --- |
+| FastAPI application skeleton | High | Reuse routing style, settings pattern, CORS setup, dependency injection, and service layering ideas. |
+| SQLAlchemy/Pydantic conventions | Medium | Reuse coding style, but rebuild models for `sys_*` tables required by V2. |
+| Next.js/Tailwind frontend setup | Medium | Reuse project setup and visual implementation experience; rebuild pages around V2 user/admin flows. |
+| H5 chat UI interaction | Medium | Reuse mobile-first interaction ideas such as compact header, chat bubbles, input ergonomics, and local loading states. Replace old callable-QA matching with real `/chat/completions` SSE. |
+| Feishu service adapter | Medium | Reuse token acquisition, HTTP wrapper, document text extraction ideas, and mock-mode pattern. V2 still needs a new Feishu knowledge retrieval service matching SpaceID/NodeID and real-time search. |
+| AI API wrapper | Medium | Reuse OpenAI-compatible request pattern. V2 needs streaming, cancellation, Prompt assembly, token logging, and model config from database. |
+| Operation/task logs | Low to Medium | Reuse audit/logging ideas. V2 needs `sys_operation_log` and `sys_ai_request_log`, not old QA audit semantics. |
+| Scheduler/task runner | Low | V2 phase one says no manual sync or vector DB sync, so scheduled QA processing is not core. Keep only if later needed for maintenance jobs. |
+| Old QA review domain | Low | Do not reuse as V2 core. Raw/standard QA review, risk/desensitization, and callable status are old-domain concepts. |
+| Existing `/standard-qa/callable` H5 API | Low | Replace with V2 chat/session/history APIs. It can stay as old system behavior but should not drive V2. |
+| Docker Compose | Medium | Reuse containerization pattern, but V2 should switch DB target to MySQL 8.x per documents or explicitly record a deviation if PostgreSQL remains. |
+
+## Gap Analysis
+
+Must build or redesign for V2:
+
+- User SMS login, token/session invalidation, account expiry, and daily quota.
+- Admin username/password login and role/permission control.
+- Knowledge base management using `SpaceID` and `NodeID`.
+- User-to-knowledge permission table with effective/expired dates.
+- Chat session and message persistence.
+- SSE streaming endpoint `POST /chat/completions`.
+- Stop-generation endpoint `POST /chat/stop`.
+- Prompt management and active Prompt loading.
+- Model management with API URL/API key/model settings.
+- System config for login expiry, daily chat count, AI timeout, Feishu retries, context length, source display, and disabled web search.
+- Real-time Feishu retrieval with permission filtering.
+- AI request log with prompt, retrieval knowledge IDs, token usage, latency, status, and error.
+- Admin chat query/detail/export APIs.
+- UAT/test coverage around permission isolation and no-hallucination fallback.
+
+## Recommended Technical Direction
+
+1. Keep `ai_knowledge_base_v2/` as the new context and planning workspace.
+2. Create a new V2 application folder after architecture confirmation, rather than mutating old V1 files in place.
+3. Keep backend modular from day one:
+   - `auth`
+   - `users`
+   - `admins`
+   - `knowledge`
+   - `chat`
+   - `rag`
+   - `prompts`
+   - `models`
+   - `config`
+   - `logs`
+4. Keep services separated:
+   - `SmsCodeService`
+   - `TokenService`
+   - `FeishuKnowledgeService`
+   - `RagService`
+   - `ModelClient`
+   - `ChatService`
+   - `AuditLogService`
+5. Make mock providers explicit for local development:
+   - mock SMS code
+   - mock model response
+   - mock Feishu retrieval
+6. Do not store production model API keys in plain text unless a security decision is documented. Prefer environment secret references or encrypted storage.
+
+## Risk Notes
+
+- The new documents specify MySQL 8.x, while the existing project uses PostgreSQL/SQLite. This is a deployment and migration decision, not a small code change.
+- The documents say "Feishu updates immediately effective" and "real-time retrieval"; this depends on actual Feishu APIs and tenant permissions. A technical spike should validate retrieval behavior early.
+- Streaming and stop-generation affect backend request lifecycle, frontend rendering, and persistence semantics. This should be implemented as a first-class chat capability, not as an afterthought.
+- Permissions are central to the product. Every chat retrieval path must filter knowledge bases before retrieval and log what was used.
+
+## Immediate Next Steps
+
+1. Confirm whether V2 should be implemented in this repository under a new app folder, or whether this repository should become the V2 repository.
+2. Draft V2 architecture and module plan in `development_records/`.
+3. Scaffold V2 backend with the `sys_*` domain model and auth boundary.
+4. Scaffold V2 H5/user frontend around the documented chat APIs and SSE.
+5. Add admin frontend pages according to the prototype document.
+6. Keep old V1 code untouched until a deliberate migration or replacement decision is made.