Compare commits

..

23 Commits

Author SHA1 Message Date
EduBrain Dev
4d5665d369 fix issue 2026-04-14 21:25:41 +08:00
EduBrain Dev
23eabca58d fix issue 2026-04-14 21:17:22 +08:00
EduBrain Dev
397a8d8641 fix issue 2026-04-14 21:10:05 +08:00
EduBrain Dev
dbb3cf622f fix issue 2026-04-14 21:04:00 +08:00
EduBrain Dev
1080bc3882 fix issue 2026-04-14 20:28:55 +08:00
EduBrain Dev
f486a18339 fix issue 2026-04-14 20:13:01 +08:00
EduBrain Dev
8d69533e11 fix issue 2026-04-14 19:55:47 +08:00
EduBrain Dev
db981b88b8 fix issue 2026-04-14 19:53:01 +08:00
EduBrain Dev
015e251727 fix issue 2026-04-14 19:07:39 +08:00
EduBrain Dev
11c5d36dfa fix issue 2026-04-14 18:53:26 +08:00
EduBrain Dev
a4b2b01c71 fix issue 2026-04-14 18:22:01 +08:00
EduBrain Dev
2a6e506416 fix deploy bug 2026-04-14 18:00:18 +08:00
EduBrain Dev
c5f96056fa fix docker issue 2026-04-14 17:25:06 +08:00
EduBrain Dev
4c6a20e5fc chore: 项目更名为 HuiBrain
全局替换 EduBrain -> HuiBrain, edu-brain -> huibrain, edu_brain -> hui_brain, EDUBRAIN -> HUIBRAIN
涉及文件:README.md, pyproject.toml, docker-compose.yml, .env, .env.example,
app/config.py, app/main.py, app/wework_bot.py, app/__init__.py, app/mcp/server.py,
static/index.html, docs/IMAGE_API_GUIDE.md
2026-04-14 15:03:43 +08:00
EduBrain Dev
496e11e26e refactor: 移除 PostgreSQL 支持,简化为纯 SQLite 部署
- config.py: DATABASE_URL 默认值改为 SQLite
- database.py: 移除 PostgreSQL 分支,简化为纯 SQLite
- models/base.py: 移除 pgvector 导入和条件分支
- search_service.py: 移除 _search_postgres 方法
- import_service.py: 移除 pgvector 相关代码
- requirements.txt: 移除 asyncpg/alembic/pgvector 依赖
- pyproject.toml: 同步移除相关依赖
- docker-compose.yml: 移除 db 服务,- 删除 alembic.ini/Dockerfile.db/sql 目录
- README.md: 更新文档,移除 PostgreSQL 相关内容

适合 NAS 等资源受限环境的轻量级部署
2026-04-14 14:50:53 +08:00
EduBrain Dev
f60b45358d feat: OCR 并发识别控制,支持多用户同时上传
- config.py: 新增 OCR_CONCURRENCY 参数(默认1,范围1-10)
- images.py: 用 asyncio.Semaphore 控制 OCR 并发,4个上传端点统一受控
- settings API: 新增 ocr_concurrency 字段读写
- 前端设置页: 新增 OCR 并发数量输入框
- .env / .env.example: 新增 OCR_CONCURRENCY 配置项
- Semaphore 动态重建:修改配置后立即生效,无需重启
2026-04-14 14:11:18 +08:00
EduBrain Dev
8152494fdc fix: 移除 settings.DEBUG 引用,修复删除 DEBUG 字段后的启动报错
- database.py: echo 改为 False
- main.py: 错误响应直接显示异常信息(不再根据 DEBUG 判断)
- 机器人重复图片回复增加 OCR 识别内容展示
2026-04-14 14:03:01 +08:00
EduBrain Dev
2954f3cb5e fix: 机器人收到重复图片时也回复OCR识别结果给用户 2026-04-14 14:00:28 +08:00
EduBrain Dev
36a6debcb3 refactor: 精简环境配置,移除废弃配置项
.env:
- 移除 HOST/PORT/DEBUG(代码未读取,由 uvicorn 命令行控制)
- 移除 WEWORK_BOT_SEARCH_LIMIT(已被 SEARCH_LIMIT 取代)
- 补充硅基流动配置注释

.env.example:
- 同步移除 HOST/PORT/DEBUG/WEWORK_BOT_SEARCH_LIMIT
- 补充 OpenAI 兼容硅基流动的说明
- 更新机器人注释(集成模式自动启动)

config.py:
- 移除 HOST/PORT/DEBUG 字段定义
- 移除 WEWORK_BOT_SEARCH_LIMIT 字段定义

wework_bot.py:
- 改为从 settings 统一读取配置(不再 os.getenv)
- _get_search_limit fallback 改为 settings.SEARCH_LIMIT
- 更新模块文档注释
2026-04-14 13:54:16 +08:00
EduBrain Dev
eb18b5cffc docs: 完善 EDUBRAIN_BASE_URL 配置描述,说明独立部署场景 2026-04-14 13:48:25 +08:00
EduBrain Dev
01ecc3fa6d docs: 更新 README.md 和 IMAGE_API_GUIDE.md
README.md:
- 新增企业微信机器人功能说明(搜索/图片入库/集成启动)
- 新增图片去重和管理功能说明
- 新增可配置参数说明
- 更新 API 概览(删除/去重/机器人管理接口)
- 更新项目结构和技术栈

IMAGE_API_GUIDE.md:
- 新增删除图片接口文档
- 新增一键去重接口文档
- 新增企业微信机器人相关接口文档
- 更新上传接口返回格式(含 duplicate 状态)
- 更新注意事项(去重/删除相关)
2026-04-14 13:44:35 +08:00
EduBrain Dev
e8a861fda2 feat: 企业微信机器人集成到后端服务,网页端可配置和重启
- BotManager: 后端启动时自动检测配置并拉起机器人(后台线程)
- WeComBotService: 新增 start_async() 异步模式,支持嵌入事件循环
- settings API: 增加 bot 配置字段(enabled/bot_id/secret)
- settings API: 增加 bot/status 和 bot/restart 端点
- 前端设置页: 增加企业微信机器人配置卡片(启用开关/ID/Secret)
- 前端设置页: 增加机器人状态徽章和重启按钮
- Secret 脱敏显示,仅展示前4位
2026-04-14 13:42:04 +08:00
EduBrain Dev
8ebf9a4587 feat: 企业微信机器人支持接收图片消息自动OCR识别入库
- 注册 message.image 事件监听
- 使用 SDK download_file() 下载并解密图片
- 调用后端 recognize-direct 接口进行 OCR 识别(自动去重)
- 根据识别结果回复:成功录入/重复内容/识别失败
- 更新欢迎语提示支持图片发送功能
2026-04-14 13:28:36 +08:00
28 changed files with 1080 additions and 1436 deletions

View File

@@ -1,21 +1,22 @@
# ============================================================
# EduBrain - 中文直播教育知识库系统 环境变量配置
# HuiBrain - 中文直播教育知识库系统 环境变量配置
# ============================================================
# 复制此文件为 .env 并根据实际情况修改配置值
# cp .env.example .env
# ============================================================
# ── 数据库 ──
DATABASE_URL=postgresql+asyncpg://postgres:postgres@db:5432/edu_brain
DATABASE_URL=postgresql+asyncpg://postgres:postgres@db:5432/hui_brain
POSTGRES_PASSWORD=postgres
# ── 嵌入模型配置 ──
# 可选值: openai / zhipu / local_bge / dashscope / minimax
# openai 模式也兼容硅基流动等第三方 OpenAI 兼容接口
EMBEDDING_PROVIDER=minimax
EMBEDDING_MODEL=MiniMax-Text-Embedding
EMBEDDING_DIMENSIONS=1024
# ── OpenAI 配置 ──
# ── OpenAI 配置(也兼容硅基流动等第三方接口) ──
OPENAI_API_KEY=sk-your-openai-api-key
OPENAI_BASE_URL=https://api.openai.com/v1
@@ -32,6 +33,7 @@ LOCAL_BGE_MODEL_PATH=
# 可选值: paddleocr / aliyun / tencent / deepseek / auto
# auto 模式: 本地 PaddleOCR 优先,失败后 fallback 到 DeepSeek / 云端 OCR
OCR_PROVIDER=deepseek
OCR_CONCURRENCY=1
# ── 阿里云 OCR 配置 ──
ALIYUN_OCR_ACCESS_KEY=your-aliyun-access-key
@@ -52,7 +54,7 @@ DEEPSEEK_API_KEY=your-deepseek-api-key
DEEPSEEK_BASE_URL=https://api.deepseek.com
DEEPSEEK_OCR_MODEL=deepseek-chat
# ── LLM 配置(用于查询扩展、实体检测) ──
# ── LLM 配置(用于查询扩展、实体检测 ──
# 可选值: minimax / deepseek / openai
LLM_PROVIDER=minimax
@@ -63,11 +65,6 @@ DATA_DIR=/app/data
CHUNK_SIZE=300
CHUNK_OVERLAP=50
# ── 服务配置 ──
HOST=0.0.0.0
PORT=8000
DEBUG=false
# ── CORS 配置(多个来源用逗号分隔) ──
CORS_ORIGINS=["*"]
@@ -76,9 +73,8 @@ APP_PORT=8000
DB_PORT=5432
# ── 企业微信智能机器人WebSocket 长连接) ──
# 作为独立进程运行python -m app.wework_bot
# 后端启动时会自动检测配置并拉起机器人(后台线程)
WEWORK_BOT_ENABLED=false
WEWORK_BOT_ID=your-bot-id
WEWORK_BOT_SECRET=your-bot-secret
WEWORK_BOT_SEARCH_LIMIT=3
EDUBRAIN_BASE_URL=http://localhost:8765
HUIBRAIN_BASE_URL=http://localhost:8765

View File

@@ -2,7 +2,7 @@ FROM python:3.11-slim
# ── 系统依赖PaddleOCR 需要的图形库) ──
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1-mesa-glx \
libgl1 \
libglib2.0-0 \
libsm6 \
libxext6 \

292
README.md
View File

@@ -1,77 +1,129 @@
# EduBrain - 中文直播教育知识库系统
# HuiBrain - 中文直播教育知识库系统
基于 FastAPI 的中文直播教育知识库系统支持直播文字稿导入、OCR 截图识别、AI 语义搜索等功能。
基于 FastAPI 的中文直播教育知识库系统支持直播文字稿导入、OCR 截图识别、AI 语义搜索、企业微信机器人、MCP 协议等功能。
## 功能特性
### 知识库管理
- **多格式导入**: Markdown / 纯文本 / Word 文档(.docx),自动按标题分页
- **批量导入**: 支持多文件批量导入,实时进度显示
- **智能分块**: 按段落边界切分文本,保留重叠区域
- **多格式导入**: Markdown(解析 frontmatter + 按 `##` 标题分页)/ 纯文本 / Word 文档(.docx,按 Heading 分页
- **批量导入**: 支持多文件批量导入、整个目录导入,实时进度显示
- **智能分块**: 按段落边界切分文本,保留重叠区域(默认 chunk_size=300, overlap=50
- **向量嵌入**: 支持 MiniMax / OpenAI / 智谱 / DashScope / 本地 BGE 五种嵌入模型
- **知识导出**: 支持 JSON 和 Markdown 两种格式导出
- **索引重建**: 支持单页面重新生成嵌入向量
### 图片 OCR
- **OCR 识别**: DeepSeek Vision 模型识别图片文字
- **关键词提取**: MiniMax M2.7 自动提取内容标签
- **批量处理**: 支持拖拽上传、选择文件、选择文件夹,按文件名去重
- **BM25 索引**: 内置倒排索引搜索引擎jieba 中文分词
- **多 Provider**: DeepSeek Vision / PaddleOCR / 阿里云 / 腾讯云,支持 `auto` 自动 fallbackPaddleOCR → DeepSeek → 阿里云 → 腾讯云)
- **批量处理**: 支持拖拽上传、选择文件、选择文件夹、服务器路径批量导入
- **上传去重**: OCR 识别后自动检查内容是否已存在,避免重复录入
- **图片管理**: 网页端管理所有图片,支持一键去重、手动删除
- **并发控制**: 通过 asyncio.Semaphore 控制 OCR 并发数量(可配置 1-10
### AI 搜索
- **知识库搜索**: 向量语义搜索 + 全文检索混合排序
- **知识库搜索**: LIKE 关键词搜索 + LLM 查询扩展(生成语义相近的替代表述)
- **图片搜索**: MiniMax M2.7 语义匹配SSE 流式实时返回结果
- **并发池**: 最多 10 个 LLM 请求并发,每批 10 张图片判断
- **并发池**: 最多 10 个 LLM 请求并发,每批数量可配置
- **实体检测**: 自动识别知识点、技巧、人物、概念等实体类型
- **可配置参数**: 搜索返回数量、LLM 批量判断数量、OCR 并发数量均可在网页端动态修改
### 企业微信机器人
- **WebSocket 长连接**: 基于 wecom-aibot-python-sdk无需公网 IP
- **关键词搜索**: 用户发文字 → 清理口语化前缀 → 搜索图片 → 流式回复文本 + 逐张发送图片
- **图片自动入库**: 用户发图片 → 下载解密 → OCR 识别 → 自动去重入库 → 回复结果
- **分片上传**: 支持大图片的三步分片上传init → chunk → finish
- **欢迎语**: 进入会话时自动发送欢迎语
- **集成启动**: 后端启动时自动检测配置并拉起机器人BotManager 后台线程)
- **网页端管理**: 在设置页面配置 Bot ID / Secret支持在线重启
### MCP 协议
- **MCP Server**: 支持 stdio 和 sse 两种传输协议,可被外部 AI Agent 直接调用
- **5 个工具**:
- `search_knowledge`: 语义搜索知识库
- `get_page`: 获取知识页面详情
- `list_courses`: 列出所有课程
- `get_page_chunks`: 获取页面的分块内容
- `search_images`: 根据关键字搜索已 OCR 识别的聊天截图
- **命令行入口**: `huibrain-mcp`(通过 pyproject.toml 定义)
### 其他
- **MCP 协议**: 支持 Model Context Protocol可被 AI Agent 调用
- **Docker 部署**: 完整的 Docker Compose 编排(应用 + PostgreSQL + pgvector
- **运行时配置**: 所有参数均可通过网页端设置 API 动态修改,无需重启服务
- **服务测试**: 网页端支持在线测试嵌入模型、LLM、OCR 服务连通性
- **Docker 部署**: 单容器部署,轻量级
## 技术栈
| 层级 | 技术 |
|------|------|
| 后端框架 | FastAPI + Uvicorn |
| ORM | SQLAlchemy 2.0 (async) |
| 数据库 | PostgreSQL (pgvector) / SQLite |
| 前端 | 纯 HTML + CSS + JS苹果风格 SPA |
| OCR | DeepSeek Vision / PaddleOCR / 阿里云 / 腾讯云 |
| LLM | MiniMax M2.7 / DeepSeek / Qwen3-8B |
| 搜索 | BM25 倒排索引 + 向量语义搜索 |
| 部署 | Docker + Docker Compose |
| 层级 | 技术 | 说明 |
|------|------|------|
| 后端框架 | FastAPI + Uvicorn | 异步 Python Web 框架 |
| ORM | SQLAlchemy 2.0 (async) | 异步 ORM |
| 数据库 | SQLite (aiosqlite) | 轻量级WAL 模式 |
| 配置管理 | pydantic-settings | 环境变量 / .env 文件加载 |
| 前端 | 纯 HTML + CSS + JS | 苹果风格 SPA 单页应用 |
| OCR | DeepSeek Vision / PaddleOCR / 阿里云 / 腾讯云 | 4 种 Provider + auto fallback |
| LLM | MiniMax M2.7 / DeepSeek / OpenAI | 搜索匹配、查询扩展、批量判断 |
| 嵌入模型 | MiniMax / OpenAI / 智谱 / DashScope / 本地 BGE | 5 种嵌入 Provider |
| 搜索 | LIKE 关键词 + LLM 查询扩展 | 知识库搜索;图片搜索走 LLM 语义匹配 |
| 企业微信 | wecom-aibot-python-sdk | WebSocket 长连接模式 |
| MCP | mcp[cli] >= 1.2.0 | Model Context Protocol |
| 部署 | Docker + docker-compose | 单容器部署 |
## 项目结构
```
edu-brain/
huibrain/
├── app/
│ ├── main.py # FastAPI 入口,路由注册,生命周期管理
│ ├── config.py # pydantic-settings 配置管理
│ ├── database.py # 数据库连接PG/SQLite 双后端
│ ├── models/base.py # ORM 模型KnowledgePage, OCRImage 等)
│ ├── schemas/ # Pydantic 请求/响应 Schema
│ ├── api/v1/ # API 路由
│ │ ── pages.py # 知识页面 CRUD
│ │ ├── search.py # 语义搜索
│ │ ├── images.py # 图片 OCR + AI 搜索SSE
│ │ ├── import_export.py # 文件导入导出
│ │ └── settings.py # 系统设置
│ ├── __init__.py # 包初始化,定义版本号
│ ├── main.py # FastAPI 入口路由注册生命周期管理BotManager
│ ├── config.py # pydantic-settings 配置管理(所有环境变量
│ ├── database.py # SQLite 异步数据库连接管理
│ ├── wework_bot.py # 企业微信智能机器人服务WebSocket 长连接)
│ ├── api/
│ │ ── v1/
│ │ ├── pages.py # 知识页面 CRUD列表/详情/创建/更新/删除/重建索引)
│ │ ├── search.py # 语义搜索(向量+全文混合
│ │ ├── images.py # 图片 OCR + AI 搜索SSE 流式)+ 去重 + 删除
│ │ ├── import_export.py # 文件导入(.md/.txt/.docx+ 目录导入 + 导出JSON/Markdown
│ │ └── settings.py # 系统设置 + 机器人管理 + 服务测试 + 统计信息
│ ├── models/
│ │ └── base.py # ORM 模型KnowledgePage, KnowledgeChunk, OCRImage, WebsiteSettings
│ ├── schemas/
│ │ ├── ocr.py # OCR 相关 Schema
│ │ ├── page.py # 页面相关 Schema
│ │ └── search.py # 搜索相关 Schema
│ ├── services/
│ │ ├── ocr_service.py # OCR 识别服务( Provider
│ │ ├── llm_service.py # LLM 服务(标签提取/搜索匹配
│ │ ├── search_engine.py # BM25 倒排索引引擎
│ │ ├── search_service.py # 混合搜索服务
│ │ ├── embedding_service.py # 嵌入模型服务
│ │ ── import_service.py # 文件导入服务
│ └── page_service.py # 页面 CRUD 服务
│ └── mcp/server.py # MCP Server 实现
├── static/index.html # 前端单页应用
├── data/images/ # 图片存储目录
├── .env.example # 环境变量模板
├── requirements.txt # Python 依赖
├── Dockerfile # 应用镜像
├── Dockerfile.db # 数据库镜像PG + pgvector
└── docker-compose.yml # Docker 编排
│ │ ├── ocr_service.py # OCR 识别服务(4 种 Provider + auto fallback
│ │ ├── llm_service.py # LLM 服务(聊天/查询扩展/批量判断
│ │ ├── search_service.py # 知识库搜索服务LIKE 关键词搜索)
│ │ ├── embedding_service.py # 嵌入模型服务5 种 Provider
│ │ ├── import_service.py # 文件导入服务(分块+向量化)
│ │ ── page_service.py # 页面 CRUD 服务
│ └── mcp/
└── server.py # MCP Server 实现5 个工具供 AI Agent 调用)
├── static/
│ └── index.html # 前端单页应用(苹果风格 SPA
├── data/
│ ├── images/ # 图片存储目录
│ └── transcripts/ # 转写文字稿(.docx/.md
├── docs/
│ ├── IMAGE_API_GUIDE.md # 图片 API 集成指南
│ └── 软著材料/ # 软件著作权申请材料
├── .env.example # 环境变量模板
├── .gitignore # Git 忽略规则
├── pyproject.toml # Python 项目元数据 + 构建配置
├── requirements.txt # Python 依赖清单
├── Dockerfile # Docker 镜像构建
└── docker-compose.yml # Docker Compose 编排
```
## 数据模型
| 模型 | 表名 | 说明 |
|------|------|------|
| `KnowledgePage` | knowledge_pages | 知识页面(标题/内容/来源/课程/讲师/日期/向量嵌入) |
| `KnowledgeChunk` | knowledge_chunks | 文本分块(关联页面/分块索引/内容/嵌入向量) |
| `OCRImage` | ocr_images | OCR 图片记录(文件路径/OCR文本/置信度/Provider/状态/文本块) |
| `WebsiteSettings` | website_settings | 网站设置(键值对,单例模式) |
## 快速开始
### 1. 环境准备
@@ -79,12 +131,12 @@ edu-brain/
```bash
# 克隆项目
git clone <repo-url>
cd edu-brain
cd huibrain
# 复制配置文件
cp .env.example .env
# 安装依赖
# 安装依赖Python >= 3.11
pip install -r requirements.txt
```
@@ -94,7 +146,7 @@ pip install -r requirements.txt
```env
# 数据库(默认 SQLite无需额外配置
DATABASE_URL=sqlite+aiosqlite:///edu_brain.db
DATABASE_URL=sqlite+aiosqlite:///data/hui_brain.db
# OCR使用 DeepSeek Vision
OCR_PROVIDER=deepseek
@@ -102,10 +154,15 @@ DEEPSEEK_API_KEY=your-deepseek-api-key
DEEPSEEK_BASE_URL=http://your-ollama-host:11434/v1
DEEPSEEK_OCR_MODEL=deepseek-ocr:latest
# LLM用于标签提取和搜索匹配)
# LLM用于搜索匹配
MINIMAX_API_KEY=your-minimax-api-key
MINIMAX_BASE_URL=https://api.minimaxi.com/v1
MINIMAX_CHAT_MODEL=MiniMax-M2.7
# 企业微信机器人(可选,不配则不启动)
WEWORK_BOT_ENABLED=true
WEWORK_BOT_ID=your-bot-id
WEWORK_BOT_SECRET=your-bot-secret
```
### 3. 启动服务
@@ -114,6 +171,8 @@ MINIMAX_CHAT_MODEL=MiniMax-M2.7
uvicorn app.main:app --host 0.0.0.0 --port 8765
```
启动后会自动检测企业微信机器人配置,如果 `WEWORK_BOT_ENABLED=true``WEWORK_BOT_ID` / `WEWORK_BOT_SECRET` 已配置,机器人会自动在后台线程启动。
访问 http://localhost:8765 即可使用。
### 4. Docker 部署
@@ -122,34 +181,108 @@ uvicorn app.main:app --host 0.0.0.0 --port 8765
docker-compose up -d
```
- 单容器部署,数据持久化通过 `./data:/app/data` 卷挂载
- 默认端口 8000可通过 `APP_PORT` 环境变量调整
- 内置健康检查,每 30 秒检查 `/health` 端点
## 图片搜索工作流
```
用户上传图片
→ DeepSeek Vision OCR 识别文字
MiniMax M2.7 提取关键词标签
→ 存入 SQLite + BM25 索引
OCR 识别文字(DeepSeek Vision / PaddleOCR / 阿里云 / 腾讯云,支持 auto fallback
检查 OCR 内容是否已存在(去重)
→ 存入 SQLite
用户搜索 "孩子不想上学"
从 DB 倒序取图片(每批 10 张
并发池(最多 10 个请求)发给 MiniMax M2.7 判断
LLM 查询扩展(生成语义相近的替代表述
从 DB 倒序取图片(每批 N 张,可配置)
→ 并发池发给 MiniMax M2.7 判断
→ 匹配结果通过 SSE 实时推送到前端
→ 找够指定数量或遍历完 DB 后结束
```
## 企业微信机器人
### 功能
| 场景 | 说明 |
|------|------|
| 单聊发关键词 | 清理口语化前缀 → 搜索图片 → 流式回复文本 + 逐张发送图片 |
| 单聊发图片 | 下载解密 → OCR 识别 → 自动去重入库 → 回复结果 |
| 进入会话 | 自动发送欢迎语 |
### 配置
1. 在企业微信管理后台创建智能机器人,开启 API 模式(长连接)
2. 获取 Bot ID 和 Secret
3.`.env` 中配置或通过网页端设置页面配置
### 注意事项
- 图片消息**仅支持单聊**,群聊中发图片不会触发机器人
- 机器人支持多人同时使用asyncio 异步架构)
- 大图片支持分片上传init → chunk → finish
- 配置修改后可通过网页端「重启机器人」按钮生效
## API 概览
### 知识页面
| 方法 | 路径 | 说明 |
|------|------|------|
| POST | `/api/v1/images/recognize-direct` | 上传并识别图片 |
| POST | `/api/v1/images/batch-recognize` | 批量上传并识别 |
| GET | `/api/v1/images/search` | AI 搜索图片SSE 流式) |
| GET | `/api/v1/images/{id}` | 获取图片识别结果 |
| GET | `/api/v1/pages` | 知识页面列表 |
| GET | `/api/v1/pages` | 知识页面列表(分页) |
| GET | `/api/v1/pages/{id}` | 获取知识页面详情 |
| POST | `/api/v1/pages` | 创建知识页面 |
| POST | `/api/v1/search` | 语义搜索知识库 |
| POST | `/api/v1/import/file` | 导入文件 |
| PUT | `/api/v1/pages/{id}` | 更新知识页面 |
| DELETE | `/api/v1/pages/{id}` | 删除知识页面 |
| POST | `/api/v1/pages/{id}/rebuild-index` | 重建页面嵌入索引 |
### 搜索
| 方法 | 路径 | 说明 |
|------|------|------|
| POST | `/api/v1/search` | 关键词搜索知识库 |
### 图片 OCR
| 方法 | 路径 | 说明 |
|------|------|------|
| POST | `/api/v1/images/recognize` | 上传并识别图片(两步:先上传再识别) |
| POST | `/api/v1/images/recognize-direct` | 一步上传并识别图片 |
| POST | `/api/v1/images/batch-recognize` | 批量上传并识别 |
| POST | `/api/v1/images/import-path` | 服务器路径批量导入 |
| GET | `/api/v1/images/search` | AI 搜索图片SSE 流式) |
| GET | `/api/v1/images` | 获取图片列表(分页) |
| GET | `/api/v1/images/{id}` | 获取图片识别结果 |
| DELETE | `/api/v1/images/{id}` | 删除图片 |
| POST | `/api/v1/images/dedup` | 一键去重 |
### 导入导出
| 方法 | 路径 | 说明 |
|------|------|------|
| POST | `/api/v1/import/file` | 导入文件(.md/.txt/.docx |
| POST | `/api/v1/import/directory` | 导入整个目录 |
| GET | `/api/v1/export` | 导出知识库JSON/Markdown |
### 系统设置
| 方法 | 路径 | 说明 |
|------|------|------|
| GET | `/api/v1/settings` | 获取系统设置 |
| PUT | `/api/v1/settings` | 更新系统设置 |
| GET | `/api/v1/settings/stats` | 获取统计信息 |
| POST | `/api/v1/settings/test/embedding` | 测试嵌入模型连通性 |
| POST | `/api/v1/settings/test/llm` | 测试 LLM 服务连通性 |
| POST | `/api/v1/settings/test/ocr` | 测试 OCR 服务连通性 |
| GET | `/api/v1/settings/bot/status` | 获取机器人状态 |
| POST | `/api/v1/settings/bot/restart` | 重启机器人 |
### 系统
| 方法 | 路径 | 说明 |
|------|------|------|
| GET | `/health` | 健康检查 |
## 配置说明
@@ -157,20 +290,39 @@ docker-compose up -d
| Provider | 说明 | 额外配置 |
|----------|------|----------|
| `deepseek` | DeepSeek Vision推荐 | `DEEPSEEK_API_KEY`, `DEEPSEEK_BASE_URL` |
| `paddleocr` | 本地 PaddleOCR | 无 |
| `deepseek` | DeepSeek Vision推荐,支持 Ollama 自部署 | `DEEPSEEK_API_KEY`, `DEEPSEEK_BASE_URL` |
| `paddleocr` | 本地 PaddleOCR(免费、离线) | 无 |
| `aliyun` | 阿里云 OCR | `ALIYUN_OCR_ACCESS_KEY`, `ALIYUN_OCR_SECRET` |
| `tencent` | 腾讯云 OCR | `TENCENT_OCR_SECRET_ID`, `TENCENT_OCR_SECRET_KEY` |
| `auto` | 自动 fallback | 配置多个 Provider |
| `auto` | 自动 fallbackPaddleOCR → DeepSeek → 阿里云 → 腾讯云) | 配置多个 Provider |
### 嵌入模型
| Provider | 说明 | 额外配置 |
|----------|------|----------|
| `minimax` | MiniMax 嵌入 | `MINIMAX_API_KEY` |
| `openai` | OpenAI 嵌入 | `OPENAI_API_KEY` |
| `openai` | OpenAI 嵌入(兼容硅基流动等第三方接口) | `OPENAI_API_KEY` |
| `zhipu` | 智谱 AI 嵌入 | `ZHIPU_API_KEY` |
| `dashscope` | 阿里云 DashScope | `DASHSCOPE_API_KEY` |
| `local_bge` | 本地 BGE 模型 | `LOCAL_BGE_MODEL_PATH` |
### 网页端可配置参数
| 参数 | 说明 | 默认值 | 范围 |
|------|------|--------|------|
| 搜索返回数量 | 每次搜索最多返回多少条匹配结果 | 3 | 1-10 |
| LLM 批量判断数量 | 每批发给 LLM 判断的图片数量 | 10 | 1-50 |
| OCR 并发数量 | 多用户同时上传时 OCR 并发识别数量 | 1 | 1-10 |
## 架构设计
- **策略模式**: OCR 和嵌入服务均使用抽象基类 + 工厂模式,支持多 Provider 热切换
- **单例模式**: EmbeddingService、OCRService、LLMService 均为类方法单例,支持 `reset()` 重置
- **异步架构**: 全异步FastAPI + SQLAlchemy async + asyncio企业微信机器人在后台线程独立事件循环运行
- **SSE 流式**: 图片搜索使用 Server-Sent Events 实时推送结果,前端可即时展示
- **并发控制**: OCR 并发通过 Semaphore 控制LLM 搜索通过并发池(最多 10 个)控制
- **运行时配置**: 所有参数均可通过网页端设置 API 动态修改,无需重启服务
## 许可证
MIT

View File

@@ -1,63 +0,0 @@
# Alembic 数据库迁移配置
[alembic]
# 迁移脚本目录
script_location = alembic
# 数据库连接 URL通过环境变量覆盖
# 注意Alembic 使用同步驱动,需要将 asyncpg 替换为 psycopg2
sqlalchemy.url = postgresql://postgres:postgres@localhost:5432/edu_brain
# 模板文件
file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# 是否自动应用迁移
# prepend_sys_path = .
# 时区设置
timezone = Asia/Shanghai
# 截断长迁移消息
truncate_slug_length = 40
# 修订 ID 格式
# revision_environment = false
# 输出编码
# output_encoding = utf-8
[post_write_hooks]
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

View File

@@ -1,67 +0,0 @@
"""
Alembic 迁移环境配置
支持异步数据库连接
"""
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
# 导入所有 ORM 模型以确保 Base.metadata 包含完整的表定义
from app.models.base import Base
# Alembic Config 对象
config = context.config
# 设置日志
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# 元数据目标,用于自动生成迁移
target_metadata = Base.metadata
def run_migrations_offline() -> None:
"""
'offline' 模式运行迁移。
只需要 URL不需要 Engine。调用 context.execute() 将迁移
直接发送到数据库。
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""
'online' 模式运行迁移。
创建 Engine 并关联 connection 到 context。
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

View File

@@ -1,26 +0,0 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}

View File

@@ -1,5 +1,5 @@
"""
中文直播教育知识库系统 - edu-brain
中文直播教育知识库系统 - huibrain
"""
__version__ = "1.2.0"

View File

@@ -1,6 +1,6 @@
"""
图片 OCR API 路由
OCR 识别 + BM25 倒排索引搜索
OCR 识别 + AI 搜索
"""
from __future__ import annotations
@@ -21,19 +21,28 @@ from sqlalchemy import select, func, text, delete
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import get_db
from app.database import get_db, async_session_factory
from app.models.base import OCRImage
from app.services.ocr_service import OCRService
from app.services.search_engine import BM25Index, tokenize
from app.services.llm_service import LLMService
logger = logging.getLogger(__name__)
router = APIRouter()
ALLOWED_EXTENSIONS = {".png", ".jpg", ".jpeg", ".bmp", ".webp"}
# OCR 并发控制信号量(动态读取配置)
_ocr_semaphore: asyncio.Semaphore | None = None
# 全局 BM25 索引实例(在 main.py 中初始化)
bm25_index: Optional[BM25Index] = None
def _get_ocr_semaphore() -> asyncio.Semaphore:
"""获取/重建 OCR 并发信号量"""
global _ocr_semaphore
from app.config import settings
concurrency = settings.OCR_CONCURRENCY
if _ocr_semaphore is None or _ocr_semaphore._value != concurrency:
_ocr_semaphore = asyncio.Semaphore(concurrency)
return _ocr_semaphore
ALLOWED_EXTENSIONS = {".png", ".jpg", ".jpeg", ".bmp", ".webp"}
def _check_image_ext(filename: str) -> str:
@@ -49,41 +58,8 @@ def _save_ocr_result(record: OCRImage, ocr_result, db: AsyncSession):
record.confidence = ocr_result.confidence
record.provider = ocr_result.provider
record.status = "completed"
# 保留 keywords如果有
if hasattr(ocr_result, 'keywords') and ocr_result.keywords:
record.tags = json.dumps(ocr_result.keywords, ensure_ascii=False)
def _add_to_index(record: OCRImage):
"""将 OCR 结果添加到 BM25 索引"""
if bm25_index is None or not record.ocr_text:
return
# 索引内容 = OCR 文本(权重最高)
index_text = record.ocr_text
# 如果有 tags/story_summary 也加入索引
if record.tags:
try:
tags = json.loads(record.tags)
if tags:
index_text += "\n" + " ".join(tags)
except Exception:
pass
if record.story_summary:
index_text += "\n" + record.story_summary
bm25_index.add_document(
doc_id=record.id,
text=index_text,
metadata={
"file_path": record.file_path,
"tags": json.loads(record.tags) if record.tags else [],
"story_summary": record.story_summary or "",
"confidence": record.confidence,
"provider": record.provider,
"created_at": str(record.created_at),
},
)
async def _check_ocr_duplicate(ocr_text: str, db: AsyncSession) -> Optional[int]:
"""检查 OCR 文本是否已存在,返回已存在记录的 ID 或 None"""
@@ -126,17 +102,16 @@ async def recognize_image(image_id: int, db: AsyncSession = Depends(get_db)):
await db.flush()
try:
ocr_result = await OCRService.recognize(image_record.file_path)
async with _get_ocr_semaphore():
ocr_result = await OCRService.recognize(image_record.file_path)
_save_ocr_result(image_record, ocr_result, db)
await db.flush()
await db.refresh(image_record)
_add_to_index(image_record)
return {
"id": image_record.id,
"file_path": image_record.file_path,
"ocr_text": image_record.ocr_text,
"tags": json.loads(image_record.tags) if image_record.tags else [],
"confidence": image_record.confidence,
"provider": image_record.provider,
"status": image_record.status,
@@ -158,10 +133,12 @@ async def batch_recognize(files: List[UploadFile], db: AsyncSession = Depends(ge
if not file.filename:
results.append({"filename": "unknown", "status": "error", "message": "文件名为空"})
continue
suffix = _check_image_ext(file.filename)
dest_path = settings.images_dir / file.filename
# 只取文件名,去掉文件夹上传时带上的相对路径
filename = os.path.basename(file.filename)
suffix = _check_image_ext(filename)
dest_path = settings.images_dir / filename
counter = 1
stem = os.path.splitext(file.filename)[0]
stem = os.path.splitext(filename)[0]
while dest_path.exists():
dest_path = settings.images_dir / f"{stem}_{counter}{suffix}"
counter += 1
@@ -171,7 +148,8 @@ async def batch_recognize(files: List[UploadFile], db: AsyncSession = Depends(ge
db.add(image_record)
await db.flush()
try:
ocr_result = await OCRService.recognize(str(dest_path))
async with _get_ocr_semaphore():
ocr_result = await OCRService.recognize(str(dest_path))
# OCR 内容去重
existing_id = await _check_ocr_duplicate(ocr_result.text, db)
if existing_id is not None:
@@ -193,12 +171,10 @@ async def batch_recognize(files: List[UploadFile], db: AsyncSession = Depends(ge
_save_ocr_result(image_record, ocr_result, db)
await db.flush()
await db.refresh(image_record)
_add_to_index(image_record)
results.append({
"id": image_record.id, "filename": file.filename,
"file_path": str(dest_path), "status": "completed",
"ocr_text": ocr_result.text,
"tags": json.loads(image_record.tags) if image_record.tags else [],
"confidence": ocr_result.confidence, "provider": ocr_result.provider,
"block_count": len(ocr_result.blocks),
})
@@ -239,7 +215,8 @@ async def import_from_paths(data: PathImportRequest, db: AsyncSession = Depends(
db.add(image_record)
await db.flush()
try:
ocr_result = await OCRService.recognize(file_path)
async with _get_ocr_semaphore():
ocr_result = await OCRService.recognize(file_path)
# OCR 内容去重
existing_id = await _check_ocr_duplicate(ocr_result.text, db)
if existing_id is not None:
@@ -250,8 +227,7 @@ async def import_from_paths(data: PathImportRequest, db: AsyncSession = Depends(
_save_ocr_result(image_record, ocr_result, db)
await db.flush()
await db.refresh(image_record)
_add_to_index(image_record)
results.append({"id": image_record.id, "filename": os.path.basename(file_path), "file_path": file_path, "status": "completed", "ocr_text": ocr_result.text, "tags": json.loads(image_record.tags) if image_record.tags else [], "confidence": ocr_result.confidence, "provider": ocr_result.provider})
results.append({"id": image_record.id, "filename": os.path.basename(file_path), "file_path": file_path, "status": "completed", "ocr_text": ocr_result.text, "confidence": ocr_result.confidence, "provider": ocr_result.provider})
except Exception as e:
image_record.status = "failed"
image_record.error_message = str(e)
@@ -266,10 +242,12 @@ async def import_from_paths(data: PathImportRequest, db: AsyncSession = Depends(
async def recognize_direct(file: UploadFile, db: AsyncSession = Depends(get_db)):
if not file.filename:
raise HTTPException(status_code=400, detail="文件名不能为空")
suffix = _check_image_ext(file.filename)
dest_path = settings.images_dir / file.filename
# 只取文件名,去掉文件夹上传时带上的相对路径(如 xx/01.jpg -> 01.jpg
filename = os.path.basename(file.filename)
suffix = _check_image_ext(filename)
dest_path = settings.images_dir / filename
counter = 1
stem = os.path.splitext(file.filename)[0]
stem = os.path.splitext(filename)[0]
while dest_path.exists():
dest_path = settings.images_dir / f"{stem}_{counter}{suffix}"
counter += 1
@@ -279,7 +257,8 @@ async def recognize_direct(file: UploadFile, db: AsyncSession = Depends(get_db))
db.add(image_record)
await db.flush()
try:
ocr_result = await OCRService.recognize(str(dest_path))
async with _get_ocr_semaphore():
ocr_result = await OCRService.recognize(str(dest_path))
# OCR 内容去重
existing_id = await _check_ocr_duplicate(ocr_result.text, db)
if existing_id is not None:
@@ -298,12 +277,10 @@ async def recognize_direct(file: UploadFile, db: AsyncSession = Depends(get_db))
_save_ocr_result(image_record, ocr_result, db)
await db.flush()
await db.refresh(image_record)
_add_to_index(image_record)
return {
"id": image_record.id, "file_path": image_record.file_path,
"original_filename": file.filename,
"ocr_text": image_record.ocr_text,
"tags": json.loads(image_record.tags) if image_record.tags else [],
"confidence": image_record.confidence, "provider": image_record.provider,
"status": image_record.status,
"blocks": [{"text": b.text, "bbox": b.bbox, "confidence": b.confidence} for b in ocr_result.blocks],
@@ -322,15 +299,11 @@ async def search_images(
keyword: str,
limit: int = Query(5, ge=1, le=100),
sort: str = Query("time_desc"),
db: AsyncSession = Depends(get_db),
):
"""
AI 搜索图片SSE 流式返回)。
1. 从 DB 取所有图片,按指定排序
2. 每批 5 条发给 Qwen3-8B 判断
3. 并发池最多 10 个请求
4. 匹配结果实时 SSE 推送
5. 找够 limit 个或遍历完 DB 后结束
注意:不使用 Depends(get_db),因为 StreamingResponse 的生命周期
超出 request scope需要在生成器内部自行管理 session。
"""
async def event_stream():
@@ -343,115 +316,116 @@ async def search_images(
# 发送开始事件
yield f"data: {json.dumps({'type': 'start', 'keyword': keyword, 'limit': limit}, ensure_ascii=False)}\n\n"
# 先获取总记录数
count_result = await db.execute(
select(func.count(OCRImage.id)).where(OCRImage.status == "completed")
)
total_records = count_result.scalar() or 0
# 在生成器内部自行管理 session避免 get_db 提前关闭
async with async_session_factory() as db:
# 先获取总记录数
count_result = await db.execute(
select(func.count(OCRImage.id)).where(OCRImage.status == "completed")
)
total_records = count_result.scalar() or 0
if total_records == 0:
yield f"data: {json.dumps({'type': 'done', 'total_found': 0, 'total_checked': 0}, ensure_ascii=False)}\n\n"
return
if total_records == 0:
yield f"data: {json.dumps({'type': 'done', 'total_found': 0, 'total_checked': 0}, ensure_ascii=False)}\n\n"
return
# 排序
order_clause = OCRImage.created_at.desc()
if sort == "time_asc":
order_clause = OCRImage.created_at.asc()
elif sort == "random":
order_clause = text("RANDOM()")
# 排序
order_clause = OCRImage.created_at.desc()
if sort == "time_asc":
order_clause = OCRImage.created_at.asc()
elif sort == "random":
order_clause = text("RANDOM()")
# 用于收集结果的队列
result_queue = asyncio.Queue()
checked_count = 0
# 用于收集结果的队列
result_queue = asyncio.Queue()
checked_count = 0
async def process_batch(batch_items: list, batch_offset: int):
"""处理一批图片:发给 LLM 判断"""
nonlocal checked_count
async with semaphore:
articles = [{"id": item.id, "text": item.ocr_text or ""} for item in batch_items]
matched_ids = await LLMService.judge_batch(keyword, articles)
async def process_batch(batch_items: list, batch_offset: int):
"""处理一批图片:发给 LLM 判断"""
nonlocal checked_count
async with semaphore:
articles = [{"id": item.id, "text": item.ocr_text or ""} for item in batch_items]
matched_ids = await LLMService.judge_batch(keyword, articles)
checked_count += len(batch_items)
checked_count += len(batch_items)
# 发送进度
progress_data = {
"type": "progress",
"checked": checked_count,
"total": total_records,
"found": found,
}
await result_queue.put(progress_data)
# 发送进度
progress_data = {
"type": "progress",
"checked": checked_count,
"total": total_records,
"found": found,
}
await result_queue.put(progress_data)
# 如果有匹配且未达上限,发送结果
for match_id in matched_ids:
# 如果有匹配且未达上限,发送结果
for match_id in matched_ids:
if found >= limit:
break
item = next((i for i in batch_items if i.id == match_id), None)
if item:
file_name = item.file_path.split('/')[-1] if item.file_path else ''
# 读取图片并转为 base64用于发送到企业微信等
image_base64 = ""
if file_name:
img_path = settings.images_dir / file_name
if img_path.exists():
image_base64 = base64.b64encode(img_path.read_bytes()).decode()
result_data = {
"type": "result",
"id": item.id,
"file_path": item.file_path,
"image_url": file_name,
"image_base64": image_base64,
"ocr_text": item.ocr_text,
"ocr_text_preview": (item.ocr_text[:150] + "...") if item.ocr_text and len(item.ocr_text) > 150 else item.ocr_text,
"confidence": item.confidence,
"provider": item.provider,
"created_at": str(item.created_at),
}
await result_queue.put(result_data)
# 启动所有批次的异步任务
tasks = []
while offset < total_records:
query = select(OCRImage).where(OCRImage.status == "completed").order_by(order_clause).offset(offset).limit(batch_size)
result = await db.execute(query)
batch_items = result.scalars().all()
if not batch_items:
break
tasks.append(asyncio.create_task(process_batch(batch_items, offset)))
offset += batch_size
# 等待所有任务完成
async def wait_all():
await asyncio.gather(*tasks)
await result_queue.put({"type": "all_done"})
asyncio.create_task(wait_all())
# 从队列读取结果并 yield
while True:
try:
data = await asyncio.wait_for(result_queue.get(), timeout=300)
except asyncio.TimeoutError:
break
if data.get("type") == "all_done":
break
if data.get("type") == "result":
found += 1
yield f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
if found >= limit:
break
item = next((i for i in batch_items if i.id == match_id), None)
if item:
file_name = item.file_path.split('/')[-1] if item.file_path else ''
# 读取图片并转为 base64用于发送到企业微信等
image_base64 = ""
if file_name:
img_path = settings.images_dir / file_name
if img_path.exists():
image_base64 = base64.b64encode(img_path.read_bytes()).decode()
result_data = {
"type": "result",
"id": item.id,
"file_path": item.file_path,
"image_url": file_name,
"image_base64": image_base64,
"ocr_text": item.ocr_text,
"ocr_text_preview": (item.ocr_text[:150] + "...") if item.ocr_text and len(item.ocr_text) > 150 else item.ocr_text,
"tags": json.loads(item.tags) if item.tags else [],
"confidence": item.confidence,
"provider": item.provider,
"created_at": str(item.created_at),
}
await result_queue.put(result_data)
# 找够了,发送结束事件
yield f"data: {json.dumps({'type': 'done', 'total_found': found, 'total_checked': checked_count}, ensure_ascii=False)}\n\n"
return
elif data.get("type") == "progress":
yield f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
# 启动所有批次的异步任务
tasks = []
while offset < total_records:
query = select(OCRImage).where(OCRImage.status == "completed").order_by(order_clause).offset(offset).limit(batch_size)
result = await db.execute(query)
batch_items = result.scalars().all()
if not batch_items:
break
tasks.append(asyncio.create_task(process_batch(batch_items, offset)))
offset += batch_size
# 等待所有任务完成
async def wait_all():
await asyncio.gather(*tasks)
await result_queue.put({"type": "all_done"})
asyncio.create_task(wait_all())
# 从队列读取结果并 yield
while True:
try:
data = await asyncio.wait_for(result_queue.get(), timeout=300)
except asyncio.TimeoutError:
break
if data.get("type") == "all_done":
break
if data.get("type") == "result":
found += 1
yield f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
if found >= limit:
# 找够了,发送结束事件
yield f"data: {json.dumps({'type': 'done', 'total_found': found, 'total_checked': checked_count}, ensure_ascii=False)}\n\n"
return
elif data.get("type") == "progress":
yield f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
# 发送结束事件
yield f"data: {json.dumps({'type': 'done', 'total_found': found, 'total_checked': checked_count}, ensure_ascii=False)}\n\n"
# 发送结束事件
yield f"data: {json.dumps({'type': 'done', 'total_found': found, 'total_checked': checked_count}, ensure_ascii=False)}\n\n"
return StreamingResponse(
event_stream(),
@@ -472,9 +446,6 @@ async def delete_image(image_id: int, db: AsyncSession = Depends(get_db)):
image_record = result.scalar_one_or_none()
if image_record is None:
raise HTTPException(status_code=404, detail="图片记录不存在")
# 从 BM25 索引移除
if bm25_index is not None:
bm25_index.remove_document(image_record.id)
# 删除磁盘上的图片文件
try:
os.unlink(image_record.file_path)
@@ -544,9 +515,6 @@ async def dedup_images(db: AsyncSession = Depends(get_db)):
duplicates_found += len(group) - 1
kept += 1
for record in group[1:]:
# 从 BM25 索引移除
if bm25_index is not None:
bm25_index.remove_document(record.id)
# 删除磁盘上的图片文件
try:
os.unlink(record.file_path)

View File

@@ -12,6 +12,15 @@ from app.config import settings
router = APIRouter()
def _mask_secret(value: str | None, visible: int = 4) -> str | None:
"""对敏感字符串脱敏,保留前 visible 个字符"""
if not value:
return None
if len(value) <= visible:
return "****"
return value[:visible] + "****"
class SettingsResponse(BaseModel):
"""设置响应(隐藏敏感信息的中间位)"""
embedding_provider: str
@@ -29,6 +38,7 @@ class SettingsResponse(BaseModel):
chunk_overlap: int
search_limit: int
judge_batch_size: int
ocr_concurrency: int
# API Key 只返回是否已配置(不返回实际值)
has_minimax_key: bool
has_deepseek_key: bool
@@ -37,6 +47,10 @@ class SettingsResponse(BaseModel):
has_dashscope_key: bool
has_aliyun_ocr: bool
has_tencent_ocr: bool
# 企业微信机器人
wework_bot_enabled: bool
wework_bot_id: str | None
wework_bot_secret_masked: str | None
@router.get("", response_model=SettingsResponse, summary="获取当前设置")
@@ -58,6 +72,7 @@ async def get_settings():
chunk_overlap=settings.CHUNK_OVERLAP,
search_limit=settings.SEARCH_LIMIT,
judge_batch_size=settings.JUDGE_BATCH_SIZE,
ocr_concurrency=settings.OCR_CONCURRENCY,
has_minimax_key=bool(settings.MINIMAX_API_KEY),
has_deepseek_key=bool(settings.DEEPSEEK_API_KEY),
has_openai_key=bool(settings.OPENAI_API_KEY),
@@ -65,6 +80,9 @@ async def get_settings():
has_dashscope_key=bool(settings.DASHSCOPE_API_KEY),
has_aliyun_ocr=bool(settings.ALIYUN_OCR_ACCESS_KEY),
has_tencent_ocr=bool(settings.TENCENT_OCR_SECRET_ID),
wework_bot_enabled=settings.WEWORK_BOT_ENABLED,
wework_bot_id=settings.WEWORK_BOT_ID,
wework_bot_secret_masked=_mask_secret(settings.WEWORK_BOT_SECRET),
)
@@ -94,6 +112,11 @@ class SettingsUpdate(BaseModel):
chunk_overlap: int | None = None
search_limit: int | None = None
judge_batch_size: int | None = None
ocr_concurrency: int | None = None
# 企业微信机器人
wework_bot_enabled: bool | None = None
wework_bot_id: str | None = None
wework_bot_secret: str | None = None
@router.put("", summary="更新设置")
@@ -139,6 +162,10 @@ async def update_settings(data: SettingsUpdate):
"chunk_overlap": "CHUNK_OVERLAP",
"search_limit": "SEARCH_LIMIT",
"judge_batch_size": "JUDGE_BATCH_SIZE",
"ocr_concurrency": "OCR_CONCURRENCY",
"wework_bot_enabled": "WEWORK_BOT_ENABLED",
"wework_bot_id": "WEWORK_BOT_ID",
"wework_bot_secret": "WEWORK_BOT_SECRET",
}
for api_field, config_field in field_map.items():
@@ -221,3 +248,18 @@ async def test_ocr():
return {"success": True, "message": f"OCR 提供商 {provider.name} 已就绪"}
except Exception as e:
return {"success": False, "message": f"OCR 测试失败: {str(e)}"}
@router.post("/bot/restart", summary="重启企业微信机器人")
async def restart_bot():
"""重启企业微信机器人服务"""
from app.main import bot_manager
result = await bot_manager.restart()
return result
@router.get("/bot/status", summary="获取企业微信机器人状态")
async def get_bot_status():
"""获取企业微信机器人运行状态"""
from app.main import bot_manager
return bot_manager.get_status()

View File

@@ -46,8 +46,8 @@ class Settings(BaseSettings):
# ──────────────────────────── 数据库 ────────────────────────────
DATABASE_URL: str = Field(
default="postgresql+asyncpg://postgres:postgres@db:5432/edu_brain",
description="PostgreSQL 异步连接字符串",
default="sqlite+aiosqlite:///./data/hui_brain.db",
description="SQLite 数据库连接字符串",
)
# ──────────────────────────── 嵌入模型 ────────────────────────────
@@ -153,10 +153,8 @@ class Settings(BaseSettings):
SEARCH_LIMIT: int = Field(default=3, description="图片搜索默认返回数量(1-10)")
JUDGE_BATCH_SIZE: int = Field(default=10, description="每次LLM判断的文章数量(1-50)")
# ──────────────────────────── 服务端口 ────────────────────────────
HOST: str = Field(default="0.0.0.0", description="服务监听地址")
PORT: int = Field(default=8000, description="服务监听端口")
DEBUG: bool = Field(default=False, description="调试模式")
# ──────────────────────────── OCR ────────────────────────────
OCR_CONCURRENCY: int = Field(default=1, ge=1, le=10, description="OCR 并发识别数量(1-10),多用户同时上传时控制并发")
# ──────────────────────────── CORS ────────────────────────────
CORS_ORIGINS: list[str] = Field(
@@ -177,15 +175,9 @@ class Settings(BaseSettings):
default=None,
description="企业微信智能机器人 Secret",
)
WEWORK_BOT_SEARCH_LIMIT: int = Field(
default=3,
ge=1,
le=10,
description="企业微信机器人搜索返回图片数量上限",
)
EDUBRAIN_BASE_URL: str = Field(
HUIBRAIN_BASE_URL: str = Field(
default="http://localhost:8765",
description="EduBrain 服务地址(供企业微信机器人调用搜索接口",
description="HuiBrain 后端服务地址企业微信机器人通过此地址调用后端 API搜索/OCR识别等。当机器人与后端部署在同一台机器时使用默认值 http://localhost:8765若机器人独立部署到其他服务器需改为后端的实际访问地址如 http://192.168.1.100:8765",
)
# ──────────────────────────── 属性方法 ────────────────────────────

View File

@@ -1,12 +1,13 @@
"""
数据库连接模块
支持 PostgreSQL生产和 SQLite本地开发两种后端
数据库连接管理
SQLite 数据库,支持异步操作。
"""
from __future__ import annotations
from collections.abc import AsyncGenerator
from pathlib import Path
from typing import AsyncGenerator
from sqlalchemy import text
from sqlalchemy.ext.asyncio import (
@@ -14,35 +15,27 @@ from sqlalchemy.ext.asyncio import (
async_sessionmaker,
create_async_engine,
)
from sqlalchemy.pool import NullPool
from app.config import settings
# ──────────────────────────── 判断数据库类型 ────────────────────────────
# ──────────────────────────── 数据库引擎 ────────────────────────────
DB_URL = settings.DATABASE_URL
IS_SQLITE = DB_URL.startswith("sqlite")
# ──────────────────────────── 异步引擎 ────────────────────────────
# 解析 SQLite 文件路径
sqlite_path = DB_URL.replace("sqlite+aiosqlite:///", "")
if not sqlite_path.startswith("/"):
sqlite_path = str(Path(__file__).parent.parent / sqlite_path)
if IS_SQLITE:
# SQLite 配置
sqlite_path = DB_URL.replace("sqlite+aiosqlite:///", "")
if not sqlite_path.startswith("/"):
sqlite_path = str(Path(__file__).parent.parent / sqlite_path)
# 确保数据目录存在
Path(sqlite_path).parent.mkdir(parents=True, exist_ok=True)
engine = create_async_engine(
f"sqlite+aiosqlite:///{sqlite_path}",
echo=settings.DEBUG,
)
else:
# PostgreSQL 配置
engine = create_async_engine(
DB_URL,
echo=settings.DEBUG,
poolclass=NullPool,
pool_pre_ping=True,
)
engine = create_async_engine(
f"sqlite+aiosqlite:///{sqlite_path}",
echo=False,
pool_size=1,
max_overflow=0,
)
# ──────────────────────────── 会话工厂 ────────────────────────────
@@ -59,6 +52,7 @@ async_session_factory = async_sessionmaker(
async def get_db() -> AsyncGenerator[AsyncSession, None]:
"""
FastAPI 依赖项:获取异步数据库会话。
请求正常结束后自动 commit异常时自动 rollback。
"""
async with async_session_factory() as session:
try:
@@ -69,39 +63,18 @@ async def get_db() -> AsyncGenerator[AsyncSession, None]:
raise
# ──────────────────────────── 生命周期辅助 ────────────────────────────
async def close_db() -> None:
"""关闭数据库连接池"""
await engine.dispose()
async def init_db() -> None:
"""初始化数据库:建表"""
from app.models.base import Base
async with engine.begin() as conn:
if IS_SQLITE:
# SQLite 启用 WAL 模式和外键约束
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA foreign_keys=ON"))
# SQLite 启用 WAL 模式和外键约束
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA foreign_keys=ON"))
# 创建所有表
await conn.run_sync(Base.metadata.create_all)
if not IS_SQLITE:
# PostgreSQL 需要创建扩展
async with engine.begin() as conn:
try:
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS vector"))
except Exception:
pass
try:
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS zhparser"))
except Exception:
pass
try:
await conn.execute(text(
"CREATE TEXT SEARCH CONFIGURATION IF NOT EXISTS chinese_zh (PARSER = zhparser)"
))
except Exception:
pass
async def close_db() -> None:
"""关闭数据库引擎"""
await engine.dispose()

View File

@@ -5,25 +5,110 @@ FastAPI 应用入口
from __future__ import annotations
import asyncio
import json
import logging
import os
import threading
from contextlib import asynccontextmanager
from pathlib import Path
from typing import AsyncGenerator
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse, RedirectResponse
from fastapi.responses import FileResponse, JSONResponse, RedirectResponse, Response
from app.config import settings
from app.database import close_db, init_db
_logger = logging.getLogger(__name__)
class BotManager:
"""管理企业微信机器人的生命周期(在后台线程中运行)"""
def __init__(self):
self._thread: threading.Thread | None = None
self._running = False
def start(self):
"""在后台线程中启动机器人"""
if self._running:
_logger.info("企业微信机器人已在运行")
return
if not settings.WEWORK_BOT_ENABLED or not settings.WEWORK_BOT_ID or not settings.WEWORK_BOT_SECRET:
_logger.info("企业微信机器人未配置,跳过启动")
return
self._thread = threading.Thread(
target=self._run_bot, daemon=True, name="wework-bot"
)
self._thread.start()
_logger.info("企业微信机器人已在后台线程启动")
def _run_bot(self):
"""在线程中运行 bot需要新建事件循环"""
self._running = True
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
loop.run_until_complete(self._run_bot_async())
except Exception as e:
_logger.error("企业微信机器人线程异常退出: %s", e, exc_info=True)
finally:
loop.close()
self._running = False
async def _run_bot_async(self):
"""异步运行 bot"""
from app.wework_bot import WeComBotService
service = WeComBotService()
await service.start_async()
async def restart(self) -> dict:
"""重启机器人(先停后启)"""
was_running = self._running
# 停止旧的(设置 enabled=False 让旧线程自然退出)
old_id = settings.WEWORK_BOT_ID
old_secret = settings.WEWORK_BOT_SECRET
old_enabled = settings.WEWORK_BOT_ENABLED
if not old_enabled or not old_id or not old_secret:
return {"success": False, "message": "请先配置机器人 ID 和 Secret 并启用"}
# 等待旧线程结束
if self._thread and self._thread.is_alive():
# SDK 的 run() 会阻塞,无法从外部优雅停止
# 通过设置 enabled=False 让下次检查时退出
_logger.info("等待旧机器人线程结束...")
# 无法强制停止,提示用户
if was_running:
return {"success": False, "message": "旧机器人正在运行请稍后重试SDK 不支持强制停止)"}
# 启动新的
self.start()
return {"success": True, "message": "机器人已启动"}
def get_status(self) -> dict:
"""获取机器人状态"""
configured = bool(settings.WEWORK_BOT_ID and settings.WEWORK_BOT_SECRET)
return {
"enabled": settings.WEWORK_BOT_ENABLED,
"configured": configured,
"running": self._running,
"bot_id": settings.WEWORK_BOT_ID,
}
# 全局 bot 管理器
bot_manager = BotManager()
@asynccontextmanager
async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
"""
应用生命周期管理:
- 启动时:初始化数据目录、验证数据库连接、预热嵌入服务、初始化 BM25 索引
- 启动时:初始化数据目录、验证数据库连接、预热嵌入服务
- 关闭时:释放数据库连接
"""
_logger = logging.getLogger(__name__)
@@ -32,6 +117,9 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
settings.ensure_dirs()
await init_db()
# 启动企业微信机器人(后台线程)
bot_manager.start()
# 预热嵌入服务(懒加载,首次调用时初始化)
try:
from app.services.embedding_service import EmbeddingService
@@ -39,53 +127,6 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
except Exception as exc:
_logger.warning("嵌入服务预热失败(将在首次使用时重试): %s", exc)
# ── 初始化 BM25 索引 ──
try:
from app.services.search_engine import BM25Index
import app.api.v1.images as images_module
_images_bm25 = BM25Index()
# 加载数据库中已有的 OCR 记录到索引
from app.database import async_session_factory
from app.models.base import OCRImage
from sqlalchemy import select
async with async_session_factory() as session:
result = await session.execute(
select(OCRImage).where(OCRImage.status == "completed")
)
records = result.scalars().all()
for record in records:
if record.ocr_text:
index_text = record.ocr_text
if record.tags:
try:
tags = json.loads(record.tags)
if tags:
index_text += "\n" + " ".join(tags)
except Exception:
pass
if record.story_summary:
index_text += "\n" + record.story_summary
_images_bm25.add_document(
doc_id=record.id,
text=index_text,
metadata={
"file_path": record.file_path,
"tags": json.loads(record.tags) if record.tags else [],
"story_summary": record.story_summary or "",
"confidence": record.confidence,
"provider": record.provider,
"created_at": str(record.created_at),
},
)
images_module.bm25_index = _images_bm25
_logger.info("BM25 索引加载完成: %d 条记录", _images_bm25.size)
except Exception as exc:
_logger.warning("BM25 索引初始化失败: %s", exc)
yield
# ── 关闭 ──
@@ -95,7 +136,7 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
# ──────────────────────────── 创建应用 ────────────────────────────
app = FastAPI(
title="EduBrain - 中文直播教育知识库",
title="HuiBrain - 中文直播教育知识库",
description="基于向量检索的中文直播教育知识库系统支持直播文字稿导入、OCR 截图识别、语义搜索等功能。",
version="1.2.0",
lifespan=lifespan,
@@ -121,7 +162,7 @@ async def global_exception_handler(request: Request, exc: Exception) -> JSONResp
logger.error("未处理异常: %s", exc, exc_info=True)
return JSONResponse(
status_code=500,
content={"detail": "服务器内部错误", "message": str(exc) if settings.DEBUG else "请稍后重试"},
content={"detail": "服务器内部错误", "message": str(exc)},
)
@@ -153,7 +194,11 @@ async def health_check():
@app.get("/data/images/{image_name}", tags=["前端"])
async def serve_image(image_name: str):
"""访问 images 目录下的图片文件(用于前端预览)"""
img_path = settings.images_dir / image_name
# 防止路径遍历攻击
safe_name = os.path.basename(image_name)
if safe_name != image_name or ".." in image_name:
return JSONResponse(status_code=400, content={"detail": "非法文件名"})
img_path = settings.images_dir / safe_name
if not img_path.exists():
return JSONResponse(status_code=404, content={"detail": "图片不存在"})
return FileResponse(str(img_path))
@@ -161,11 +206,20 @@ async def serve_image(image_name: str):
# ──────────────────────────── 前端页面 ────────────────────────────
@app.get("/favicon.ico", include_in_schema=False)
async def favicon():
"""返回空响应,避免浏览器 404 报错"""
return Response(status_code=204)
@app.get("/", tags=["前端"])
async def index():
"""返回前端管理页面"""
static_dir = Path(__file__).parent.parent / "static"
index_path = static_dir / "index.html"
if index_path.exists():
return FileResponse(str(index_path))
return FileResponse(
str(index_path),
headers={"Cache-Control": "no-cache, no-store, must-revalidate"},
)
return RedirectResponse(url="/docs")

View File

@@ -21,7 +21,7 @@ logger = logging.getLogger(__name__)
# ──────────────────────────── 创建 MCP Server ────────────────────────────
mcp = FastMCP(
name="EduBrain-Knowledge",
name="HuiBrain-Knowledge",
instructions=(
"中文直播教育知识库助手。你可以使用以下工具来搜索和查询知识库中的内容:\n"
"- search_knowledge: 语义搜索知识库,支持按课程、讲师、日期过滤\n"

View File

@@ -1,26 +1,18 @@
"""
SQLAlchemy 基础模型 + 通用 Mixin
提供 DeclarativeBase、时间戳 Mixin 和公共序列化方法
兼容 PostgreSQLpgvector和 SQLite本地开发
ORM 模型定义
SQLite 数据库模型,使用 Text 存储 JSON 格式的向量。
"""
from __future__ import annotations
import json
from datetime import datetime
from typing import Any, Dict
from sqlalchemy import DateTime, Float, Integer, String, Text, func
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
from app.database import IS_SQLITE
# SQLite 下不使用 pgvector用 Text 存储向量JSON 格式)
if IS_SQLITE:
VectorType = Text
else:
from pgvector.sqlalchemy import Vector
VectorType = Vector
class Base(DeclarativeBase):
"""所有 ORM 模型的基类"""
@@ -28,96 +20,96 @@ class Base(DeclarativeBase):
class TimestampMixin:
"""
时间戳 Mixin为模型自动添加 created_at / updated_at 字段。
"""
"""时间戳 Mixin"""
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
comment="创建时间",
DateTime, default=func.now(), comment="创建时间"
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
comment="更新时间",
DateTime, default=func.now(), onupdate=func.now(), comment="更新时间"
)
def to_dict(self) -> Dict[str, Any]:
"""将模型实例转换为字典"""
result: Dict[str, Any] = {}
for column in self.__table__.columns: # type: ignore[attr-defined]
value = getattr(self, column.name, None)
if isinstance(value, datetime):
value = value.isoformat()
elif hasattr(value, "tolist"):
value = value.tolist()
result[column.name] = value
"""将模型为字典"""
result = {}
for col in self.__table__.columns:
val = getattr(self, col.name)
if isinstance(val, datetime):
val = val.isoformat()
result[col.name] = val
return result
# ──────────────────────────── 业务模型 ────────────────────────────
class KnowledgePage(Base, TimestampMixin):
"""知识页面模型"""
"""知识页面"""
__tablename__ = "knowledge_pages"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
title: Mapped[str] = mapped_column(String(500), nullable=False, comment="页面标题")
content: Mapped[str] = mapped_column(Text, nullable=False, comment="页面正文内容")
source_file: Mapped[str | None] = mapped_column(String(500), nullable=True, comment="来源文件")
content: Mapped[str] = mapped_column(Text, nullable=False, comment="页面内容")
source_file: Mapped[str | None] = mapped_column(String(500), nullable=True, comment="来源文件路径")
page_number: Mapped[int | None] = mapped_column(Integer, nullable=True, comment="原文件页码")
course_name: Mapped[str | None] = mapped_column(String(200), nullable=True, comment="课程名称")
teacher_name: Mapped[str | None] = mapped_column(String(100), nullable=True, comment="讲师名称")
live_date: Mapped[str | None] = mapped_column(String(20), nullable=True, comment="直播日期")
page_number: Mapped[int | None] = mapped_column(Integer, nullable=True, comment="原始页码")
metadata_json: Mapped[str | None] = mapped_column(Text, nullable=True, comment="额外元数据")
metadata_json: Mapped[str | None] = mapped_column(Text, nullable=True, comment="额外元数据JSON")
embedding: Mapped[str | None] = mapped_column(Text, nullable=True, comment="向量嵌入JSON")
class KnowledgeChunk(Base, TimestampMixin):
"""知识分块模型"""
"""知识页面分块"""
__tablename__ = "knowledge_chunks"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
page_id: Mapped[int] = mapped_column(Integer, nullable=False, index=True, comment="关联页面 ID")
chunk_index: Mapped[int] = mapped_column(Integer, nullable=False, comment="分块序号")
content: Mapped[str] = mapped_column(Text, nullable=False, comment="分块文本内容")
embedding: Mapped[list | None] = mapped_column(
VectorType(768) if not IS_SQLITE else Text,
nullable=True,
comment="嵌入向量",
)
search_vector: Mapped[Any | None] = mapped_column(
Text, nullable=True, comment="全文搜索向量(仅 PG",
)
def to_dict(self) -> Dict[str, Any]:
result = super().to_dict()
result.pop("embedding", None)
result.pop("search_vector", None)
return result
page_id: Mapped[int] = mapped_column(Integer, nullable=False, comment="关联页面 ID")
chunk_index: Mapped[int] = mapped_column(Integer, nullable=False, comment="分块索引")
content: Mapped[str] = mapped_column(Text, nullable=False, comment="分块内容")
embedding: Mapped[str | None] = mapped_column(Text, nullable=True, comment="向量嵌入JSON")
class OCRImage(Base, TimestampMixin):
"""OCR 图片记录模型"""
"""OCR 图片记录"""
__tablename__ = "ocr_images"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
file_path: Mapped[str] = mapped_column(String(500), nullable=False, comment="图片文件路径")
ocr_text: Mapped[str | None] = mapped_column(Text, nullable=True, comment="OCR 识别全文")
confidence: Mapped[float | None] = mapped_column(Float, nullable=True, comment="平均置信度")
original_filename: Mapped[str | None] = mapped_column(String(200), nullable=True, comment="原始文件名")
ocr_text: Mapped[str | None] = mapped_column(Text, nullable=True, comment="OCR 识别文本")
confidence: Mapped[float | None] = mapped_column(Float, nullable=True, comment="OCR 置信度")
provider: Mapped[str | None] = mapped_column(String(50), nullable=True, comment="OCR 提供商")
status: Mapped[str] = mapped_column(String(20), default="pending", comment="处理状态")
status: Mapped[str] = mapped_column(
String(20), default="pending", comment="状态: pending/processing/completed/failed"
)
blocks: Mapped[str | None] = mapped_column(Text, nullable=True, comment="OCR 文本块JSON")
error_message: Mapped[str | None] = mapped_column(Text, nullable=True, comment="错误信息")
tags: Mapped[str | None] = mapped_column(Text, nullable=True, comment="LLM 提取的标签JSON 数组格式")
story_summary: Mapped[str | None] = mapped_column(Text, nullable=True, comment="Qwen3-8B 提炼的故事摘要")
@property
def blocks_list(self) -> list[dict]:
if self.blocks is None:
return []
return json.loads(self.blocks)
@blocks_list.setter
def blocks_list(self, value: list[dict]):
self.blocks = json.dumps(value) if value else None
class WebsiteSettings(Base):
"""网站设置(单例)"""
__tablename__ = "website_settings"
key: Mapped[str] = mapped_column(String(100), primary_key=True, comment="设置键")
value: Mapped[str | None] = mapped_column(Text, nullable=True, comment="设置值JSON")
updated_at: Mapped[datetime] = mapped_column(
DateTime, default=func.now(), onupdate=func.now(), comment="更新时间"
)
def to_dict(self) -> Dict[str, Any]:
result = super().to_dict()
# 将 tags 从 JSON 字符串解析为列表返回
if result.get("tags") and isinstance(result["tags"], str):
try:
import json
result["tags"] = json.loads(result["tags"])
except (json.JSONDecodeError, TypeError):
pass
"""将模型转为字典"""
result = {}
for col in self.__table__.columns:
val = getattr(self, col.name)
if isinstance(val, datetime):
val = val.isoformat()
result[col.name] = val
return result

View File

@@ -14,7 +14,6 @@ from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import IS_SQLITE
logger = logging.getLogger(__name__)
@@ -343,21 +342,13 @@ class ImportService:
# 批量插入
for idx, (chunk_text, embedding) in enumerate(zip(chunks, embeddings)):
if IS_SQLITE:
# SQLite: 向量存为 JSON 文本
import json as _json
embedding_str = _json.dumps(embedding) if embedding else None
sql = text("""
INSERT INTO knowledge_chunks (page_id, chunk_index, content, embedding)
VALUES (:page_id, :chunk_index, :content, :embedding)
""")
else:
# PostgreSQL: 使用 pgvector 类型
embedding_str = "[" + ",".join(str(x) for x in embedding) + "]" if embedding else None
sql = text("""
INSERT INTO knowledge_chunks (page_id, chunk_index, content, embedding)
VALUES (:page_id, :chunk_index, :content, :embedding::vector)
""")
# SQLite: 向量存为 JSON 文本
import json as _json
embedding_str = _json.dumps(embedding) if embedding else None
sql = text("""
INSERT INTO knowledge_chunks (page_id, chunk_index, content, embedding)
VALUES (:page_id, :chunk_index, :content, :embedding)
""")
await self.db.execute(sql, {
"page_id": page_id,
"chunk_index": idx,
@@ -365,14 +356,6 @@ class ImportService:
"embedding": embedding_str,
})
# 更新全文搜索向量(仅 PostgreSQL
if not IS_SQLITE:
await self.db.execute(text("""
UPDATE knowledge_chunks
SET search_vector = to_tsvector('chinese_zh', content)
WHERE page_id = :page_id
"""), {"page_id": page_id})
await self.db.flush()
return len(chunks)

View File

@@ -236,158 +236,6 @@ class LLMService:
return []
@classmethod
async def extract_tags(cls, text: str) -> List[str]:
"""
从文本中提取语义标签。
用于图片 OCR 内容的标签化和搜索查询的标签化。
Args:
text: 待提取标签的文本
Returns:
标签列表,如 ["教育", "高考数学", "二次方程", "解题技巧"]
"""
messages = [
{
"role": "system",
"content": (
"你是一个内容标签提取助手。用户会给你一段文字内容,"
"你需要从中提取 3-10 个能概括内容主题的标签。\n"
"标签要求:\n"
"1. 用简短的词语2-6个字\n"
"2. 涵盖主题、场景、情感、关键实体等维度\n"
"3. 考虑同义词和近义词(如'不想工作'也打上'躺平'标签)\n"
"4. 只输出 JSON 数组,不要其他文字\n"
'例如:["高考数学", "二次方程", "韦达定理", "解题技巧"]'
),
},
{
"role": "user",
"content": f"请从以下内容中提取标签:\n\n{text[:2000]}",
},
]
try:
result = await cls.chat(messages, temperature=0.3, max_tokens=256)
result = result.strip()
if result.startswith("```"):
result = result.split("\n", 1)[-1]
if result.endswith("```"):
result = result[:-3]
result = result.strip()
tags = json.loads(result)
if isinstance(tags, list):
# 去重、清洗
seen = set()
clean = []
for t in tags:
t = str(t).strip()
if 1 <= len(t) <= 20 and t not in seen:
seen.add(t)
clean.append(t)
return clean[:15]
except Exception as exc:
logger.warning("标签提取失败: %s", exc)
return []
@classmethod
async def extract_search_tags(cls, query: str) -> List[str]:
"""
从用户搜索查询中提取标签,用于匹配数据库中存储的标签。
和 extract_tags 类似,但更侧重于提取搜索意图相关的标签。
"""
messages = [
{
"role": "system",
"content": (
"你是一个搜索意图分析助手。用户会给你一个搜索查询,"
"你需要提取 3-8 个可能匹配的标签词。\n"
"标签要求:\n"
"1. 用简短的词语2-6个字\n"
"2. 包含同义词和近义词\n"
"3. 包含上义词和下义词\n"
"4. 只输出 JSON 数组\n"
'例如:用户搜"孩子躺平",输出 ["躺平", "不上班", "年轻人", "就业", "摆烂", "啃老"]'
),
},
{
"role": "user",
"content": f"请从以下搜索查询中提取匹配标签:{query}",
},
]
try:
result = await cls.chat(messages, temperature=0.3, max_tokens=256)
result = result.strip()
if result.startswith("```"):
result = result.split("\n", 1)[-1]
if result.endswith("```"):
result = result[:-3]
result = result.strip()
tags = json.loads(result)
if isinstance(tags, list):
seen = set()
clean = []
for t in tags:
t = str(t).strip()
if 1 <= len(t) <= 20 and t not in seen:
seen.add(t)
clean.append(t)
return clean[:10]
except Exception as exc:
logger.warning("搜索标签提取失败: %s", exc)
# 降级:直接用原始查询词作为标签
return [query.strip()] if query.strip() else []
@classmethod
async def summarize_story(cls, text: str) -> str:
"""
用 Qwen3-8B 提炼图片 OCR 文本的故事内容。
非思考模式,直接输出。
"""
messages = [
{
"role": "system",
"content": (
"你是一个内容提炼助手。用户会给你一段从图片中识别出的文字内容,"
"你需要提炼出其中讲述的故事或事件。\n"
"要求:\n"
"1. 用简洁的自然语言描述图片内容讲述的故事或事件\n"
"2. 保留关键人物、事件、情感、场景等核心信息\n"
"3. 50-200字\n"
"4. 直接输出提炼内容,不要加前缀"
),
},
{
"role": "user",
"content": f"/no_think\n请提炼以下内容的故事:\n\n{text[:3000]}",
},
]
try:
from openai import AsyncOpenAI
client = AsyncOpenAI(
api_key=settings.OPENAI_API_KEY,
base_url=settings.OPENAI_BASE_URL,
)
response = await client.chat.completions.create(
model="Qwen/Qwen3-8B",
messages=messages,
temperature=0.3,
max_tokens=512,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
content = response.choices[0].message.content or ""
# 清理可能的 /no_think 回显
content = content.replace("/no_think", "").strip()
return content
except Exception as exc:
logger.warning("故事提炼失败: %s", exc)
return ""
@classmethod
async def judge_batch(cls, query: str, articles: List[dict]) -> List[int]:
"""
@@ -438,37 +286,41 @@ class LLMService:
try:
from openai import AsyncOpenAI
client = AsyncOpenAI(
api_key=settings.MINIMAX_API_KEY,
base_url=settings.MINIMAX_BASE_URL,
http_client=httpx.AsyncClient(timeout=60.0),
)
response = await client.chat.completions.create(
model=settings.MINIMAX_CHAT_MODEL,
messages=messages,
temperature=0.0,
max_tokens=512,
stream=False,
extra_body={"reasoning_split": True},
)
content = response.choices[0].message.content or ""
content = cls._clean_reasoning_content(content)
http_client = httpx.AsyncClient(timeout=60.0)
try:
client = AsyncOpenAI(
api_key=settings.MINIMAX_API_KEY,
base_url=settings.MINIMAX_BASE_URL,
http_client=http_client,
)
response = await client.chat.completions.create(
model=settings.MINIMAX_CHAT_MODEL,
messages=messages,
temperature=0.0,
max_tokens=512,
stream=False,
extra_body={"reasoning_split": True},
)
content = response.choices[0].message.content or ""
content = cls._clean_reasoning_content(content)
# 解析返回的编号
numbers = re.findall(r'\d+', content)
# 解析返回的编号
numbers = re.findall(r'\d+', content)
if not numbers:
return []
if not numbers:
return []
matched_ids = []
for num_str in numbers:
num = int(num_str)
if num == 0:
continue # 0 表示都不匹配
if 1 <= num <= len(articles):
matched_ids.append(articles[num - 1]["id"])
matched_ids = []
for num_str in numbers:
num = int(num_str)
if num == 0:
continue # 0 表示都不匹配
if 1 <= num <= len(articles):
matched_ids.append(articles[num - 1]["id"])
return matched_ids
return matched_ids
finally:
await http_client.aclose()
except Exception as exc:
logger.warning("LLM 批量判断失败: %s", exc)
return []

View File

@@ -102,7 +102,7 @@ class PaddleOCRProvider(OCRProviderBase):
async def recognize(self, image_path: str) -> OCRResult:
"""使用 PaddleOCR 识别图片"""
self._init_ocr()
loop = asyncio.get_event_loop()
loop = asyncio.get_running_loop()
def _run_ocr():
result = self._ocr.ocr(image_path, cls=True)
@@ -164,7 +164,7 @@ class AliyunOCRProvider(OCRProviderBase):
import asyncio
loop = asyncio.get_event_loop()
loop = asyncio.get_running_loop()
def _call_api():
from alibabacloud_tea_openapi.models import Config
@@ -249,7 +249,7 @@ class TencentOCRProvider(OCRProviderBase):
import base64
import json
loop = asyncio.get_event_loop()
loop = asyncio.get_running_loop()
def _call_api():
from tencentcloud.common import credential

View File

@@ -181,22 +181,15 @@ class PageService:
if embedding:
embedding_str = "[" + ",".join(str(x) for x in embedding) + "]"
await self.db.execute(text("""
INSERT INTO knowledge_chunks (page_id, chunk_index, content, embedding)
VALUES (:page_id, :chunk_index, :content, :embedding::vector)
"""), {
await self.db.execute(text(
"INSERT INTO knowledge_chunks (page_id, chunk_index, content, embedding) "
"VALUES (:page_id, :chunk_index, :content, :embedding)"
), {
"page_id": page_id,
"chunk_index": idx,
"content": chunk_text,
"embedding": embedding_str,
})
# 更新全文搜索向量
await self.db.execute(text("""
UPDATE knowledge_chunks
SET search_vector = to_tsvector('chinese_zh', content)
WHERE page_id = :page_id
"""), {"page_id": page_id})
await self.db.flush()
return {"page_id": page_id, "chunks": len(chunks)}

View File

@@ -1,185 +0,0 @@
"""
BM25 倒排索引搜索引擎
用于图片 OCR 内容的全文搜索
参考 Meilisearch / Whoosh 设计
"""
from __future__ import annotations
import re
import math
import logging
from collections import Counter, defaultdict
from typing import List, Dict, Optional, Tuple
logger = logging.getLogger(__name__)
# 尝试导入 jieba失败则用简单的字符分割
try:
import jieba
jieba.setLogLevel(logging.WARNING)
def tokenize(text: str) -> List[str]:
"""中文分词"""
return [w for w in jieba.cut_for_search(text) if len(w.strip()) > 1]
except ImportError:
logger.warning("jieba 未安装使用简单分词pip install jieba 获得更好效果)")
def tokenize(text: str) -> List[str]:
"""简单分词:按空格和标点分割"""
return [w for w in re.split(r'[\s,.\-;:!?"\'/\\|()\[\]{}<>,。;:!?""''、()【】]+', text) if len(w.strip()) > 1]
class BM25Index:
"""
BM25 倒排索引
- 支持增量添加文档
- 支持删除文档
- 支持中文分词jieba
- BM25 评分排序
"""
def __init__(self, k1: float = 1.5, b: float = 0.75):
"""
Args:
k1: 词频饱和参数(默认 1.5
b: 文档长度归一化参数(默认 0.75
"""
self.k1 = k1
self.b = b
# 倒排索引term -> {doc_id: tf}
self.inverted_index: Dict[str, Dict[int, int]] = defaultdict(lambda: defaultdict(int))
# 文档信息
self.doc_lengths: Dict[int, int] = {} # doc_id -> 文档长度(词数)
self.doc_count: int = 0
self.avg_doc_length: float = 0.0
# 文档存储(存完整数据)
self.documents: Dict[int, dict] = {}
# IDF 缓存
self._idf_cache: Dict[str, float] = {}
def add_document(self, doc_id: int, text: str, metadata: Optional[dict] = None):
"""
添加文档到索引
Args:
doc_id: 文档 ID
text: 待索引的文本内容
metadata: 附加元数据(如 file_path, tags 等)
"""
# 如果已存在,先删除
if doc_id in self.documents:
self.remove_document(doc_id)
tokens = tokenize(text)
self.doc_lengths[doc_id] = len(tokens)
self.documents[doc_id] = {
"text": text,
"tokens": tokens,
"metadata": metadata or {},
}
# 构建倒排索引
tf = Counter(tokens)
for term, count in tf.items():
self.inverted_index[term][doc_id] = count
self.doc_count += 1
self._update_avg_doc_length()
self._idf_cache.clear() # 文档数变了IDF 需要重算
def remove_document(self, doc_id: int):
"""从索引中删除文档"""
if doc_id not in self.documents:
return
# 从倒排索引中移除
tokens = self.documents[doc_id].get("tokens", [])
for term in set(tokens):
if term in self.inverted_index and doc_id in self.inverted_index[term]:
del self.inverted_index[term][doc_id]
if not self.inverted_index[term]:
del self.inverted_index[term]
del self.documents[doc_id]
del self.doc_lengths[doc_id]
self.doc_count -= 1
self._update_avg_doc_length()
self._idf_cache.clear()
def _update_avg_doc_length(self):
"""更新平均文档长度"""
if self.doc_count > 0:
self.avg_doc_length = sum(self.doc_lengths.values()) / self.doc_count
else:
self.avg_doc_length = 0.0
def _idf(self, term: str) -> float:
"""
计算逆文档频率 IDF
IDF(t) = ln((N - df(t) + 0.5) / (df(t) + 0.5) + 1)
其中 N 是文档总数df(t) 是包含 term 的文档数
"""
if term in self._idf_cache:
return self._idf_cache[term]
df = len(self.inverted_index.get(term, {}))
idf = math.log((self.doc_count - df + 0.5) / (df + 0.5) + 1)
self._idf_cache[term] = idf
return idf
def search(self, query: str, top_k: int = 10) -> List[Tuple[int, float]]:
"""
BM25 搜索
Args:
query: 搜索查询(会自动分词)
top_k: 返回前 K 个结果
Returns:
[(doc_id, score), ...] 按 BM25 分数降序排列
"""
if self.doc_count == 0 or self.avg_doc_length == 0:
return []
query_tokens = tokenize(query)
if not query_tokens:
return []
scores: Dict[int, float] = defaultdict(float)
for term in query_tokens:
idf = self._idf(term)
postings = self.inverted_index.get(term, {})
for doc_id, tf in postings.items():
dl = self.doc_lengths[doc_id]
# BM25 公式
numerator = tf * (self.k1 + 1)
denominator = tf + self.k1 * (1 - self.b + self.b * dl / self.avg_doc_length)
scores[doc_id] += idf * numerator / denominator
# 按分数降序排序
ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)
return ranked[:top_k]
def get_document(self, doc_id: int) -> Optional[dict]:
"""获取文档完整数据"""
return self.documents.get(doc_id)
def clear(self):
"""清空索引"""
self.inverted_index.clear()
self.doc_lengths.clear()
self.documents.clear()
self._idf_cache.clear()
self.doc_count = 0
self.avg_doc_length = 0.0
@property
def size(self) -> int:
"""索引中的文档数量"""
return self.doc_count

View File

@@ -1,20 +1,18 @@
"""
语义搜索服务
支持 PostgreSQL向量+全文混合)和 SQLiteLIKE 关键词搜索)
SQLite 数据库,使用 LIKE 关键词搜索。
"""
from __future__ import annotations
import json
import logging
import time
from typing import List, Optional
from typing import List
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import IS_SQLITE
from app.schemas.search import SearchRequest, SearchResult, SearchResponse
logger = logging.getLogger(__name__)
@@ -28,21 +26,10 @@ class SearchService:
async def search(self, request: SearchRequest) -> SearchResponse:
"""
执行搜索
- PostgreSQL: 向量搜索 + 中文全文搜索混合排序
- SQLite: LIKE 关键词搜索(无向量能力)
执行搜索:使用 LIKE 关键词搜索
"""
start_time = time.time()
if IS_SQLITE:
return await self._search_sqlite(request, start_time)
else:
return await self._search_postgres(request, start_time)
# ──────────────────────────── SQLite 搜索 ────────────────────────────
async def _search_sqlite(self, request: SearchRequest, start_time: float) -> SearchResponse:
"""SQLite 模式:使用 LIKE 关键词搜索"""
# LLM 查询扩展(可选)
expanded_queries = [request.query]
try:
@@ -109,178 +96,3 @@ class SearchService:
results=results,
elapsed_ms=round(elapsed_ms, 2),
)
# ──────────────────────────── PostgreSQL 搜索 ────────────────────────────
async def _search_postgres(self, request: SearchRequest, start_time: float) -> SearchResponse:
"""PostgreSQL 模式:向量搜索 + 中文全文搜索混合"""
# LLM 查询扩展
expanded_queries = [request.query]
if request.use_fulltext:
try:
from app.services.llm_service import LLMService
expanded_queries = await LLMService.expand_query(request.query)
except Exception as exc:
logger.debug("查询扩展跳过: %s", exc)
# 生成查询向量
from app.services.embedding_service import EmbeddingService
all_query_embeddings = await EmbeddingService.embed_batch(expanded_queries)
# 构建过滤条件
where_clauses = []
params: dict = {}
if request.course_name:
where_clauses.append("kp.course_name = :course_name")
params["course_name"] = request.course_name
if request.teacher_name:
where_clauses.append("kp.teacher_name = :teacher_name")
params["teacher_name"] = request.teacher_name
if request.live_date_from:
where_clauses.append("kp.live_date >= :live_date_from")
params["live_date_from"] = request.live_date_from
if request.live_date_to:
where_clauses.append("kp.live_date <= :live_date_to")
params["live_date_to"] = request.live_date_to
where_sql = ""
if where_clauses:
where_sql = "AND " + " AND ".join(where_clauses)
# 向量搜索
vector_results = []
params["limit"] = request.top_k * 2
for q_embedding in all_query_embeddings:
embedding_str = "[" + ",".join(str(x) for x in q_embedding) + "]"
vector_sql = f"""
SELECT
kc.id AS chunk_id,
kp.id AS page_id,
kp.title AS page_title,
kc.content,
1 - (kc.embedding <=> :embedding::vector) AS vector_score,
0.0 AS text_score,
kp.course_name,
kp.teacher_name,
kp.live_date
FROM knowledge_chunks kc
JOIN knowledge_pages kp ON kc.page_id = kp.id
WHERE kc.embedding IS NOT NULL
{where_sql}
ORDER BY kc.embedding <=> :embedding::vector
LIMIT :limit
"""
v_params = {**params, "embedding": embedding_str}
result = await self.db.execute(text(vector_sql), v_params)
vector_results.extend(result.fetchall())
# 全文搜索
text_results = []
if request.use_fulltext:
for q in expanded_queries:
clean_query = q.replace("'", "''")
fulltext_sql = f"""
SELECT
kc.id AS chunk_id,
kp.id AS page_id,
kp.title AS page_title,
kc.content,
0.0 AS vector_score,
ts_rank_cd(kc.search_vector, to_tsquery('chinese_zh', :query)) AS text_score,
kp.course_name,
kp.teacher_name,
kp.live_date,
ts_headline('chinese_zh', kc.content, to_tsquery('chinese_zh', :query),
'MaxWords=50, MinWords=20, ShortWord=2') AS highlight
FROM knowledge_chunks kc
JOIN knowledge_pages kp ON kc.page_id = kp.id
WHERE kc.search_vector @@ to_tsquery('chinese_zh', :query)
{where_sql}
ORDER BY text_score DESC
LIMIT :limit
"""
ft_params = {**params, "query": clean_query}
ft_result = await self.db.execute(text(fulltext_sql), ft_params)
text_results.extend(ft_result.fetchall())
# 合并结果
merged: dict[int, dict] = {}
for row in vector_results:
chunk_id = row.chunk_id
if chunk_id not in merged:
merged[chunk_id] = {
"chunk_id": chunk_id,
"page_id": row.page_id,
"page_title": row.page_title,
"content": row.content,
"vector_score": row.vector_score,
"text_score": 0.0,
"course_name": row.course_name,
"teacher_name": row.teacher_name,
"live_date": row.live_date,
"highlight": None,
}
else:
merged[chunk_id]["vector_score"] = max(
merged[chunk_id]["vector_score"], row.vector_score
)
for row in text_results:
chunk_id = row.chunk_id
if chunk_id not in merged:
merged[chunk_id] = {
"chunk_id": chunk_id,
"page_id": row.page_id,
"page_title": row.page_title,
"content": row.content,
"vector_score": 0.0,
"text_score": row.text_score or 0.0,
"course_name": row.course_name,
"teacher_name": row.teacher_name,
"live_date": row.live_date,
"highlight": row.highlight,
}
else:
merged[chunk_id]["text_score"] = max(
merged[chunk_id]["text_score"], row.text_score or 0.0
)
if row.highlight:
merged[chunk_id]["highlight"] = row.highlight
# 计算综合得分
max_text_score = max(
(r["text_score"] for r in merged.values()), default=1.0
) or 1.0
scored_results = []
for item in merged.values():
normalized_text = item["text_score"] / max_text_score if max_text_score > 0 else 0
combined_score = 0.7 * item["vector_score"] + 0.3 * normalized_text
if combined_score >= request.threshold:
scored_results.append(
SearchResult(
chunk_id=item["chunk_id"],
page_id=item["page_id"],
page_title=item["page_title"],
content=item["content"],
score=round(combined_score, 4),
course_name=item["course_name"],
teacher_name=item["teacher_name"],
live_date=item["live_date"],
highlight=item["highlight"],
)
)
scored_results.sort(key=lambda x: x.score, reverse=True)
scored_results = scored_results[:request.top_k]
elapsed_ms = (time.time() - start_time) * 1000
return SearchResponse(
query=request.query,
total=len(scored_results),
results=scored_results,
elapsed_ms=round(elapsed_ms, 2),
)

View File

@@ -4,7 +4,9 @@
使用官方 SDK (wecom-aibot-python-sdk) 的 WSClient 类,
封装了连接管理、心跳、断线重连等能力。
作为独立进程运行python -m app.wework_bot
支持两种运行方式:
1. 集成模式后端启动时自动拉起BotManager
2. 独立进程python -m app.wework_bot
"""
from __future__ import annotations
@@ -16,6 +18,10 @@ import logging
import math
import os
import re
import tempfile
import time
import asyncio
from datetime import datetime
import httpx
from aibot import WSClient, WSClientOptions, generate_req_id
@@ -24,6 +30,8 @@ from dotenv import load_dotenv
# 加载 .env 文件(独立运行时需要)
load_dotenv()
from app.config import settings
logger = logging.getLogger("wework_bot")
# 分片上传参数(参考官方文档)
@@ -34,17 +42,14 @@ class WeComBotService:
"""企业微信智能机器人服务WebSocket 长连接模式)"""
def __init__(self):
self.bot_id = os.getenv("WEWORK_BOT_ID", "")
self.bot_secret = os.getenv("WEWORK_BOT_SECRET", "")
self.edubrain_base_url = os.getenv(
"EDUBRAIN_BASE_URL", "http://localhost:8765"
)
self.search_limit = int(os.getenv("WEWORK_BOT_SEARCH_LIMIT", "3"))
self.enabled = os.getenv("WEWORK_BOT_ENABLED", "false").lower() == "true"
self.bot_id = settings.WEWORK_BOT_ID or ""
self.bot_secret = settings.WEWORK_BOT_SECRET or ""
self.huibrain_base_url = settings.HUIBRAIN_BASE_URL
self.enabled = settings.WEWORK_BOT_ENABLED
self.ws_client: WSClient | None = None
def start(self):
"""启动机器人服务"""
"""启动机器人服务(阻塞模式,用于独立进程运行)"""
if not self.enabled:
logger.info("企业微信机器人未启用 (WEWORK_BOT_ENABLED=false)")
return
@@ -60,16 +65,42 @@ class WeComBotService:
)
)
# 注册事件
self._register_events()
logger.info("启动企业微信智能机器人...")
self.ws_client.run()
async def start_async(self):
"""启动机器人服务(异步模式,用于嵌入到其他事件循环)"""
if not self.enabled:
logger.info("企业微信机器人未启用 (WEWORK_BOT_ENABLED=false)")
return
if not self.bot_id or not self.bot_secret:
logger.error("WEWORK_BOT_ID 或 WEWORK_BOT_SECRET 未配置")
return
self.ws_client = WSClient(
WSClientOptions(
bot_id=self.bot_id,
secret=self.bot_secret,
)
)
self._register_events()
logger.info("启动企业微信智能机器人(异步模式)...")
await self.ws_client.connect()
# 永远等待,保持连接
await asyncio.Future()
def _register_events(self):
"""注册所有事件处理器"""
self.ws_client.on("authenticated", self._on_authenticated)
self.ws_client.on("message.text", self._on_text_message)
self.ws_client.on("message.image", self._on_image_message)
self.ws_client.on("event.enter_chat", self._on_enter_chat)
self.ws_client.on("disconnected", self._on_disconnected)
self.ws_client.on("error", self._on_error)
logger.info("启动企业微信智能机器人...")
self.ws_client.run()
# ──────────────────────────── 事件处理 ────────────────────────────
def _on_authenticated(self):
@@ -83,7 +114,7 @@ class WeComBotService:
{
"msgtype": "text",
"text": {
"content": "你好!我是知识库图片搜索助手,发送关键词即可搜索相关图片"
"content": "你好!我是知识库助手,支持以下功能:\n\n1. 发送关键词搜索相关图片\n2. 发送图片 → 自动识别并录入知识库"
},
},
)
@@ -121,7 +152,7 @@ class WeComBotService:
False,
)
# 调用 EduBrain 搜索
# 调用 HuiBrain 搜索
try:
results = await self._search_images(keyword)
@@ -194,6 +225,104 @@ class WeComBotService:
"""连接断开回调"""
logger.warning("企业微信连接断开: %s", reason)
async def _on_image_message(self, frame: dict):
"""收到图片消息 -> 下载解密 -> 调用OCR识别 -> 回复结果"""
body = frame.get("body", {})
image_info = body.get("image", {})
image_url = image_info.get("url", "")
aes_key = image_info.get("aeskey", "")
if not image_url:
logger.warning("收到图片消息但无下载地址")
return
stream_id = generate_req_id("stream")
await self.ws_client.reply_stream(
frame, stream_id, "收到图片,正在识别...", False
)
try:
# 1. 下载并解密图片
image_data, original_filename = await self.ws_client.download_file(
image_url, aes_key
)
if not image_data:
await self.ws_client.reply_stream(
frame, stream_id, "图片下载失败,请重试", True
)
return
logger.info(
"图片下载成功: filename=%s, size=%d bytes",
original_filename, len(image_data),
)
# 2. 生成保存文件名(避免重名)
ext = os.path.splitext(original_filename or "image.jpg")[1] or ".jpg"
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
save_filename = f"wecom_{timestamp}{ext}"
# 3. 调用后端 recognize-direct 接口
async with httpx.AsyncClient(timeout=120.0) as client:
resp = await client.post(
f"{self.huibrain_base_url}/api/v1/images/recognize-direct",
files={"file": (save_filename, image_data, "image/jpeg")},
)
if resp.status_code != 200:
logger.error("OCR 识别失败: status=%d, body=%s", resp.status_code, resp.text)
await self.ws_client.reply_stream(
frame, stream_id, "图片识别失败,请稍后重试", True
)
return
result = resp.json()
# 4. 根据识别结果回复
status = result.get("status", "")
if status == "duplicate":
dup_id = result.get("duplicate_of", "?")
ocr_text = result.get("ocr_text", "")
preview = ocr_text[:200].replace("\n", " ") if ocr_text else "无识别内容"
await self.ws_client.reply_stream(
frame,
stream_id,
f"🔄 该图片内容已存在于知识库中ID: {dup_id}),无需重复录入。\n\n识别内容:\n{preview}{'...' if len(ocr_text) > 200 else ''}",
True,
)
elif status == "completed":
ocr_text = result.get("ocr_text", "")
preview = ocr_text[:200].replace("\n", " ") if ocr_text else "无识别内容"
record_id = result.get("id", "?")
await self.ws_client.reply_stream(
frame,
stream_id,
f"✅ 图片已录入知识库ID: {record_id}\n\n识别内容预览:\n{preview}{'...' if len(ocr_text) > 200 else ''}",
True,
)
elif status == "failed":
error_msg = result.get("message", result.get("detail", "未知错误"))
await self.ws_client.reply_stream(
frame,
stream_id,
f"图片识别失败:{error_msg}",
True,
)
else:
await self.ws_client.reply_stream(
frame,
stream_id,
f"图片处理结果:{json.dumps(result, ensure_ascii=False)}",
True,
)
except Exception as e:
logger.error("处理图片消息失败: %s", e, exc_info=True)
await self.ws_client.reply_stream(
frame, stream_id, f"处理图片时出错:{e}", True
)
def _on_error(self, error):
"""错误回调"""
logger.error("企业微信机器人错误: %s", error)
@@ -301,17 +430,19 @@ class WeComBotService:
"""从后端 API 实时获取当前 search_limit 配置"""
try:
async with httpx.AsyncClient(timeout=5.0) as client:
resp = await client.get(f"{self.edubrain_base_url}/api/v1/settings")
resp = await client.get(f"{self.huibrain_base_url}/api/v1/settings")
if resp.status_code == 200:
data = resp.json()
return data.get("search_limit", self.search_limit)
limit = data.get("search_limit")
if limit:
return limit
except Exception as e:
logger.warning("获取 search_limit 失败,使用默认值: %s", e)
return self.search_limit
return settings.SEARCH_LIMIT
async def _search_images(self, keyword: str) -> list:
"""
调用 EduBrain 搜索接口SSE 流式),收集所有匹配结果。
调用 HuiBrain 搜索接口SSE 流式),收集所有匹配结果。
Args:
keyword: 搜索关键词
@@ -320,7 +451,7 @@ class WeComBotService:
匹配结果列表
"""
results = []
url = f"{self.edubrain_base_url}/api/v1/images/search"
url = f"{self.huibrain_base_url}/api/v1/images/search"
limit = await self._get_search_limit()
async with httpx.AsyncClient(timeout=300.0) as client:

View File

@@ -1,55 +1,21 @@
version: "3.8"
services:
# ── 应用服务 ──
app:
build:
context: .
dockerfile: Dockerfile
container_name: edu-brain-app
container_name: huibrain-app
restart: unless-stopped
ports:
- "${APP_PORT:-8000}:8000"
volumes:
- ./data:/app/data # 数据目录挂载
- ./data:/app/data
env_file:
- .env
depends_on:
db:
condition: service_healthy
environment:
- DATABASE_URL=postgresql+asyncpg://postgres:${POSTGRES_PASSWORD:-postgres}@db:5432/edu_brain
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
# ── PostgreSQL 数据库pgvector + zhparser 自定义镜像) ──
db:
build:
context: .
dockerfile: Dockerfile.db
container_name: edu-brain-db
restart: unless-stopped
ports:
- "${DB_PORT:-5432}:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./sql/init.sql:/docker-entrypoint-initdb.d/01-init.sql:ro
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
POSTGRES_DB: edu_brain
POSTGRES_HOST_AUTH_METHOD: scram-sha-256
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d edu_brain"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
volumes:
postgres_data:
driver: local

View File

@@ -1,6 +1,6 @@
# EduBrain 图片功能集成指南
# HuiBrain 图片功能集成指南
> 本文档介绍如何在 ADK Agent 项目中集成 EduBrain 的图片 OCR 和搜索功能。
> 本文档介绍如何在 ADK Agent 项目中集成 HuiBrain 的图片 OCR 和搜索功能。
## 服务地址
@@ -20,8 +20,10 @@ BASE_URL = http://localhost:8765
| POST | `/batch-recognize` | 批量上传并识别 | multipart/form-data |
| POST | `/import-paths` | 从服务器路径导入 | JSON |
| GET | `/search` | AI 搜索图片SSE 流式) | query params |
| GET | `/{image_id}` | 获取单张图片详情 | path param |
| GET | `` | 获取图片列表(分页) | query params |
| GET | `/{image_id}` | 获取单张图片详情 | path param |
| DELETE | `/{image_id}` | 删除图片 | path param |
| POST | `/dedup` | 一键去重 | 无 |
---
@@ -42,6 +44,8 @@ Content-Type: multipart/form-data
### 返回
**正常识别:**
```json
{
"id": 386,
@@ -58,6 +62,21 @@ Content-Type: multipart/form-data
}
```
**内容重复(自动去重):**
```json
{
"id": 386,
"file_path": "data/images/test.png",
"original_filename": "test.png",
"status": "duplicate",
"duplicate_of": 100,
"ocr_text": "识别出的文字内容..."
}
```
> `duplicate_of` 指向数据库中已存在的相同内容记录 ID。
### Python 调用示例
```python
@@ -70,7 +89,10 @@ async def recognize_image(file_path: str) -> dict:
f"{BASE_URL}/api/v1/images/recognize-direct",
files={"file": (file_path, f)}
)
return resp.json()
result = resp.json()
if result.get("status") == "duplicate":
print(f"内容已存在,重复于 ID: {result['duplicate_of']}")
return result
```
---
@@ -96,7 +118,8 @@ Content-Type: multipart/form-data
{
"total": 3,
"success": 2,
"failed": 1,
"failed": 0,
"duplicates": 1,
"results": [
{
"id": 387,
@@ -110,8 +133,15 @@ Content-Type: multipart/form-data
},
{
"id": 388,
"filename": "img2.jpg",
"file_path": "data/images/img2.jpg",
"filename": "img2.png",
"status": "duplicate",
"duplicate_of": 100,
"ocr_text": "重复内容..."
},
{
"id": 389,
"filename": "img3.jpg",
"file_path": "data/images/img3.jpg",
"status": "failed",
"message": "OCR 识别失败: ..."
}
@@ -160,7 +190,7 @@ GET /api/v1/images/search?keyword=搜索词&limit=5&sort=time_desc
| 参数 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| keyword | string | 是 | - | 搜索关键词 |
| limit | int | 否 | 5 | 最多返回多少条匹配结果 |
| limit | int | 否 | 3 | 最多返回多少条匹配结果(可在设置中配置默认值) |
| sort | string | 否 | time_desc | 排序方式:`time_desc`(时间倒序)、`time_asc`(时间正序)、`random`(随机) |
### 搜索原理
@@ -168,7 +198,7 @@ GET /api/v1/images/search?keyword=搜索词&limit=5&sort=time_desc
```
用户搜索 "孩子不想上学"
→ 从数据库取所有已识别图片(按 sort 排序)
→ 每批 10 张发给 MiniMax M2.7 判断是否匹配
→ 每批 N 张发给 MiniMax M2.7 判断N 可在设置中配置,默认 10
→ 并发池最多 10 个 LLM 请求同时执行
→ 匹配结果通过 SSE 实时推送(找到就立刻返回)
→ 找够 limit 条或遍历完所有图片后结束
@@ -260,7 +290,7 @@ async def search_images(
) -> AsyncGenerator[dict, None]:
"""
AI 搜索图片SSE 流式)
Yields:
dict: 每个 SSE 事件解析后的字典
"""
@@ -281,23 +311,23 @@ async def search_images(
async def demo_search():
results = []
async for event in search_images("孩子不想上学", limit=5):
event_type = event.get("type")
if event_type == "start":
print(f"开始搜索: {event['keyword']}")
elif event_type == "progress":
print(f"进度: {event['checked']}/{event['total']},已找到 {event['found']} 条")
elif event_type == "result":
results.append(event)
print(f"找到匹配: [{event['id']}] {event['ocr_text_preview'][:50]}")
elif event_type == "done":
print(f"搜索完成: 共找到 {event['total_found']} 条,检查了 {event['total_checked']} 张")
return results
```
@@ -310,10 +340,10 @@ from google.adk import Tool
class ImageSearchTool(Tool):
"""搜索图片知识库"""
name = "search_images"
description = "搜索已识别的图片库,返回与查询语义相关的图片及其 Base64 数据。支持自然语言查询。"
async def execute(self, keyword: str, limit: int = 5) -> str:
results = []
async for event in search_images(keyword, limit=limit):
@@ -327,10 +357,10 @@ class ImageSearchTool(Tool):
})
elif event["type"] == "done":
pass # 搜索结束
if not results:
return f"未找到与 '{keyword}' 相关的图片"
# 返回 JSON方便 Agent 后续处理(如发送到企业微信)
return json.dumps({
"keyword": keyword,
@@ -413,7 +443,55 @@ GET /api/v1/images?page=1&page_size=20&status=completed
---
## 八、图片文件访问
## 八、删除图片
### 接口
```
DELETE /api/v1/images/{image_id}
```
### 返回
成功返回 `204 No Content`。
### 说明
删除操作会同时清理:
1. BM25 倒排索引中的记录
2. 磁盘上的图片文件
3. 数据库中的记录
---
## 九、一键去重
### 接口
```
POST /api/v1/images/dedup
```
### 返回
```json
{
"total_checked": 385,
"duplicates_found": 12,
"deleted": 12,
"kept": 373
}
```
### 说明
- 按 OCR 识别文本精确匹配去重
- 每组重复内容保留最早录入的记录
- 删除操作与单条删除相同(清理索引 + 磁盘文件 + 数据库记录)
---
## 十、图片文件访问
识别后的图片可通过静态文件路径访问:
@@ -428,7 +506,51 @@ http://localhost:8765/data/images/001.png
---
## 九、注意事项
## 十一、企业微信机器人相关接口
### 获取机器人状态
```
GET /api/v1/settings/bot/status
```
```json
{
"enabled": true,
"configured": true,
"running": true,
"bot_id": "aibPl3zE65xj..."
}
```
### 重启机器人
```
POST /api/v1/settings/bot/restart
```
```json
{
"success": true,
"message": "机器人已启动"
}
```
### 系统设置(含机器人配置)
```
GET /api/v1/settings
PUT /api/v1/settings
```
机器人相关字段:
- `wework_bot_enabled` (bool): 是否启用
- `wework_bot_id` (string): Bot ID
- `wework_bot_secret` (string): Secret更新时传入明文获取时返回脱敏值
---
## 十二、注意事项
1. **搜索接口是 SSE 流式**,不能用普通的 `requests.get()` 或 `httpx.get()` 直接获取 JSON必须用流式读取`stream()` / `iter_lines()`
2. **搜索耗时较长**385 张图片全部遍历约需 2-5 分钟建议设置较长的超时300 秒)
@@ -436,3 +558,5 @@ http://localhost:8765/data/images/001.png
4. **limit 参数控制返回数量**:找到 `limit` 条匹配后自动停止搜索
5. **OCR 识别耗时**:单张图片约 5-15 秒,建议设置 120 秒超时
6. **文件格式**:仅支持 .png .jpg .jpeg .bmp .webp
7. **上传自动去重**所有上传接口recognize-direct、batch-recognize、import-paths都会在 OCR 识别后检查内容是否已存在
8. **删除不可恢复**:删除操作会同时清理索引、磁盘文件和数据库记录

View File

@@ -3,14 +3,14 @@ requires = ["setuptools>=75.0", "wheel"]
build-backend = "setuptools.backends._legacy:_Backend"
[project]
name = "edu-brain"
name = "huibrain"
version = "1.2.0"
description = "中文直播教育知识库系统 - 基于向量检索的知识管理平台"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
authors = [
{name = "EduBrain Team"},
{name = "HuiBrain Team"},
]
classifiers = [
"Development Status :: 3 - Alpha",
@@ -24,9 +24,7 @@ dependencies = [
"uvicorn[standard]>=0.32.0",
"pydantic-settings>=2.6.0",
"sqlalchemy[asyncio]>=2.0.36",
"asyncpg>=0.30.0",
"alembic>=1.14.0",
"pgvector>=0.3.6",
"aiosqlite>=0.20.0",
"openai>=1.58.0",
"zhipuai>=2.4.0",
"dashscope>=1.20.0",
@@ -47,8 +45,8 @@ dev = [
]
[project.scripts]
edu-brain = "app.main:app"
edu-brain-mcp = "app.mcp.server:run_mcp_server"
huibrain = "app.main:app"
huibrain-mcp = "app.mcp.server:run_mcp_server"
[tool.ruff]
target-version = "py311"

View File

@@ -7,9 +7,6 @@ python-multipart>=0.0.18
# ── 数据库 ──
sqlalchemy[asyncio]>=2.0.36,<3.0.0
aiosqlite>=0.20.0,<1.0.0
asyncpg>=0.30.0,<1.0.0
alembic>=1.14.0,<2.0.0
pgvector>=0.3.6,<1.0.0
# ── 嵌入模型 SDK ──
openai>=1.58.0,<2.0.0
@@ -31,9 +28,6 @@ python-frontmatter>=1.0.0,<2.0.0
# ── Word 文档解析 ──
python-docx>=1.1.0,<2.0.0
# ── 中文分词BM25 搜索引擎) ──
jieba>=0.42.1
# ── 工具库 ──
numpy>=1.26.0,<3.0.0

View File

@@ -1,129 +0,0 @@
-- ============================================================
-- EduBrain 数据库初始化脚本
-- 包含 pgvector 扩展、zhparser 中文分词扩展
-- ============================================================
-- 创建 pgvector 扩展(向量相似度搜索)
CREATE EXTENSION IF NOT EXISTS vector;
-- 创建 zhparser 中文分词扩展(全文搜索)
CREATE EXTENSION IF NOT EXISTS zhparser;
-- 创建基于 zhparser 的中文全文搜索配置
CREATE TEXT SEARCH CONFIGURATION chinese_zh (PARSER = zhparser);
-- 配置中文全文搜索映射:名词、动词、形容词、成语、叹词、习语使用 simple 字典
ALTER TEXT SEARCH CONFIGURATION chinese_zh ADD MAPPING FOR n,v,a,i,e,l WITH simple;
-- ============================================================
-- 知识页面表
-- ============================================================
CREATE TABLE IF NOT EXISTS knowledge_pages (
id SERIAL PRIMARY KEY,
title VARCHAR(500) NOT NULL,
content TEXT NOT NULL,
source_file VARCHAR(500),
course_name VARCHAR(200),
teacher_name VARCHAR(100),
live_date VARCHAR(20),
page_number INTEGER,
metadata_json TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- 索引
CREATE INDEX IF NOT EXISTS idx_pages_course ON knowledge_pages(course_name);
CREATE INDEX IF NOT EXISTS idx_pages_teacher ON knowledge_pages(teacher_name);
CREATE INDEX IF NOT EXISTS idx_pages_date ON knowledge_pages(live_date);
CREATE INDEX IF NOT EXISTS idx_pages_created ON knowledge_pages(created_at DESC);
-- ============================================================
-- 知识分块表(向量检索单元)
-- ============================================================
CREATE TABLE IF NOT EXISTS knowledge_chunks (
id SERIAL PRIMARY KEY,
page_id INTEGER NOT NULL REFERENCES knowledge_pages(id) ON DELETE CASCADE,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
embedding vector(1024), -- 嵌入向量(默认 1024 维)
search_vector TSVECTOR, -- 中文全文搜索向量
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- 索引
CREATE INDEX IF NOT EXISTS idx_chunks_page ON knowledge_chunks(page_id);
CREATE INDEX IF NOT EXISTS idx_chunks_page_index ON knowledge_chunks(page_id, chunk_index);
-- HNSW 向量索引(适合高维向量,查询速度快)
CREATE INDEX IF NOT EXISTS idx_chunks_embedding
ON knowledge_chunks
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);
-- GIN 全文搜索索引
CREATE INDEX IF NOT EXISTS idx_chunks_search
ON knowledge_chunks
USING gin(search_vector);
-- ============================================================
-- OCR 图片记录表
-- ============================================================
CREATE TABLE IF NOT EXISTS ocr_images (
id SERIAL PRIMARY KEY,
file_path VARCHAR(500) NOT NULL,
ocr_text TEXT,
confidence FLOAT,
provider VARCHAR(50),
status VARCHAR(20) DEFAULT 'pending',
error_message TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- 索引
CREATE INDEX IF NOT EXISTS idx_ocr_status ON ocr_images(status);
CREATE INDEX IF NOT EXISTS idx_ocr_created ON ocr_images(created_at DESC);
-- ============================================================
-- updated_at 自动更新触发器
-- ============================================================
CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- 为所有表添加 updated_at 触发器
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_trigger
WHERE tgname = 'tr_pages_updated_at'
) THEN
CREATE TRIGGER tr_pages_updated_at
BEFORE UPDATE ON knowledge_pages
FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_trigger
WHERE tgname = 'tr_chunks_updated_at'
) THEN
CREATE TRIGGER tr_chunks_updated_at
BEFORE UPDATE ON knowledge_chunks
FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_trigger
WHERE tgname = 'tr_ocr_updated_at'
) THEN
CREATE TRIGGER tr_ocr_updated_at
BEFORE UPDATE ON ocr_images
FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
END IF;
END $$;

View File

@@ -3,10 +3,10 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>EduBrain - 知识库管理</title>
<title>HuiBrain - 知识库管理</title>
<style>
/* ═══════════════════════════════════════════════════════════════
EduBrain — Stripe-inspired Design System
HuiBrain — Stripe-inspired Design System
系统字体栈,零外部依赖
═══════════════════════════════════════════════════════════════ */
@@ -521,7 +521,7 @@
<!-- Logo 区域 -->
<div style="padding: 20px 20px 16px; border-bottom: 1px solid #e5edf5;">
<div style="display:flex; align-items:center; gap:8px;">
<span style="font-size:20px; font-weight:600; color:#061b31; letter-spacing:-0.02em;">EduBrain</span>
<span style="font-size:20px; font-weight:600; color:#061b31; letter-spacing:-0.02em;">HuiBrain</span>
</div>
</div>
@@ -551,7 +551,7 @@
<!-- 底部版本信息 -->
<div style="padding:12px 20px; border-top:1px solid #e5edf5; color:#64748d; font-size:11px;">
EduBrain v0.1.0
HuiBrain v0.1.0
</div>
</aside>
@@ -1041,6 +1041,38 @@
</div>
</div>
<!-- 企业微信机器人 -->
<div class="card" style="margin-bottom:16px;">
<div style="display:flex; align-items:center; justify-content:space-between; margin-bottom:16px;">
<h3 class="section-title" style="margin-bottom:0;">企业微信机器人</h3>
<div style="display:flex; align-items:center; gap:8px;">
<span id="bot-status-badge" class="badge badge-pending">检测中</span>
<button class="btn btn-secondary" onclick="restartBot()" id="bot-restart-btn" style="font-size:13px; padding:4px 12px;">重启机器人</button>
</div>
</div>
<div style="display:grid; grid-template-columns:1fr 1fr; gap:0 24px;">
<div class="form-group">
<label class="form-label">启用机器人</label>
<label style="display:flex; align-items:center; gap:8px; cursor:pointer; padding:6px 0;">
<input type="checkbox" id="setting-wework-bot-enabled" onchange="toggleBotEnabled()" style="width:16px; height:16px;">
<span style="font-size:13px; color:var(--text-secondary);">启用后服务启动时自动连接</span>
</label>
</div>
<div class="form-group">
<label class="form-label">Bot ID</label>
<input type="text" id="setting-wework-bot-id" placeholder="企业微信机器人 ID" class="input-field">
</div>
<div class="form-group">
<label class="form-label">Secret</label>
<input type="password" id="setting-wework-bot-secret" placeholder="企业微信机器人 Secret" class="input-field">
</div>
<div class="form-group">
<label class="form-label">服务地址</label>
<input type="text" id="setting-huibrain-base-url" value="http://localhost:8765" class="input-field" style="color:var(--text-secondary); font-size:13px;" readonly>
</div>
</div>
</div>
<!-- 图片搜索设置 -->
<div class="card" style="margin-bottom:16px;">
<h3 class="section-title">图片搜索设置</h3>
@@ -1061,6 +1093,14 @@
<span style="color:var(--text-secondary); font-size:13px;">1-50</span>
</div>
</div>
<div class="form-group">
<label class="form-label">OCR 并发识别数量</label>
<div style="display:flex; align-items:center; gap:8px;">
<input type="number" id="setting-ocr-concurrency" min="1" max="10" value="1"
style="width:80px;" class="input-field">
<span style="color:var(--text-secondary); font-size:13px;">1-10多用户同时上传时控制并发</span>
</div>
</div>
</div>
</div>
@@ -2101,13 +2141,6 @@
queue.innerHTML = `
<div style="display:flex; flex-direction:column; gap:6px;">
${ocrFiles.map((item, i) => {
const keywords = (item.result && item.result.tags) ? item.result.tags : [];
const kwHtml = keywords.length > 0
? `<div style="display:flex; flex-wrap:wrap; gap:4px; margin-top:4px;">${keywords.map(k => `<span style="display:inline-block; padding:1px 8px; background:#e8f4fd; color:#0071e3; border-radius:980px; font-size:11px;">${escapeHtml(k)}</span>`).join('')}</div>`
: '';
const storyHtml = (item.result && item.result.story_summary)
? `<p style="font-size:12px; color:#6e6e73; margin-top:2px; font-style:italic;">${escapeHtml(item.result.story_summary)}</p>`
: '';
return `
<div style="padding:8px 12px; background:#f5f5f7; border-radius:10px;">
<div style="display:flex; align-items:center; gap:10px;">
@@ -2116,8 +2149,6 @@
<span style="font-size:12px; color:#86868b; white-space:nowrap;">${statusText(item.status)}${item.elapsed ? ' (' + item.elapsed.toFixed(1) + 's)' : ''}</span>
${item.status === 'pending' && !ocrInProgress ? `<button class="btn btn-sm btn-secondary" onclick="removeOcrFile(${i})" style="padding:2px 8px; font-size:12px;">移除</button>` : ''}
</div>
${kwHtml}
${storyHtml}
</div>`;
}).join('')}
</div>
@@ -2450,7 +2481,6 @@
路径: ${escapeHtml(result.file_path || '')}
</p>
${imgHtml}
${result.story_summary ? `<div style="background:#f0f5ff; border-radius:10px; padding:12px; margin-bottom:12px; border-left:3px solid #0071e3;"><p style="font-size:13px; color:#1d1d1f; margin:0; font-weight:500;">故事摘要</p><p style="font-size:13px; color:#424245; margin:4px 0 0 0; line-height:1.5;">${escapeHtml(result.story_summary)}</p></div>` : ''}
<div style="background:#f5f5f7; border-radius:12px; padding:16px; max-height:300px; overflow-y:auto;">
<pre style="white-space:pre-wrap; font-size:14px; color:#424245; line-height:1.6; font-family:inherit; margin:0;">${escapeHtml(result.ocr_text || '无识别结果')}</pre>
</div>
@@ -2501,13 +2531,6 @@
}
} else if (data.type === 'result') {
foundCount++;
const tagsHtml = (tags) => {
if (!tags || tags.length === 0) return '';
return '<div style="display:flex; flex-wrap:wrap; gap:4px; margin-top:4px;">' +
tags.map(t => `<span style="display:inline-block; padding:2px 8px; border-radius:12px; font-size:11px; background:rgba(0,122,255,0.1); color:#007aff; white-space:nowrap;">${escapeHtml(t)}</span>`).join('') +
'</div>';
};
const itemHtml = `
<div style="background:#f5f5f7; border-radius:10px; padding:12px; cursor:pointer; animation:fadeIn 0.3s;" onclick="viewOcrResult(${data.id})">
<div style="display:flex; align-items:center; gap:8px; margin-bottom:4px;">
@@ -2515,7 +2538,6 @@
<span style="font-size:12px; color:#86868b;">#${foundCount}</span>
</div>
<p style="font-size:13px; color:#86868b;">${escapeHtml(data.ocr_text_preview || '')}</p>
${tagsHtml(data.tags)}
</div>`;
const itemsContainer = document.getElementById('ocr-search-items');
@@ -2612,12 +2634,25 @@
if (data.judge_batch_size != null) {
document.getElementById('setting-judge-batch-size').value = data.judge_batch_size;
}
if (data.ocr_concurrency != null) {
document.getElementById('setting-ocr-concurrency').value = data.ocr_concurrency;
}
// 企业微信机器人
document.getElementById('setting-wework-bot-enabled').checked = data.wework_bot_enabled;
document.getElementById('setting-wework-bot-id').value = data.wework_bot_id || '';
// secret 只显示脱敏值作为 placeholder
const secretInput = document.getElementById('setting-wework-bot-secret');
if (data.wework_bot_secret_masked) {
secretInput.placeholder = '当前: ' + data.wework_bot_secret_masked;
}
} catch (_) {
// 静默处理
}
// 初始化字段显示状态
toggleEmbedFields();
toggleOcrFields();
// 加载机器人状态
loadBotStatus();
}
/** 保存设置 */
@@ -2631,6 +2666,16 @@
if (!isNaN(judgeBatchSize) && judgeBatchSize >= 1 && judgeBatchSize <= 50) {
body.judge_batch_size = judgeBatchSize;
}
const ocrConcurrency = parseInt(document.getElementById('setting-ocr-concurrency').value);
if (!isNaN(ocrConcurrency) && ocrConcurrency >= 1 && ocrConcurrency <= 10) {
body.ocr_concurrency = ocrConcurrency;
}
// 企业微信机器人
body.wework_bot_enabled = document.getElementById('setting-wework-bot-enabled').checked;
const botId = document.getElementById('setting-wework-bot-id').value.trim();
if (botId) body.wework_bot_id = botId;
const botSecret = document.getElementById('setting-wework-bot-secret').value.trim();
if (botSecret) body.wework_bot_secret = botSecret;
if (Object.keys(body).length === 0) {
showToast('没有需要保存的更改', 'info');
return;
@@ -2643,6 +2688,53 @@
}
}
/** 加载机器人状态 */
async function loadBotStatus() {
try {
const data = await api('GET', '/settings/bot/status');
const badge = document.getElementById('bot-status-badge');
if (data.running) {
badge.className = 'badge badge-completed';
badge.textContent = '运行中';
} else if (data.configured) {
badge.className = 'badge badge-failed';
badge.textContent = '已停止';
} else {
badge.className = 'badge badge-pending';
badge.textContent = '未配置';
}
} catch (_) {
document.getElementById('bot-status-badge').className = 'badge badge-pending';
document.getElementById('bot-status-badge').textContent = '未知';
}
}
/** 重启企业微信机器人 */
async function restartBot() {
const btn = document.getElementById('bot-restart-btn');
btn.disabled = true;
btn.textContent = '重启中...';
try {
const result = await api('POST', '/settings/bot/restart');
if (result.success) {
showToast(result.message, 'success');
} else {
showToast(result.message, 'error');
}
setTimeout(() => loadBotStatus(), 2000);
} catch (err) {
showToast('重启失败: ' + err.message, 'error');
} finally {
btn.disabled = false;
btn.textContent = '重启机器人';
}
}
/** 切换机器人启用状态 */
function toggleBotEnabled() {
// 仅切换 UI 状态,实际保存需要点"保存设置"
}
/** 测试嵌入模型连接 */
async function testEmbedding() {
showToast('正在测试嵌入模型连接...', 'info');
@@ -2686,7 +2778,7 @@
async function loadSystemInfo() {
// 数据库状态检测
try {
await api('GET', '/../health');
await fetch(window.location.origin + '/health');
document.getElementById('sys-db-status').innerHTML =
'<span class="badge badge-completed">正常</span>';
} catch (_) {