phase 5 prep: backfill-users.ts + deployment runbook in .memory
- scripts/backfill-users.ts: 扫 PG users 表,对每人幂等调 orchestrator POST /users - orchestrator/package.json: 加 postgres 依赖 - .memory/project.md: Phase 3/4 完成状态,Phase 5 上线 checklist Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -81,18 +81,35 @@ Debian 13 默认 Python 3.13(不是计划的 3.12)。不影响功能,更新。
|
|||||||
- **不预装** pandas/numpy/torch/playwright/ffmpeg/libreoffice
|
- **不预装** pandas/numpy/torch/playwright/ffmpeg/libreoffice
|
||||||
- LLM 按需 `uv pip install xxx` — 后续加本地 PyPI 缓存镜像让它快到秒级(尚未做,列在 TODO)
|
- LLM 按需 `uv pip install xxx` — 后续加本地 PyPI 缓存镜像让它快到秒级(尚未做,列在 TODO)
|
||||||
|
|
||||||
## 进度(2026-04-18)
|
## 进度(2026-04-19 更新)
|
||||||
|
|
||||||
- ✅ Phase 0 Pre-flight(VPS 摸底:31GB RAM / 387GB 磁盘 / Incus 6.0.0 / hermes-box 已跑)
|
- ✅ Phase 0 Pre-flight(VPS 摸底:31GB RAM / 387GB 磁盘 / Incus 6.0.0 / hermes-box 已跑)
|
||||||
- ✅ Phase 1 宿主初始化(btrfs pool + project + profile + **UFW iptables 修复并持久化**)
|
- ✅ Phase 1 宿主初始化(btrfs pool + project + profile + **UFW iptables 修复并持久化**)
|
||||||
- 🔄 **Phase 2 base 镜像构建**(apt 完成,Node 20 安装中/卡)
|
- ✅ Phase 2 base 镜像 `lobe-sandbox-base` 已发布(500MiB,fingerprint `de8d9b6b73c4`)
|
||||||
- 已装:python3.13, build-essential, git, curl, ripgrep, fd, bat, 中文字体, tzdata, locale
|
- Debian 13 trixie + Python 3.13.5 + Node 20.20.2 + Bun 1.3.12 + uv 0.11.7 + git/ripgrep/fd/bat + 中文字体 + zh_CN locale
|
||||||
- 已装:nodejs 20.20.2(via NodeSource)
|
- **两个坑踩过并写进脚本**:
|
||||||
- **待装**:corepack + uv + bun + sandbox 用户 + /workspace + publish
|
1. `INSTALL_DIR` 装 uv 无效,要用 `UV_INSTALL_DIR` 环境变量
|
||||||
- **已踩的坑**:用 `nohup bash -s <<EOF &` 跑后台任务,SSH 断开时脚本跟着死。应改 `systemd-run --unit=xxx` 或 `setsid` 真正脱离 shell
|
2. `nohup bash -s <<EOF &` 脱不开 SSH,后续全改 `systemd-run --unit=xxx --slice=system.slice`
|
||||||
- ⏸ Phase 3 Orchestrator 服务(Bun HTTP,15 工具)
|
- ✅ Phase 3 Orchestrator(Bun + Hono,已部署 VPS systemd)
|
||||||
- ⏸ Phase 4 LobeChat 侧接入(4 文件小改)
|
- 服务:`sandbox-orchestrator.service` 监听 `127.0.0.1:8700`
|
||||||
- ⏸ Phase 5 存量补建 + 联调 + 上线
|
- 配置:`/etc/lobe-sandbox/orchestrator.env`(含 SANDBOX_ORCH_SECRET)
|
||||||
|
- 端到端验证:POST /users → executeCode → runCommand + uv pip install numpy → writeLocalFile → listLocalFiles 全 OK
|
||||||
|
- 已实现 13 工具(executeCode/runCommand/readLocalFile/writeLocalFile/editLocalFile/listLocalFiles/moveLocalFiles/renameLocalFile/searchLocalFiles/grepContent/globLocalFiles + ensureRunning 逻辑 + 30min idle reaper + HMAC auth + S3 export 占位)
|
||||||
|
- **长驻命令(getCommandOutput/killCommand)留到 v2**,当前 runCommand 是同步 2min 超时
|
||||||
|
- ✅ Phase 4 LobeChat 侧接入代码已提交(分支 `feat/self-hosted-sandbox`,commit `12b53e90b1`)
|
||||||
|
- `src/envs/app.ts` 新增 `SANDBOX_BACKEND_URL` + `SANDBOX_BACKEND_SECRET`
|
||||||
|
- `src/server/services/sandbox/localSandbox.ts` 新 LocalSandboxService
|
||||||
|
- `src/server/services/toolExecution/serverRuntimes/cloudSandbox.ts` env 分叉
|
||||||
|
- `src/libs/better-auth/define-config.ts` user.create.after 追加 provision(fire-and-forget)
|
||||||
|
- **注意**: main 分支 pre-commit 会跑全量 tsc 撑爆内存 + 碰到别人的 pre-existing TS 错误,所以必须在 feature branch 上 commit
|
||||||
|
- ⏸ Phase 5 待做(上线):
|
||||||
|
- [ ] 确定 LobeChat docker 容器访问 orchestrator 的网络路径(host.docker.internal / docker0 网关 172.17.0.1 / 绑 0.0.0.0 + UFW 限制)
|
||||||
|
- [ ] merge `feat/self-hosted-sandbox` 到 main(需先解决 main 上别人的 pre-existing TS 错误,或单独 deploy 这个 branch)
|
||||||
|
- [ ] Coolify 里 LobeChat 容器加 `SANDBOX_BACKEND_URL` + `SANDBOX_BACKEND_SECRET` 环境变量
|
||||||
|
- [ ] 重启 LobeChat(一次性,不要频繁重启)
|
||||||
|
- [ ] 跑 `scripts/backfill-users.ts` 给存量用户补建沙箱
|
||||||
|
- [ ] exportFile 接入:当前 `S3_ENDPOINT=192.168.2.221:9000` 是内网,VPS 够不着 → 改用 LobeChat 同机上的 RustFS(`lobemile2026/RustFS@2026MileSecure`)
|
||||||
|
- [ ] 端到端 smoke:用户 ai.milejoy.com 发"写个 Python 打印 hello" → 查 orchestrator 日志命中 → 不是 market.lobehub.com
|
||||||
|
|
||||||
## 关键路径速查
|
## 关键路径速查
|
||||||
|
|
||||||
|
|||||||
@@ -160,6 +160,13 @@
|
|||||||
"message": "orchestrator MVP: 13 工具跑通 + systemd + 部署脚本",
|
"message": "orchestrator MVP: 13 工具跑通 + systemd + 部署脚本",
|
||||||
"hash": "ce8e22f",
|
"hash": "ce8e22f",
|
||||||
"files_changed": 1
|
"files_changed": 1
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ts": "2026-04-19T21:12:27+08:00",
|
||||||
|
"type": "commit",
|
||||||
|
"message": "auto-save 2026-04-19 21:12 (~1)",
|
||||||
|
"hash": "e7155b7",
|
||||||
|
"files_changed": 1
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,7 @@
|
|||||||
"@aws-sdk/client-s3": "^3.670.0",
|
"@aws-sdk/client-s3": "^3.670.0",
|
||||||
"@aws-sdk/s3-request-presigner": "^3.670.0",
|
"@aws-sdk/s3-request-presigner": "^3.670.0",
|
||||||
"hono": "^4.6.0",
|
"hono": "^4.6.0",
|
||||||
|
"postgres": "^3.4.5",
|
||||||
"zod": "^3.23.0"
|
"zod": "^3.23.0"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
|
|||||||
59
scripts/backfill-users.ts
Normal file
59
scripts/backfill-users.ts
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
#!/usr/bin/env bun
|
||||||
|
/**
|
||||||
|
* 给存量 LobeChat 用户补建沙箱容器(幂等,已存在就跳过)。
|
||||||
|
*
|
||||||
|
* 用法(在 VPS 上跑):
|
||||||
|
* export DATABASE_URL=postgresql://...
|
||||||
|
* export SANDBOX_BACKEND_URL=http://127.0.0.1:8700
|
||||||
|
* export SANDBOX_BACKEND_SECRET=<same as orchestrator>
|
||||||
|
* bun run scripts/backfill-users.ts
|
||||||
|
*/
|
||||||
|
import postgres from 'postgres';
|
||||||
|
|
||||||
|
const DATABASE_URL = process.env.DATABASE_URL;
|
||||||
|
const BACKEND_URL = process.env.SANDBOX_BACKEND_URL ?? 'http://127.0.0.1:8700';
|
||||||
|
const SECRET = process.env.SANDBOX_BACKEND_SECRET;
|
||||||
|
|
||||||
|
if (!DATABASE_URL) throw new Error('DATABASE_URL required');
|
||||||
|
if (!SECRET) throw new Error('SANDBOX_BACKEND_SECRET required');
|
||||||
|
|
||||||
|
const sql = postgres(DATABASE_URL, { max: 1 });
|
||||||
|
|
||||||
|
const users: { id: string; email: string | null }[] = await sql`
|
||||||
|
SELECT id, email FROM users ORDER BY created_at ASC
|
||||||
|
`;
|
||||||
|
|
||||||
|
console.log(`[backfill] ${users.length} users to check`);
|
||||||
|
|
||||||
|
let ok = 0,
|
||||||
|
skip = 0,
|
||||||
|
fail = 0;
|
||||||
|
|
||||||
|
for (const u of users) {
|
||||||
|
try {
|
||||||
|
const res = await fetch(`${BACKEND_URL}/api/v1/users`, {
|
||||||
|
body: JSON.stringify({ userId: u.id }),
|
||||||
|
headers: { 'Content-Type': 'application/json', 'X-Sandbox-Secret': SECRET },
|
||||||
|
method: 'POST',
|
||||||
|
});
|
||||||
|
const text = await res.text();
|
||||||
|
if (res.ok) {
|
||||||
|
if (text.includes('"success":true')) {
|
||||||
|
ok++;
|
||||||
|
console.log(`[backfill] ✓ ${u.id} (${u.email ?? '-'})`);
|
||||||
|
} else {
|
||||||
|
skip++;
|
||||||
|
console.log(`[backfill] ~ ${u.id}: ${text.slice(0, 100)}`);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
fail++;
|
||||||
|
console.error(`[backfill] ✗ ${u.id}: HTTP ${res.status} ${text.slice(0, 100)}`);
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
fail++;
|
||||||
|
console.error(`[backfill] ✗ ${u.id}:`, (e as Error).message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`[backfill] done: ok=${ok} skip=${skip} fail=${fail}`);
|
||||||
|
await sql.end();
|
||||||
Reference in New Issue
Block a user