auto-save 2026-05-19 10:35 (+3, ~8, -1)
This commit is contained in:
@@ -451,6 +451,19 @@
|
||||
"type": "session-heartbeat",
|
||||
"message": "Codex 会话活跃 · 最近命令：codex · 分支 master · 1 项未提交变更 · 最近提交：docs: record anchored image pipeline",
|
||||
"files_changed": 1
|
||||
},
|
||||
{
|
||||
"ts": "2026-05-19T10:24:13+08:00",
|
||||
"type": "commit",
|
||||
"message": "auto-save 2026-05-19 10:24 (~2)",
|
||||
"hash": "b317abe",
|
||||
"files_changed": 2
|
||||
},
|
||||
{
|
||||
"ts": "2026-05-19T02:30:00Z",
|
||||
"type": "session-heartbeat",
|
||||
"message": "Codex 会话活跃 · 最近命令：codex · 分支 master · 1 项未提交变更 · 最近提交：auto-save 2026-05-19 10:24 (~2)",
|
||||
"files_changed": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -665,3 +665,440 @@ UI 上传图区域要醒目提示：
|
||||
|
||||
如果只做一个，先做 Mode B —— 它对"前后一致"的帮助最直接，相当于直接拿用户图当 L0 锚图，跳过最容易漂移的"prompt → 意向图"阶段。
|
||||
|
||||
---
|
||||
|
||||
## 9. 实例：上传一张 lookbook 整图的工作流
|
||||
|
||||
### 9.1 场景描述
|
||||
|
||||
用户拿到一张已经完整的商品 lookbook 图（如 MUSE MATE 街头潮玩公仔的 14 区块大图），里面已经包含核心形象、包装、三视图、细节、场景、配件、社媒图、专利六视图、产品信息等。这是 Mode C 复刻+补全的极端情况——**几乎所有 slot 都已经有素材**，只需要补少量缺失视角和细节。
|
||||
|
||||
### 9.2 上传图的内容分类（以 MUSE MATE lookbook 为例）
|
||||
|
||||
```
|
||||
01. 核心形象 → 单只主角图
|
||||
02. 包装展示 → 礼盒 + 配件平铺
|
||||
03. 三视图 → Front / Side / Back
|
||||
04. 细节展示 → 头部 / 滑板 / 卫衣特写 ×4
|
||||
05. 场景展示 → 涂鸦墙 / 唱片店 / 滑板公园 / 书桌 / 车载 / 包挂 ×6
|
||||
06. 配件展示 → 帽子 / 耳机 / 滑板 / 喷漆 / 卫衣 / 钥匙扣 / 编号卡 / 贴纸 ×8
|
||||
07. 可替换造型 → 黑 / 灰 / 橙 / 绿 4 套服饰
|
||||
08. 灯光效果 → 白光 / 暖光 2 张
|
||||
09. 证书 + 编号卡 → 收藏卡
|
||||
10. 社媒展示 → 明星种草 3 张
|
||||
11. 系列款展示 → 6 个配色变体
|
||||
12. 专利图纸 → 已完整的六视图
|
||||
13. 产品信息 → ABS/PVC、高度 12cm、包装尺寸文字
|
||||
14. 合作流程 → 流程图（非产品素材）
|
||||
```
|
||||
|
||||
### 9.3 系统映射表
|
||||
|
||||
| Lookbook 区块 | 系统 slot | 数量 | 备注 |
|
||||
|---|---|---|---|
|
||||
| 01 核心形象 | L0 主体图 → `subject` role | 1 | 净化后做 L1 锚图 |
|
||||
| 02 包装 | `mkt_packaging_render` + `prod_packaging_structure` | 2 | 切出 |
|
||||
| 03 三视图 | `patent_front` / `patent_left` / `patent_back` | 3 | 直接占用 |
|
||||
| 04 细节 | `patent_detail_face` + `patent_detail_accessory` + `mkt_detail_face` + `mkt_detail_material` | 4 | 切出 |
|
||||
| 05 场景 | `mkt_scene_bedroom/desk/gift` + 新增「街头 / 车载 / 包挂」slot | 6 | 拓展模板 |
|
||||
| 06 配件 | `acc_inventory_sheet` + 8 个配件孤立锚图 | 9 | 触发 8 个 AccessoryGroup |
|
||||
| 07 服饰变体 | **新 slot：`variant_outfit`** | 4 | 拓展（系列变体） |
|
||||
| 08 灯光变体 | **新 slot：`variant_lighting`** | 2 | 拓展 |
|
||||
| 09 证书卡 | **新 slot：`cert_card`** | 1 | 收藏品需要 |
|
||||
| 10 社媒 | `mkt_social_vertical` | 3 | 占用 |
|
||||
| 11 系列款 | **新 slot：`series_lineup`** | 1 | 拓展 |
|
||||
| 12 专利六视图 | `patent_front/back/left/right/top/bottom` | 6 | 完全占满 |
|
||||
| 13 产品信息 | OCR 后填到 `text_production_brief` / `text_production_cmf` | - | 文字 slot |
|
||||
| 14 合作流程 | 忽略 | - | 非素材 |
|
||||
|
||||
### 9.4 用户操作流程
|
||||
|
||||
```
|
||||
1. 上传 lookbook 整图（role: 'lookbook-composite'）
|
||||
2. 系统检测到合成图 → 弹出区块切割界面
|
||||
- Vision 识别"01."至"14."编号定位分区线
|
||||
- 用户可手动调整裁剪框
|
||||
- 每块标 role
|
||||
3. 切完得到 30-40 张独立图，写入 data/uploads/
|
||||
4. 系统按 role 自动分配 slot
|
||||
5. 调 Vision 看 L0 + 三视图 + 配件清单 → 自动推断 CharacterSpec
|
||||
6. 用户进入 PackPanel：
|
||||
- 已占用 slot 显示 ✓
|
||||
- 缺失 slot 显示「待补生成」
|
||||
7. 用户决定一键补全 / 挑重要 slot 补全
|
||||
```
|
||||
|
||||
### 9.5 算力节省
|
||||
|
||||
对这张 lookbook 来说：
|
||||
|
||||
| Pack | 全量生成需要 | 上传图已占 | 实际需补生成 |
|
||||
|---|---|---|---|
|
||||
| 专利包 | 12 张 | 7 张 | 5 张（右/上/下/立体×2） |
|
||||
| 配件包 | 9 张（清单）+ 6×8 = 57 张 | 9 张（清单 + 各 1 视图） | ~48 张（每件还缺 5 视图 + 组合图） |
|
||||
| 生产包 | 18 张 | 0 张（lookbook 没生产图） | 18 张全补 |
|
||||
| 宣发包 | 22 张 | 11 张（KV/包装/场景/社媒） | 11 张 |
|
||||
| **合计** | **≈118 张** | **≈40 张** | **≈82 张** |
|
||||
|
||||
省下约 **34% API 调用**。更重要的是：用户自己的图是最强 anchor，前后一致性最高。
|
||||
|
||||
### 9.6 需要新增的模板 / 数据结构
|
||||
|
||||
为支撑 lookbook 场景，建议扩展：
|
||||
|
||||
```typescript
|
||||
// 新增 role 类型
|
||||
export type UploadRole =
|
||||
| 'subject' | 'reference'
|
||||
| 'view-front' | 'view-back' | 'view-left' | 'view-right' | 'view-top' | 'view-bottom'
|
||||
| 'accessory-isolated' | 'accessory-named'
|
||||
| 'scene-bedroom' | 'scene-desk' | 'scene-gift'
|
||||
| 'scene-street' | 'scene-car' | 'scene-bag' // 新增场景
|
||||
| 'detail-face' | 'detail-accessory' | 'detail-material'
|
||||
| 'social-vertical' | 'social-square'
|
||||
| 'packaging-overview' | 'packaging-structure'
|
||||
| 'variant-outfit' | 'variant-lighting' // 新拓展
|
||||
| 'cert-card' | 'series-lineup' // 新拓展
|
||||
| 'lookbook-composite'; // 整张 lookbook
|
||||
```
|
||||
|
||||
新增模板（templates.ts 里追加）：
|
||||
|
||||
- `mkt_scene_street` / `mkt_scene_car` / `mkt_scene_bag`（场景包补 3 个）
|
||||
- `variant_outfit_*` × 4（服饰变体包）
|
||||
- `variant_lighting_white` / `variant_lighting_warm`（灯光变体）
|
||||
- `cert_card`（收藏品类附件）
|
||||
- `series_lineup`（系列陈列图）
|
||||
|
||||
新增 API：
|
||||
|
||||
```
|
||||
POST /api/uploads/split-composite
|
||||
Body: { uploadedImageId, regions: Array<{ role, bbox, accessoryName? }> }
|
||||
Resp: { sessionId, splitImages: UploadedImage[] }
|
||||
```
|
||||
|
||||
### 9.7 这个实例对实施顺序的影响
|
||||
|
||||
如果用户主要场景是"已有完整或半完整 lookbook"，那 §4 实施 Checklist 的优先级应该调整：
|
||||
|
||||
1. **优先做 §8 上传图模式（Mode B 复刻）**
|
||||
2. 其次做 §1 锚图链
|
||||
3. 再做 §9 区块切割 + role 标注 + slot 自动占用
|
||||
4. 最后做风格库、Vision 配件识别等增强功能
|
||||
|
||||
因为 lookbook 用户根本不需要"从 prompt 生意向图"，他们要的是"把这套素材合理拆分填进系统，缺什么补什么"。
|
||||
|
||||
---
|
||||
|
||||
## 10. 完整 Agent 编排：从任意输入到完整 lookbook
|
||||
|
||||
### 10.1 目标
|
||||
|
||||
用户无论上传什么（一句话 / 单张主角图 / 完整 lookbook 大图 / 几张零碎参考图），系统都能自动跑到同一个终态：**一套完整的专利包 + 配件包 + 生产包 + 宣发包 + 视频任务 + 设计说明文字**，并显式区分「已占用」「AI 补生成」「需人工确认」三种状态。
|
||||
|
||||
### 10.2 三层 Agent 架构
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Orchestrator Agent — 决策总指挥 │
|
||||
│ · 决定走哪条路径（Mode A/B/C） │
|
||||
│ · 调度拓扑生成顺序 │
|
||||
│ · 触发自检 & 重做 │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||||
│ Vision Analyst │ │ Generation Worker│ │ Quality Checker │
|
||||
│ · 识图分类 │ │ · 调 GPT 生图 │ │ · 角色一致性 │
|
||||
│ · 区块切割 │ │ · 调 Seedance │ │ · 视角正确性 │
|
||||
│ · 推断 Spec │ │ · multipart 上传 │ │ · 风格统一 │
|
||||
│ · 配件识别 │ │ · 锚图链解析 │ │ · 标红需重做 │
|
||||
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||||
```
|
||||
|
||||
实现层面：
|
||||
- 三个 Agent 可以是同一个 GPT 模型不同 prompt
|
||||
- 也可以分别用：`gpt-5.5-vision` 做识图、`gpt-image-2` 做生图、`gpt-5.5` 做质检
|
||||
- 编排可以用 Vercel AI SDK / LangChain，**也可以纯 TypeScript 状态机**（推荐先用后者，可控性强）
|
||||
|
||||
### 10.3 完整流程状态机
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: idle │
|
||||
│ 用户输入：prompt? upload? both? │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: input-analysis │
|
||||
│ Vision Agent 看输入图（如有） │
|
||||
│ 输出 InputClassification： │
|
||||
│ { mode: 'prompt-only' | 'single-subject' | 'lookbook' │
|
||||
│ | 'multi-reference', │
|
||||
│ blocksDetected?: BlockBBox[], │
|
||||
│ detectedSubject?: SubjectGuess, │
|
||||
│ detectedAccessories?: AccessoryGuess[], │
|
||||
│ confidence: 0..1 } │
|
||||
│ confidence < 0.7 → 询问用户 │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌──────────┴──────────┐
|
||||
▼ ▼
|
||||
┌──────────────────┐ ┌──────────────────┐
|
||||
│ Path A: prompt │ │ Path B: image │
|
||||
│ → 批量生意向图 │ │ ┌──────────────┤
|
||||
│ → 九宫格筛选 │ │ ▼ │
|
||||
│ → 选中 │ │ Mode B 单图 │
|
||||
│ │ │ Mode C lookbook │
|
||||
│ │ │ Mode A multi-ref│
|
||||
└──────────────────┘ └──────────────────┘
|
||||
└──────────┬──────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: anchor-preparation │
|
||||
│ · L0 = 选中图或主体图 │
|
||||
│ · L1 = L0 经 cleanup 净化（preserveLevel=strict 复刻； │
|
||||
│ normal 二创可允许微调） │
|
||||
│ · 若是 lookbook：先做区块切割 → slot 自动占用 │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: character-inference │
|
||||
│ Vision Agent 看 L1 + 已占用 slot │
|
||||
│ 输出 CharacterSpec（含 accessoriesDetected[]） │
|
||||
│ 用户确认/编辑 │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: pack-generation（拓扑） │
|
||||
│ │
|
||||
│ Wave 1（并行）： │
|
||||
│ · patent_front（用 L1） │
|
||||
│ · acc_inventory_sheet（用 L1） │
|
||||
│ · mkt_white_front（用 L1） │
|
||||
│ │
|
||||
│ Wave 2（并行）： │
|
||||
│ · patent_back/left/right/top/bottom（用 patent_front） │
|
||||
│ · 每个配件 accessory_isolated（用 acc_inventory） │
|
||||
│ · mkt_white_45/back（用 mkt_white_front） │
|
||||
│ · prod_front_spec/back_spec/...（用 patent_front） │
|
||||
│ │
|
||||
│ Wave 3（并行）： │
|
||||
│ · patent_perspective_front/back / detail_* │
|
||||
│ · 每个配件的 6 视图（用对应 accessory_isolated） │
|
||||
│ · mkt_scene_* / mkt_detail_* │
|
||||
│ · prod_material_board / color_board / part_breakdown │
|
||||
│ │
|
||||
│ Wave 4： │
|
||||
│ · acc_with_doll_assembly（用 L1 + 各 isolated） │
|
||||
│ · mkt_size_lifestyle / longpage / packaging_render │
|
||||
│ │
|
||||
│ Wave 5： │
|
||||
│ · 设计说明文字（GPT text，基于 CharacterSpec + 各 anchor）│
|
||||
│ · 视频任务（用 mkt_white_front） │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: quality-check │
|
||||
│ Quality Checker Agent 看每张产物 │
|
||||
│ 对比 anchor → 一致性评分 │
|
||||
│ 标记需重做的图（红色） │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: review │
|
||||
│ 用户在 PackPanel 看完整产出 │
|
||||
│ 每张图状态：✓ 已占用 / ✨ AI 生成 / 🔴 待重做 / ⚠ 需人工确认 │
|
||||
│ 一键重做标红的图 / 手动重做某张 │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ STATE: export │
|
||||
│ 导出 ZIP / PDF / manifest.json │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 10.4 关键 Agent 函数（不写代码，只列接口）
|
||||
|
||||
```typescript
|
||||
// === Vision Analyst ===
|
||||
|
||||
inferInputClassification(uploads: UploadedImage[], prompt?: string): InputClassification
|
||||
// 决定走 prompt / single-subject / lookbook / multi-reference
|
||||
|
||||
detectLookbookBlocks(imageUrl: string): BlockBBox[]
|
||||
// 返回每个区块的 bbox + 自动建议 role
|
||||
|
||||
inferCharacterSpec(anchorImageUrl: string, userHint?: string): CharacterSpec
|
||||
// 看图推断完整 CharacterSpec
|
||||
|
||||
detectAccessories(anchorImageUrl: string): DetectedAccessory[]
|
||||
// 看图识别所有独立配件
|
||||
|
||||
// === Generation Worker ===
|
||||
|
||||
generateImage({ prompt, anchorBuffer, maskBuffer?, size, negative }): GenImage
|
||||
// 真图生图，multipart /images/edits
|
||||
|
||||
generateText({ prompt, format: 'json' | 'markdown' | 'plain' }): string
|
||||
// GPT text
|
||||
|
||||
generateVideo({ prompt, anchorImageUrl, duration, ratio }): VideoTask
|
||||
// Seedance
|
||||
|
||||
// === Quality Checker ===
|
||||
|
||||
assessConsistency({ generatedImage, anchorImage }): {
|
||||
score: 0..1, // 角色一致性评分
|
||||
drifts: string[], // 漂移点说明
|
||||
needsRedo: boolean
|
||||
}
|
||||
|
||||
assessViewAccuracy({ image, expectedView: 'front' | 'left' | ... }): {
|
||||
score: 0..1,
|
||||
notes: string[]
|
||||
}
|
||||
|
||||
// === Orchestrator ===
|
||||
|
||||
planTopologicalGeneration(session): GenerationWave[]
|
||||
// 计算各 wave 依赖关系
|
||||
|
||||
runGenerationLoop(session): AsyncGenerator<ProgressEvent>
|
||||
// 跑完整生成 + 自检 + 重试
|
||||
```
|
||||
|
||||
### 10.5 Topological Generation 详解
|
||||
|
||||
每个 `AssetTemplate` 加 `anchorTemplateId` 字段后，可以构建 DAG：
|
||||
|
||||
```typescript
|
||||
type GenerationNode = {
|
||||
templateId: string;
|
||||
packKind: PackKind;
|
||||
dependsOn: string[]; // 上游 templateIds
|
||||
alreadySatisfied: boolean; // 已由上传图占用？
|
||||
};
|
||||
|
||||
function buildDAG(session): GenerationNode[]
|
||||
function topologicalSort(nodes): GenerationNode[][] // 分波次
|
||||
```
|
||||
|
||||
**关键**：每个 Wave 内的节点可以**并行执行**（concurrency=4 或 8），跨 Wave 必须串行（因为下游需要上游图作为 anchor）。
|
||||
|
||||
实测一张主角图全量生成（专利 12 + 配件清单 9 + 配件六视图 48 + 生产 18 + 宣发 22 + 视频 5 = 114 张图）+ 16 段文字，按 5 Wave 并行（concurrency=4），用时大约：
|
||||
|
||||
- Wave 1：3 张并行 → ~10s
|
||||
- Wave 2：~20 张并行（分 5 批）→ ~50s
|
||||
- Wave 3：~70 张并行（分 18 批）→ ~3min
|
||||
- Wave 4：~10 张 → ~25s
|
||||
- Wave 5：文字 + 视频提交（视频是异步任务）→ ~30s
|
||||
|
||||
**总计约 5 分钟出完整 lookbook**（视频是异步任务还要等几分钟）。比串行生成（每张 3s × 114 = 5.7min 还要排队）快不少，且一致性最强。
|
||||
|
||||
### 10.6 Quality Check 的具体策略
|
||||
|
||||
让 Vision Agent 做 4 项检查：
|
||||
|
||||
1. **角色一致性**：把生成图和 L1 锚图拼成一张图，问 GPT "这两张是同一个角色吗？打分 0-1，列出差异"
|
||||
2. **视角正确性**：问 "这张图是正面/左视图/俯视图吗？"
|
||||
3. **背景清洁度**（专利图必须）：问 "是否有水印、文字、场景道具？"
|
||||
4. **配件完整性**：问 "源图上的 X 配件在这张里是否清晰可见？"
|
||||
|
||||
每项分数 < 0.7 → 标红待重做。重做时把上一次的差异点写进 `userRefinement` 反馈给 prompt：
|
||||
|
||||
```
|
||||
追加约束：上次生成中 ${drifts} 出现问题，本次必须修正。
|
||||
```
|
||||
|
||||
### 10.7 UI 上的 Agent 进度展示
|
||||
|
||||
`PackPanel` 顶部加一条**生成总进度条**：
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ 🤖 Agent 工作中 · Wave 3/5 · 已生成 47/114 张 │
|
||||
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41% │
|
||||
│ 当前批次：配件六视图（帽子/耳机/滑板...） │
|
||||
│ 已完成自检 ✓ 33 张 · 🔴 待重做 2 张 │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
每个 Pack 内的 AssetRow 显示状态徽章：
|
||||
- ✓ 绿色 = 已占用（来自上传图）
|
||||
- ✨ 紫色 = AI 已生成（通过自检）
|
||||
- 🔴 红色 = AI 生成但自检不过，建议重做
|
||||
- ⚠ 黄色 = 自检不确定，需人工确认
|
||||
- ⏳ 灰色 = 等待生成
|
||||
|
||||
点单张图可看详情：`anchor 来源 / prompt / 自检评分 / 漂移点`。
|
||||
|
||||
### 10.8 Agent 配置（环境变量补充）
|
||||
|
||||
```bash
|
||||
# Agent 并发度
|
||||
AGENT_CONCURRENCY=4 # 单 Wave 并行数
|
||||
AGENT_MAX_RETRY=2 # 自检失败最多重试次数
|
||||
AGENT_AUTO_REDO_THRESHOLD=0.7 # 自检分数低于此值自动重做
|
||||
|
||||
# Vision 模型
|
||||
GPT_VISION_MODEL=gpt-5.5 # 用于识图、自检
|
||||
```
|
||||
|
||||
### 10.9 失败恢复
|
||||
|
||||
Agent 跑到中途失败（API 超时、Key 限流）的处理：
|
||||
|
||||
- 每个 Wave 完成后**写一次 session.json 到 data/sessions/**
|
||||
- Wave 中单张失败 → 标记 `status: 'failed'`，记录错误，**不阻塞其它节点**
|
||||
- 用户刷新页面看到失败的 slot 显示红色，可一键重做
|
||||
- 全 Wave 完成后，Orchestrator 输出失败摘要
|
||||
|
||||
### 10.10 Agent 输入两种输入的对比
|
||||
|
||||
| 输入 | Vision 分析判定 | 走的路径 | 实际工作量 |
|
||||
|---|---|---|---|
|
||||
| **一张单主角图**（普通玩具照） | `single-subject` | Mode B 复刻 | L1 净化 → 推断 Spec → 全量补 ~114 张 + 文字 |
|
||||
| **lookbook 大图** | `lookbook` | Mode C 拆解+补全 | 切 30-40 块 → 自动占用 → 补 ~80 张 |
|
||||
| **多张参考图**（同一角色多视角） | `multi-reference` | 自动分发 + 复刻 | 已有视角占用 → 补缺失 |
|
||||
| **概念参考 + Prompt** | `multi-reference + prompt` | Mode A 二创 | 批量变体 → 选 → 复刻流程 |
|
||||
| **纯文字 prompt** | `prompt-only` | 原 prompt-first | 批量生意向图 → 选 → 复刻流程 |
|
||||
|
||||
无论哪种入口，都最终汇入同一个 **anchor-preparation → character-inference → pack-generation** 状态机，**输出统一**。
|
||||
|
||||
### 10.11 实施 Checklist 增量（在 §4 和 §8.10 基础上）
|
||||
|
||||
- [ ] 10.A 设计 `InputClassification` + `inferInputClassification` Vision 调用
|
||||
- [ ] 10.B 实现 `buildDAG` + `topologicalSort` 拓扑生成
|
||||
- [ ] 10.C 实现 `runGenerationLoop` 异步生成器（emit ProgressEvent）
|
||||
- [ ] 10.D 实现 `assessConsistency / assessViewAccuracy` 质量检查
|
||||
- [ ] 10.E `PackPanel` 顶部加总进度条 + 每张图状态徽章
|
||||
- [ ] 10.F session.json 增量写入（每 Wave 完成后保存）
|
||||
- [ ] 10.G 失败恢复 UI（红色 slot 一键重做）
|
||||
- [ ] 10.H 自动重做循环（自检不过 → 加 refinement → 最多重试 N 次）
|
||||
|
||||
### 10.12 推荐实施分期
|
||||
|
||||
**第 1 期：手动模式跑通**（不上 agent）
|
||||
- 完成 §1（真图生图）+ §8 Mode B（单图复刻）+ §9 lookbook 拆解
|
||||
- 用户手动点每个包的"生成"按钮
|
||||
- 没有自动拓扑、没有自检
|
||||
|
||||
**第 2 期：拓扑批量生成**
|
||||
- 完成 §10.5（buildDAG + topologicalSort）+ §10.C（runGenerationLoop）
|
||||
- 用户点一次"一键全包"，agent 按 wave 并行跑完
|
||||
- 还没有自检
|
||||
|
||||
**第 3 期：自检 + 自动重做**
|
||||
- 完成 §10.6 + §10.H
|
||||
- agent 自检不过的图自动重试 N 次
|
||||
|
||||
**第 4 期：完全自主 agent**
|
||||
- 完成 §10.A（InputClassification）+ 自动路径选择
|
||||
- 用户只需上传图，剩下全部 agent 自主完成
|
||||
- 用户只看进度条和最终结果
|
||||
|
||||
**建议**：先做完第 1+2 期，能覆盖 80% 场景；第 3+4 期是质量优化和体验升级，可以按用户反馈再迭代。
|
||||
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ export const runtime = 'nodejs';
|
||||
export const dynamic = 'force-dynamic';
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const { sessionId, imageId, force = false } = (await req.json()) as CleanupCharacterRequest;
|
||||
const { sessionId, imageId, force = false, preserveLevel = 'normal' } = (await req.json()) as CleanupCharacterRequest;
|
||||
|
||||
if (!sessionId || !imageId) {
|
||||
return NextResponse.json({ error: 'sessionId and imageId required' }, { status: 400 });
|
||||
@@ -23,7 +23,7 @@ export async function POST(req: Request) {
|
||||
const characterSpec = session.characterSpec?.sourceImageId === imageId
|
||||
? session.characterSpec
|
||||
: await buildCharacterSpec(session, sourceImage);
|
||||
const cleaned = await cleanupCharacterAnchor({ session, sourceImage, characterSpec, force });
|
||||
const cleaned = await cleanupCharacterAnchor({ session, sourceImage, characterSpec, force, preserveLevel });
|
||||
session.characterSpec = cleaned.characterSpec;
|
||||
await saveSession(session);
|
||||
|
||||
|
||||
52
src/app/api/character/lock-from-upload/route.ts
Normal file
52
src/app/api/character/lock-from-upload/route.ts
Normal file
@@ -0,0 +1,52 @@
|
||||
import { NextResponse } from 'next/server';
|
||||
import { buildCharacterSpec, cleanupCharacterAnchor } from '@/lib/packGenerator';
|
||||
import { detectProvider } from '@/lib/providers';
|
||||
import { loadSession, saveSession } from '@/lib/storage';
|
||||
import type { LockCharacterFromUploadRequest, LockCharacterResponse } from '@/lib/types';
|
||||
|
||||
export const runtime = 'nodejs';
|
||||
export const dynamic = 'force-dynamic';
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const { sessionId, subjectImageId, userHint, force = false } = (await req.json()) as LockCharacterFromUploadRequest;
|
||||
|
||||
if (!sessionId || !subjectImageId) {
|
||||
return NextResponse.json({ error: 'sessionId and subjectImageId required' }, { status: 400 });
|
||||
}
|
||||
|
||||
const session = await loadSession(sessionId);
|
||||
if (!session) return NextResponse.json({ error: 'session not found' }, { status: 404 });
|
||||
|
||||
const sourceImage = session.images.find(image =>
|
||||
image.id === subjectImageId || image.meta?.uploadedImageId === subjectImageId
|
||||
);
|
||||
if (!sourceImage) return NextResponse.json({ error: 'subject image not found' }, { status: 404 });
|
||||
|
||||
if (!force && session.characterSpec?.sourceImageId === sourceImage.id && session.characterSpec.cleanReferenceImageUrl) {
|
||||
return NextResponse.json({
|
||||
characterSpec: session.characterSpec,
|
||||
provider: detectProvider(),
|
||||
} satisfies LockCharacterResponse);
|
||||
}
|
||||
|
||||
try {
|
||||
if (userHint?.trim()) session.prompt = userHint.trim();
|
||||
const characterSpec = await buildCharacterSpec(session, sourceImage);
|
||||
const cleaned = await cleanupCharacterAnchor({
|
||||
session,
|
||||
sourceImage,
|
||||
characterSpec,
|
||||
force: true,
|
||||
preserveLevel: 'strict',
|
||||
});
|
||||
session.characterSpec = cleaned.characterSpec;
|
||||
await saveSession(session);
|
||||
|
||||
return NextResponse.json({
|
||||
characterSpec: cleaned.characterSpec,
|
||||
provider: cleaned.provider,
|
||||
} satisfies LockCharacterResponse);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: String(error) }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -1,17 +1,17 @@
|
||||
import { NextResponse } from 'next/server';
|
||||
import { readImageFile } from '@/lib/storage';
|
||||
import { readImageFile, type ImageBucket } from '@/lib/storage';
|
||||
|
||||
export const runtime = 'nodejs';
|
||||
|
||||
export async function GET(_req: Request, ctx: { params: Promise<{ bucket: string; filename: string }> }) {
|
||||
const { bucket, filename } = await ctx.params;
|
||||
if (!['generated', 'selected', 'refs', 'packs', 'anchors'].includes(bucket)) {
|
||||
if (!['generated', 'selected', 'refs', 'packs', 'anchors', 'uploads'].includes(bucket)) {
|
||||
return NextResponse.json({ error: 'bad bucket' }, { status: 400 });
|
||||
}
|
||||
if (filename.includes('..') || filename.includes('/')) {
|
||||
return NextResponse.json({ error: 'bad filename' }, { status: 400 });
|
||||
}
|
||||
const r = await readImageFile(bucket as 'generated' | 'selected' | 'refs' | 'packs' | 'anchors', filename);
|
||||
const r = await readImageFile(bucket as ImageBucket, filename);
|
||||
if (!r) return NextResponse.json({ error: 'not found' }, { status: 404 });
|
||||
return new NextResponse(new Uint8Array(r.buf), {
|
||||
headers: { 'Content-Type': r.type, 'Cache-Control': 'public, max-age=31536000, immutable' },
|
||||
|
||||
166
src/app/api/projects/from-upload/route.ts
Normal file
166
src/app/api/projects/from-upload/route.ts
Normal file
@@ -0,0 +1,166 @@
|
||||
import { NextResponse } from 'next/server';
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import { buildCharacterSpec, cleanupCharacterAnchor } from '@/lib/packGenerator';
|
||||
import { detectProvider, generateGptImageEdit, generateMock } from '@/lib/providers';
|
||||
import { saveGeneratedImage, saveSession } from '@/lib/storage';
|
||||
import type {
|
||||
GenImage,
|
||||
GenSession,
|
||||
PreFilledSlot,
|
||||
ProjectFromUploadRequest,
|
||||
ProjectFromUploadResponse,
|
||||
UploadedImage,
|
||||
} from '@/lib/types';
|
||||
|
||||
export const runtime = 'nodejs';
|
||||
export const dynamic = 'force-dynamic';
|
||||
|
||||
const VIEW_SLOT: Partial<Record<UploadedImage['role'], string>> = {
|
||||
'view-front': 'patent_front',
|
||||
'view-back': 'patent_back',
|
||||
'view-left': 'patent_left',
|
||||
'view-right': 'patent_right',
|
||||
'view-top': 'patent_top',
|
||||
'view-bottom': 'patent_bottom',
|
||||
};
|
||||
|
||||
function clampCount(count: unknown): number {
|
||||
const n = Number(count);
|
||||
if (n === 4 || n === 8 || n === 12) return n;
|
||||
return 8;
|
||||
}
|
||||
|
||||
function assertUpload(image: UploadedImage | undefined): UploadedImage {
|
||||
if (!image) throw new Error('subject upload required');
|
||||
if (!image.url.startsWith('/api/img/uploads/')) throw new Error('uploaded image URL must point to uploads bucket');
|
||||
return image;
|
||||
}
|
||||
|
||||
function preFilledSlotsFromUploads(images: UploadedImage[]): PreFilledSlot[] {
|
||||
return images.flatMap(image => {
|
||||
const templateId = VIEW_SLOT[image.role];
|
||||
if (!templateId) return [];
|
||||
return [{
|
||||
uploadedImageId: image.id,
|
||||
templateId,
|
||||
role: image.role,
|
||||
url: image.url,
|
||||
}];
|
||||
});
|
||||
}
|
||||
|
||||
async function createRemixSession(body: ProjectFromUploadRequest, sessionId: string): Promise<ProjectFromUploadResponse> {
|
||||
const reference = assertUpload(body.uploadedImages[0]);
|
||||
const count = clampCount(body.count);
|
||||
const prompt = [
|
||||
body.remixPrompt?.trim() || '基于上传参考图生成原创玩具风格变体',
|
||||
body.styleId ? `风格 ID：${body.styleId}` : '',
|
||||
'保留主体轮廓、五官相对位置、配件轮廓和核心识别点；可以改变材质、色彩和整体风格。',
|
||||
'避免直接复刻迪士尼、三丽鸥、泡泡玛特等已注册 IP；生成结果必须偏向原创玩具设计。',
|
||||
].filter(Boolean).join('\n');
|
||||
const provider = detectProvider();
|
||||
|
||||
const rawImages = provider === 'gpt'
|
||||
? await Promise.all(Array.from({ length: count }).map(() => generateGptImageEdit({
|
||||
sessionId,
|
||||
prompt,
|
||||
anchorImage: reference.url,
|
||||
size: '1024x1024',
|
||||
})))
|
||||
: await generateMock({ sessionId, prompt, count });
|
||||
|
||||
const images = await Promise.all(rawImages.map(async (image, index) => {
|
||||
const id = `img_${sessionId}_${index}`;
|
||||
const url = image.url.startsWith('data:') ? await saveGeneratedImage(sessionId, id, image.url) : image.url;
|
||||
return {
|
||||
...image,
|
||||
id,
|
||||
url,
|
||||
prompt,
|
||||
meta: { ...(image.meta ?? {}), mode: 'remix', uploadedImageId: reference.id },
|
||||
};
|
||||
}));
|
||||
|
||||
const session: GenSession = {
|
||||
id: sessionId,
|
||||
createdAt: Date.now(),
|
||||
prompt,
|
||||
refImages: body.uploadedImages.map(image => image.url),
|
||||
count,
|
||||
inputMode: 'remix',
|
||||
uploadedImages: body.uploadedImages,
|
||||
images,
|
||||
};
|
||||
await saveSession(session);
|
||||
return { sessionId, images, provider };
|
||||
}
|
||||
|
||||
async function createReplicateOrExtendSession(body: ProjectFromUploadRequest, sessionId: string): Promise<ProjectFromUploadResponse> {
|
||||
const subject = assertUpload(body.uploadedImages.find(image => image.role === 'subject') ?? body.uploadedImages[0]);
|
||||
const prompt = body.userHint?.trim() || body.remixPrompt?.trim() || subject.originalFilename || '复刻上传主体玩具';
|
||||
const preFilledSlots = body.mode === 'extend' ? preFilledSlotsFromUploads(body.uploadedImages) : [];
|
||||
const sourceImage: GenImage = {
|
||||
id: `img_${sessionId}_upload_l0`,
|
||||
url: subject.url,
|
||||
prompt,
|
||||
status: 'selected',
|
||||
meta: {
|
||||
provider: 'upload',
|
||||
source: 'upload',
|
||||
mode: body.mode,
|
||||
uploadedImageId: subject.id,
|
||||
uploadRole: subject.role,
|
||||
},
|
||||
};
|
||||
const session: GenSession = {
|
||||
id: sessionId,
|
||||
createdAt: Date.now(),
|
||||
prompt,
|
||||
refImages: body.uploadedImages.map(image => image.url),
|
||||
count: 1,
|
||||
inputMode: body.mode,
|
||||
uploadedImages: body.uploadedImages,
|
||||
preFilledSlots,
|
||||
images: [sourceImage],
|
||||
};
|
||||
|
||||
const characterSpec = await buildCharacterSpec(session, sourceImage);
|
||||
const cleaned = await cleanupCharacterAnchor({
|
||||
session,
|
||||
sourceImage,
|
||||
characterSpec,
|
||||
force: true,
|
||||
preserveLevel: 'strict',
|
||||
});
|
||||
session.characterSpec = cleaned.characterSpec;
|
||||
await saveSession(session);
|
||||
|
||||
return {
|
||||
sessionId,
|
||||
characterSpec: cleaned.characterSpec,
|
||||
l1AnchorUrl: cleaned.cleanReferenceImageUrl,
|
||||
preFilledSlots,
|
||||
provider: cleaned.provider,
|
||||
};
|
||||
}
|
||||
|
||||
export async function POST(req: Request) {
|
||||
try {
|
||||
const body = (await req.json()) as ProjectFromUploadRequest;
|
||||
if (!Array.isArray(body.uploadedImages) || body.uploadedImages.length === 0) {
|
||||
return NextResponse.json({ error: 'uploadedImages required' }, { status: 400 });
|
||||
}
|
||||
if (!['remix', 'replicate', 'extend'].includes(body.mode)) {
|
||||
return NextResponse.json({ error: 'valid mode required' }, { status: 400 });
|
||||
}
|
||||
|
||||
const sessionId = `s_${Date.now().toString(36)}_${randomBytes(3).toString('hex')}`;
|
||||
const response = body.mode === 'remix'
|
||||
? await createRemixSession(body, sessionId)
|
||||
: await createReplicateOrExtendSession(body, sessionId);
|
||||
|
||||
return NextResponse.json(response satisfies ProjectFromUploadResponse);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: String(error) }, { status: 500 });
|
||||
}
|
||||
}
|
||||
53
src/app/api/uploads/route.ts
Normal file
53
src/app/api/uploads/route.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
import { NextResponse } from 'next/server';
|
||||
import { saveUploadedImage } from '@/lib/storage';
|
||||
import type { UploadImageResponse, UploadedImageRole } from '@/lib/types';
|
||||
|
||||
export const runtime = 'nodejs';
|
||||
export const dynamic = 'force-dynamic';
|
||||
|
||||
const ROLES: UploadedImageRole[] = [
|
||||
'reference',
|
||||
'subject',
|
||||
'view-front',
|
||||
'view-back',
|
||||
'view-left',
|
||||
'view-right',
|
||||
'view-top',
|
||||
'view-bottom',
|
||||
'accessory-isolated',
|
||||
'accessory-named',
|
||||
];
|
||||
|
||||
function isFileLike(value: FormDataEntryValue | null): value is File {
|
||||
return !!value && typeof value === 'object' && 'arrayBuffer' in value && 'type' in value && 'name' in value;
|
||||
}
|
||||
|
||||
export async function POST(req: Request) {
|
||||
const form = await req.formData();
|
||||
const file = form.get('image') ?? form.get('file');
|
||||
const roleValue = String(form.get('role') ?? 'reference');
|
||||
const role = ROLES.includes(roleValue as UploadedImageRole) ? roleValue as UploadedImageRole : 'reference';
|
||||
const accessoryName = String(form.get('accessoryName') ?? '').trim() || undefined;
|
||||
const needsCleanup = String(form.get('needsCleanup') ?? 'true') !== 'false';
|
||||
|
||||
if (!isFileLike(file)) {
|
||||
return NextResponse.json({ error: 'image file required' }, { status: 400 });
|
||||
}
|
||||
if (!file.type.startsWith('image/')) {
|
||||
return NextResponse.json({ error: 'only image uploads are supported' }, { status: 400 });
|
||||
}
|
||||
if (file.size > 12 * 1024 * 1024) {
|
||||
return NextResponse.json({ error: 'image must be <= 12MB' }, { status: 400 });
|
||||
}
|
||||
|
||||
const uploadedImage = await saveUploadedImage({
|
||||
buffer: Buffer.from(await file.arrayBuffer()),
|
||||
mimeType: file.type,
|
||||
originalFilename: file.name,
|
||||
role,
|
||||
accessoryName,
|
||||
needsCleanup,
|
||||
});
|
||||
|
||||
return NextResponse.json({ uploadedImage } satisfies UploadImageResponse);
|
||||
}
|
||||
@@ -1,153 +0,0 @@
|
||||
'use client';
|
||||
|
||||
import { useRef, useState } from 'react';
|
||||
|
||||
const PRESET_STYLES = [
|
||||
{ id: 'plush', label: '毛绒玩偶' },
|
||||
{ id: 'mecha', label: '机甲风' },
|
||||
{ id: 'kawaii', label: '可爱萌系' },
|
||||
{ id: 'blueprint', label: '专利蓝图' },
|
||||
{ id: 'cyber', label: '赛博朋克' },
|
||||
{ id: 'minimal', label: '极简' },
|
||||
];
|
||||
|
||||
export type PromptPanelProps = {
|
||||
onGenerate: (opts: { prompt: string; refImages: string[]; count: number; style?: string }) => void;
|
||||
loading: boolean;
|
||||
};
|
||||
|
||||
export default function PromptPanel({ onGenerate, loading }: PromptPanelProps) {
|
||||
const [prompt, setPrompt] = useState('AI 毛绒陪伴玩具，机甲头盔，胸前挂 M logo，橙白配色，圆胖体型');
|
||||
const [refs, setRefs] = useState<string[]>([]);
|
||||
const [count, setCount] = useState(8);
|
||||
const [style, setStyle] = useState<string>('');
|
||||
const fileInput = useRef<HTMLInputElement>(null);
|
||||
|
||||
function handleFiles(files: FileList | null) {
|
||||
if (!files) return;
|
||||
Array.from(files).slice(0, 4 - refs.length).forEach(f => {
|
||||
const r = new FileReader();
|
||||
r.onload = () => setRefs(prev => [...prev, r.result as string]);
|
||||
r.readAsDataURL(f);
|
||||
});
|
||||
}
|
||||
|
||||
function submit() {
|
||||
if (!prompt.trim() || loading) return;
|
||||
onGenerate({ prompt: prompt.trim(), refImages: refs, count, style: style || undefined });
|
||||
}
|
||||
|
||||
return (
|
||||
<section className="card p-7 space-y-6">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="section-eyebrow">Step · 01 · Ideation</span>
|
||||
<span className="text-[10px] text-white/30">· 描述意向 + 数量 + 风格</span>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className="block text-[11px] font-medium text-white/55 uppercase tracking-[0.14em] mb-2.5">
|
||||
Prompt
|
||||
</label>
|
||||
<textarea
|
||||
value={prompt}
|
||||
onChange={e => setPrompt(e.target.value)}
|
||||
onKeyDown={e => { if (e.key === 'Enter' && (e.metaKey || e.ctrlKey)) submit(); }}
|
||||
rows={3}
|
||||
placeholder="描述要生成的玩具意向…"
|
||||
className="field text-[15px] leading-relaxed"
|
||||
/>
|
||||
<p className="mt-2 text-[11px] text-white/35 flex items-center gap-1.5">
|
||||
按 <kbd className="kbd">⌘</kbd><kbd className="kbd">↵</kbd> 提交
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className="block text-[11px] font-medium text-white/55 uppercase tracking-[0.14em] mb-2.5">
|
||||
参考图 <span className="text-white/35 normal-case tracking-normal">· 可选，最多 4 张</span>
|
||||
</label>
|
||||
<div className="flex flex-wrap gap-2.5">
|
||||
{refs.map((r, i) => (
|
||||
<div key={i} className="relative w-20 h-20 rounded-xl overflow-hidden ring-1 ring-white/[0.1] group">
|
||||
<img src={r} alt="ref" className="w-full h-full object-cover" />
|
||||
<button
|
||||
onClick={() => setRefs(prev => prev.filter((_, j) => j !== i))}
|
||||
className="absolute top-1 right-1 w-5 h-5 rounded-full bg-black/80 text-white text-[10px] opacity-0 group-hover:opacity-100 transition-opacity shadow-md"
|
||||
>✕</button>
|
||||
</div>
|
||||
))}
|
||||
{refs.length < 4 && (
|
||||
<button
|
||||
onClick={() => fileInput.current?.click()}
|
||||
className="w-20 h-20 rounded-xl border-2 border-dashed border-white/15 hover:border-violet-400/50 hover:bg-white/[0.03] text-white/30 hover:text-violet-300 text-2xl transition-all flex items-center justify-center"
|
||||
>+</button>
|
||||
)}
|
||||
<input
|
||||
ref={fileInput}
|
||||
type="file"
|
||||
accept="image/*"
|
||||
multiple
|
||||
hidden
|
||||
onChange={e => handleFiles(e.target.files)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className="block text-[11px] font-medium text-white/55 uppercase tracking-[0.14em] mb-2.5">
|
||||
风格
|
||||
</label>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
<button
|
||||
onClick={() => setStyle('')}
|
||||
className={style === '' ? 'btn btn-primary text-xs px-3 py-1.5' : 'btn btn-outline text-xs px-3 py-1.5'}
|
||||
>无</button>
|
||||
{PRESET_STYLES.map(s => (
|
||||
<button
|
||||
key={s.id}
|
||||
onClick={() => setStyle(s.label)}
|
||||
className={style === s.label ? 'btn btn-primary text-xs px-3 py-1.5' : 'btn btn-outline text-xs px-3 py-1.5'}
|
||||
>{s.label}</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex items-end justify-between gap-4 pt-2">
|
||||
<div>
|
||||
<label className="block text-[11px] font-medium text-white/55 uppercase tracking-[0.14em] mb-2.5">
|
||||
数量
|
||||
</label>
|
||||
<div className="seg">
|
||||
{[4, 8, 12].map(n => (
|
||||
<button
|
||||
key={n}
|
||||
onClick={() => setCount(n)}
|
||||
className={`seg-item ${count === n ? 'seg-item-active' : ''}`}
|
||||
>{n} 张</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
<button
|
||||
onClick={submit}
|
||||
disabled={loading || !prompt.trim()}
|
||||
className="btn btn-primary px-5 py-2.5 disabled:opacity-40 disabled:cursor-not-allowed"
|
||||
>
|
||||
{loading ? (
|
||||
<>
|
||||
<svg width="14" height="14" viewBox="0 0 24 24" className="animate-spin" fill="none" stroke="currentColor" strokeWidth="2">
|
||||
<path d="M12 2a10 10 0 0 1 10 10" strokeLinecap="round" />
|
||||
</svg>
|
||||
生成中
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
批量生成
|
||||
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2.5">
|
||||
<path d="M5 12h14M13 5l7 7-7 7" strokeLinecap="round" strokeLinejoin="round" />
|
||||
</svg>
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -8,7 +8,7 @@ import type {
|
||||
PackKind,
|
||||
ToyAsset,
|
||||
} from './types';
|
||||
import { detectProvider, generateGptImageEdit, generateGptJson, generateMock } from './providers';
|
||||
import { detectProvider, generateGptImageEdit, generateGptJson, generateMock, inferCharacterSpecFromImage } from './providers';
|
||||
import { saveAnchorImage, saveExportManifest, savePackImage } from './storage';
|
||||
import { FILENAME_SCHEMA, getPackTemplates, PACK_LABELS, renderCharacterSummary, TEMPLATE_FREEZE_VERSION } from './templates';
|
||||
|
||||
@@ -57,9 +57,50 @@ function buildFallbackCharacterSpec(session: GenSession, sourceImage: GenImage):
|
||||
};
|
||||
}
|
||||
|
||||
function asStringArray(value: unknown, fallback: string[]): string[] {
|
||||
if (Array.isArray(value)) {
|
||||
const items = value.map(item => String(item).trim()).filter(Boolean);
|
||||
return items.length > 0 ? items : fallback;
|
||||
}
|
||||
if (typeof value === 'string' && value.trim()) return [value.trim()];
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function normalizeCharacterSpec(spec: CharacterSpec, fallback: CharacterSpec, sourceImage: GenImage): CharacterSpec {
|
||||
return {
|
||||
...fallback,
|
||||
...spec,
|
||||
name: spec.name || fallback.name,
|
||||
oneLiner: spec.oneLiner || fallback.oneLiner,
|
||||
targetUser: spec.targetUser || fallback.targetUser,
|
||||
speciesShape: spec.speciesShape || fallback.speciesShape,
|
||||
bodyRatio: spec.bodyRatio || fallback.bodyRatio,
|
||||
faceFeatures: spec.faceFeatures || fallback.faceFeatures,
|
||||
colorPalette: asStringArray(spec.colorPalette, fallback.colorPalette),
|
||||
materials: asStringArray(spec.materials, fallback.materials),
|
||||
accessories: asStringArray(spec.accessories, fallback.accessories),
|
||||
signatureElements: asStringArray(spec.signatureElements, fallback.signatureElements),
|
||||
manufacturingNotes: asStringArray(spec.manufacturingNotes, fallback.manufacturingNotes),
|
||||
patentFocus: asStringArray(spec.patentFocus, fallback.patentFocus),
|
||||
marketingAngle: asStringArray(spec.marketingAngle, fallback.marketingAngle),
|
||||
negativePrompt: spec.negativePrompt || fallback.negativePrompt,
|
||||
sourceImageId: sourceImage.id,
|
||||
sourceImageUrl: sourceImage.url,
|
||||
lockedAt: typeof spec.lockedAt === 'number' ? spec.lockedAt : Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
export async function buildCharacterSpec(session: GenSession, sourceImage: GenImage): Promise<CharacterSpec> {
|
||||
const fallback = buildFallbackCharacterSpec(session, sourceImage);
|
||||
return generateGptJson<CharacterSpec>({
|
||||
if (session.inputMode === 'replicate' || session.inputMode === 'extend' || sourceImage.meta?.source === 'upload') {
|
||||
const inferred = await inferCharacterSpecFromImage({
|
||||
imageUrl: sourceImage.url,
|
||||
userHint: session.prompt,
|
||||
fallback,
|
||||
});
|
||||
return normalizeCharacterSpec(inferred, fallback, sourceImage);
|
||||
}
|
||||
const generated = await generateGptJson<CharacterSpec>({
|
||||
fallback,
|
||||
prompt: [
|
||||
'你是资深毛绒玩具产品经理、外观专利素材规划师和工厂打样顾问。',
|
||||
@@ -73,6 +114,7 @@ export async function buildCharacterSpec(session: GenSession, sourceImage: GenIm
|
||||
`兜底 JSON：${JSON.stringify(fallback)}`,
|
||||
].join('\n'),
|
||||
});
|
||||
return normalizeCharacterSpec(generated, fallback, sourceImage);
|
||||
}
|
||||
|
||||
function renderPrompt(template: string, spec: CharacterSpec, sourceImageUrl: string): string {
|
||||
@@ -114,6 +156,7 @@ export async function cleanupCharacterAnchor(opts: {
|
||||
sourceImage: GenImage;
|
||||
characterSpec?: CharacterSpec;
|
||||
force?: boolean;
|
||||
preserveLevel?: 'normal' | 'strict';
|
||||
}): Promise<{ characterSpec: CharacterSpec; cleanReferenceImageUrl: string; provider: 'mock' | 'gpt' }> {
|
||||
const provider = detectProvider();
|
||||
const characterSpec = opts.characterSpec ?? opts.session.characterSpec ?? await buildCharacterSpec(opts.session, opts.sourceImage);
|
||||
@@ -122,7 +165,23 @@ export async function cleanupCharacterAnchor(opts: {
|
||||
return { characterSpec, cleanReferenceImageUrl: characterSpec.cleanReferenceImageUrl, provider };
|
||||
}
|
||||
|
||||
const prompt = [
|
||||
const strictPrompt = [
|
||||
'保持原图完全一致，仅做以下修改：',
|
||||
'1. 把背景换成纯白色',
|
||||
'2. 去除任何水印、文字、价格标签、网页 UI 元素',
|
||||
'3. 居中并适当裁剪到正方形构图',
|
||||
'',
|
||||
'绝对不要修改：',
|
||||
'- 角色五官、表情、姿态',
|
||||
'- 主体配色、材质、纹理',
|
||||
'- 配件位置、轮廓、细节',
|
||||
'- 任何品牌符号或识别符号',
|
||||
'',
|
||||
'输出风格：商业产品图，柔和均匀打光，无阴影。',
|
||||
`角色设定：${renderCharacterSummary(characterSpec)}`,
|
||||
].join('\n');
|
||||
|
||||
const normalPrompt = [
|
||||
'保持参考图中的玩具角色完全一致，只做产品图净化。',
|
||||
'把背景换成纯白色，产品居中，正面或轻微正面视角，光线均匀。',
|
||||
'不要改变五官、主配色、身体比例、毛绒材质、核心配件和识别元素。',
|
||||
@@ -130,6 +189,8 @@ export async function cleanupCharacterAnchor(opts: {
|
||||
`角色设定：${renderCharacterSummary(characterSpec)}`,
|
||||
].join('\n');
|
||||
|
||||
const prompt = opts.preserveLevel === 'strict' ? strictPrompt : normalPrompt;
|
||||
|
||||
const image = provider === 'gpt'
|
||||
? await generateGptImageEdit({
|
||||
sessionId: `${opts.session.id}_clean`,
|
||||
@@ -233,6 +294,39 @@ export async function generateAssetPack(opts: {
|
||||
}
|
||||
const anchorImageUrl = anchorAsset?.url ?? resolveRootAnchor(characterSpec, opts.sourceImage);
|
||||
const prompt = renderPrompt(template.promptTemplate, characterSpec, anchorImageUrl);
|
||||
const preFilledSlot = opts.session.preFilledSlots?.find(slot => slot.templateId === template.id);
|
||||
if (preFilledSlot) {
|
||||
assets.push({
|
||||
id: assetId,
|
||||
templateId: template.id,
|
||||
kind: opts.kind,
|
||||
view: template.view,
|
||||
title: template.title,
|
||||
description: template.description,
|
||||
url: preFilledSlot.url,
|
||||
prompt: [
|
||||
`用户上传图已占用槽位：${template.id}`,
|
||||
prompt,
|
||||
].join('\n'),
|
||||
status: 'draft',
|
||||
version,
|
||||
aspectRatio: template.aspectRatio,
|
||||
required: template.required,
|
||||
createdAt: Date.now(),
|
||||
anchorAssetId: anchorAsset?.id,
|
||||
anchorImageUrl,
|
||||
derivationLevel: anchorAsset ? 3 : 2,
|
||||
meta: {
|
||||
provider: 'upload',
|
||||
uploadedImageId: preFilledSlot.uploadedImageId,
|
||||
uploadRole: preFilledSlot.role,
|
||||
packLabel: PACK_LABELS[opts.kind],
|
||||
templateFreezeVersion: TEMPLATE_FREEZE_VERSION,
|
||||
anchorTemplateId: template.anchorTemplateId,
|
||||
},
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const generated = await generateAssetImage({
|
||||
packId,
|
||||
assetId,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { GenImage } from './types';
|
||||
import type { CharacterSpec, GenImage } from './types';
|
||||
import { readImageUrl } from './storage';
|
||||
|
||||
export type Provider = 'mock' | 'gpt';
|
||||
@@ -102,6 +102,13 @@ function dataUrlFromImageResponse(payload: unknown): string {
|
||||
return `data:image/png;base64,${readEditImageBase64(payload)}`;
|
||||
}
|
||||
|
||||
function readResponseText(data: {
|
||||
output_text?: string;
|
||||
output?: Array<{ content?: Array<{ text?: string }> }>;
|
||||
}): string {
|
||||
return data.output_text || data.output?.flatMap(item => item.content ?? []).map(item => item.text ?? '').join('') || '';
|
||||
}
|
||||
|
||||
export async function generateGptImageEdit(opts: {
|
||||
sessionId: string;
|
||||
prompt: string;
|
||||
@@ -173,6 +180,92 @@ export async function generateGptJson<T>(opts: {
|
||||
output_text?: string;
|
||||
output?: Array<{ content?: Array<{ text?: string }> }>;
|
||||
};
|
||||
const text = data.output_text || data.output?.flatMap(item => item.content ?? []).map(item => item.text ?? '').join('') || '';
|
||||
const text = readResponseText(data);
|
||||
return text.trim() ? JSON.parse(text) as T : opts.fallback;
|
||||
}
|
||||
|
||||
function genericCharacterSpec(imageUrl: string, userHint?: string): CharacterSpec {
|
||||
const now = Date.now();
|
||||
return {
|
||||
name: userHint?.trim() || '上传主体玩具',
|
||||
oneLiner: userHint?.trim() || '基于用户上传主体图复刻的玩具 IP',
|
||||
targetUser: '潮玩/礼品/品牌周边用户',
|
||||
speciesShape: '根据上传图识别的玩具主体形态',
|
||||
bodyRatio: '保持上传图中的头身比例、姿态和整体轮廓',
|
||||
faceFeatures: '保持上传图中的五官、表情和识别特征',
|
||||
colorPalette: ['按上传图原始配色保留'],
|
||||
materials: ['按上传图材质推断，后续由人工确认'],
|
||||
accessories: ['按上传图可见配件保留'],
|
||||
signatureElements: ['上传图中的主体轮廓', '上传图中的五官组合', '上传图中的核心配件和标识'],
|
||||
manufacturingNotes: ['以原图为外观基准，尺寸、材料和工艺待人工确认'],
|
||||
patentFocus: ['整体轮廓', '五官组合', '配色关系', '配件造型'],
|
||||
marketingAngle: ['复刻原始角色外观', '扩展为专利/生产/宣发素材包'],
|
||||
negativePrompt: '不要改变上传图的角色五官、表情、姿态、配色、材质纹理、配件位置和品牌符号',
|
||||
sourceImageUrl: imageUrl,
|
||||
lockedAt: now,
|
||||
};
|
||||
}
|
||||
|
||||
async function imageUrlToVisionInput(url: string): Promise<{ image_url: string } | null> {
|
||||
const local = /^\/api\/img\//.test(url) || /^data:/i.test(url);
|
||||
if (!local && /^https?:\/\//i.test(url)) return { image_url: url };
|
||||
|
||||
const image = await readImageUrl(url);
|
||||
if (image.type.includes('svg')) return null;
|
||||
return {
|
||||
image_url: `data:${image.type};base64,${image.buf.toString('base64')}`,
|
||||
};
|
||||
}
|
||||
|
||||
export async function inferCharacterSpecFromImage(opts: {
|
||||
imageUrl: string;
|
||||
userHint?: string;
|
||||
fallback?: CharacterSpec;
|
||||
}): Promise<CharacterSpec> {
|
||||
const fallback = opts.fallback ?? genericCharacterSpec(opts.imageUrl, opts.userHint);
|
||||
const key = process.env.OPENAI_API_KEY;
|
||||
if (!key) return fallback;
|
||||
|
||||
const imageInput = await imageUrlToVisionInput(opts.imageUrl);
|
||||
if (!imageInput) return fallback;
|
||||
|
||||
const prompt = [
|
||||
'你是玩具产品经理、外观专利素材规划师和工厂打样顾问。',
|
||||
'根据上传图片推断 CharacterSpec，严格输出 JSON，不要 markdown，不要解释。',
|
||||
'字段必须完整匹配：name, oneLiner, targetUser, speciesShape, bodyRatio, faceFeatures, colorPalette, materials, accessories, signatureElements, manufacturingNotes, patentFocus, marketingAngle, negativePrompt, sourceImageId, sourceImageUrl, lockedAt。',
|
||||
'数组字段必须为字符串数组。',
|
||||
'不要把已知商业 IP 当成可用授权素材；若图像疑似迪士尼、三丽鸥、泡泡玛特等已注册 IP，在 negativePrompt 中明确提示需替换为原创元素。',
|
||||
opts.userHint?.trim() ? `用户提示：${opts.userHint.trim()}` : '用户没有提供命名提示，请根据图像生成一个中性原创名称。',
|
||||
`当前时间戳：${Date.now()}`,
|
||||
`源图 URL：${opts.imageUrl}`,
|
||||
`兜底 JSON：${JSON.stringify(fallback)}`,
|
||||
].join('\n');
|
||||
|
||||
const res = await fetch(`${GPT_API_BASE}/responses`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Authorization: `Bearer ${key}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: GPT_TEXT_MODEL,
|
||||
input: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'input_text', text: prompt },
|
||||
{ type: 'input_image', image_url: imageInput.image_url },
|
||||
],
|
||||
},
|
||||
],
|
||||
text: { format: { type: 'json_object' } },
|
||||
}),
|
||||
});
|
||||
if (!res.ok) throw new Error(`GPT vision ${res.status}: ${await res.text()}`);
|
||||
const data = await res.json() as {
|
||||
output_text?: string;
|
||||
output?: Array<{ content?: Array<{ text?: string }> }>;
|
||||
};
|
||||
const text = readResponseText(data);
|
||||
return text.trim() ? JSON.parse(text) as CharacterSpec : fallback;
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { promises as fs } from 'node:fs';
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import path from 'node:path';
|
||||
import type { ExportManifest, GenSession } from './types';
|
||||
import type { ExportManifest, GenSession, UploadedImage, UploadedImageRole } from './types';
|
||||
|
||||
const ROOT = path.join(process.cwd(), 'data');
|
||||
const SESS_DIR = path.join(ROOT, 'sessions');
|
||||
@@ -9,10 +10,11 @@ const REF_DIR = path.join(ROOT, 'refs');
|
||||
const GEN_DIR = path.join(ROOT, 'generated');
|
||||
const PACK_DIR = path.join(ROOT, 'packs');
|
||||
const ANCHOR_DIR = path.join(ROOT, 'anchors');
|
||||
const UPLOAD_DIR = path.join(ROOT, 'uploads');
|
||||
const EXPORT_DIR = path.join(ROOT, 'exports');
|
||||
|
||||
async function ensureDirs() {
|
||||
await Promise.all([SESS_DIR, SEL_DIR, REF_DIR, GEN_DIR, PACK_DIR, ANCHOR_DIR, EXPORT_DIR].map(d => fs.mkdir(d, { recursive: true })));
|
||||
await Promise.all([SESS_DIR, SEL_DIR, REF_DIR, GEN_DIR, PACK_DIR, ANCHOR_DIR, UPLOAD_DIR, EXPORT_DIR].map(d => fs.mkdir(d, { recursive: true })));
|
||||
}
|
||||
|
||||
export async function saveSession(s: GenSession) {
|
||||
@@ -49,6 +51,14 @@ function extFromMime(mime: string): string {
|
||||
return 'bin';
|
||||
}
|
||||
|
||||
function safePart(input: string): string {
|
||||
return input
|
||||
.normalize('NFKD')
|
||||
.replace(/[^a-zA-Z0-9._-]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
.slice(0, 60) || 'image';
|
||||
}
|
||||
|
||||
export async function saveGeneratedImage(sessionId: string, imageId: string, dataUrl: string): Promise<string> {
|
||||
await ensureDirs();
|
||||
const m = dataUrl.match(/^data:([^;]+);base64,(.+)$/);
|
||||
@@ -80,6 +90,34 @@ export async function saveAnchorImage(sessionId: string, imageId: string, dataUr
|
||||
return `/api/img/anchors/${filename}`;
|
||||
}
|
||||
|
||||
export async function saveUploadedImage(opts: {
|
||||
buffer: Buffer;
|
||||
mimeType: string;
|
||||
originalFilename?: string;
|
||||
role: UploadedImageRole;
|
||||
accessoryName?: string;
|
||||
needsCleanup?: boolean;
|
||||
}): Promise<UploadedImage> {
|
||||
await ensureDirs();
|
||||
const uploadedAt = Date.now();
|
||||
const id = `upl_${uploadedAt.toString(36)}_${randomBytes(3).toString('hex')}`;
|
||||
const ext = extFromMime(opts.mimeType);
|
||||
const baseName = opts.originalFilename ? safePart(path.parse(opts.originalFilename).name) : 'upload';
|
||||
const filename = `${id}_${baseName}.${ext}`;
|
||||
await fs.writeFile(path.join(UPLOAD_DIR, filename), opts.buffer);
|
||||
return {
|
||||
id,
|
||||
url: `/api/img/uploads/${filename}`,
|
||||
filename,
|
||||
originalFilename: opts.originalFilename,
|
||||
mimeType: opts.mimeType,
|
||||
uploadedAt,
|
||||
role: opts.role,
|
||||
accessoryName: opts.accessoryName,
|
||||
needsCleanup: opts.needsCleanup ?? true,
|
||||
};
|
||||
}
|
||||
|
||||
export async function copyToSelected(sessionId: string, imageId: string, srcUrl: string): Promise<string> {
|
||||
await ensureDirs();
|
||||
// srcUrl 形如 /api/img/generated/xxx.png
|
||||
@@ -92,13 +130,16 @@ export async function copyToSelected(sessionId: string, imageId: string, srcUrl:
|
||||
return `/api/img/selected/${filename}`;
|
||||
}
|
||||
|
||||
export async function readImageFile(bucket: 'generated' | 'selected' | 'refs' | 'packs' | 'anchors', filename: string): Promise<{ buf: Buffer; type: string } | null> {
|
||||
export type ImageBucket = 'generated' | 'selected' | 'refs' | 'packs' | 'anchors' | 'uploads';
|
||||
|
||||
export async function readImageFile(bucket: ImageBucket, filename: string): Promise<{ buf: Buffer; type: string } | null> {
|
||||
try {
|
||||
const dir = bucket === 'generated' ? GEN_DIR
|
||||
: bucket === 'selected' ? SEL_DIR
|
||||
: bucket === 'refs' ? REF_DIR
|
||||
: bucket === 'packs' ? PACK_DIR
|
||||
: ANCHOR_DIR;
|
||||
: bucket === 'anchors' ? ANCHOR_DIR
|
||||
: UPLOAD_DIR;
|
||||
const buf = await fs.readFile(path.join(dir, filename));
|
||||
const ext = path.extname(filename).slice(1).toLowerCase();
|
||||
const type = ext === 'jpg' ? 'image/jpeg'
|
||||
@@ -120,9 +161,9 @@ export async function readImageUrl(url: string): Promise<{ buf: Buffer; type: st
|
||||
};
|
||||
}
|
||||
|
||||
const localMatch = url.match(/^\/api\/img\/(generated|selected|refs|packs|anchors)\/([^/?#]+)$/);
|
||||
const localMatch = url.match(/^\/api\/img\/(generated|selected|refs|packs|anchors|uploads)\/([^/?#]+)$/);
|
||||
if (localMatch) {
|
||||
const bucket = localMatch[1] as 'generated' | 'selected' | 'refs' | 'packs' | 'anchors';
|
||||
const bucket = localMatch[1] as ImageBucket;
|
||||
const filename = decodeURIComponent(localMatch[2]);
|
||||
const image = await readImageFile(bucket, filename);
|
||||
if (!image) throw new Error(`anchor image not found: ${url}`);
|
||||
|
||||
@@ -5,6 +5,9 @@ export type GenSession = {
|
||||
refImages: string[];
|
||||
count: number;
|
||||
images: GenImage[];
|
||||
inputMode?: ProjectInputMode;
|
||||
uploadedImages?: UploadedImage[];
|
||||
preFilledSlots?: PreFilledSlot[];
|
||||
characterSpec?: CharacterSpec;
|
||||
packs?: AssetPack[];
|
||||
exports?: ExportManifest[];
|
||||
@@ -31,6 +34,39 @@ export type GenerateResponse = {
|
||||
provider: 'mock' | 'gpt';
|
||||
};
|
||||
|
||||
export type ProjectInputMode = 'idea' | 'remix' | 'replicate' | 'extend';
|
||||
|
||||
export type UploadedImageRole =
|
||||
| 'reference'
|
||||
| 'subject'
|
||||
| 'view-front'
|
||||
| 'view-back'
|
||||
| 'view-left'
|
||||
| 'view-right'
|
||||
| 'view-top'
|
||||
| 'view-bottom'
|
||||
| 'accessory-isolated'
|
||||
| 'accessory-named';
|
||||
|
||||
export type UploadedImage = {
|
||||
id: string;
|
||||
url: string;
|
||||
filename: string;
|
||||
originalFilename?: string;
|
||||
mimeType: string;
|
||||
uploadedAt: number;
|
||||
role: UploadedImageRole;
|
||||
accessoryName?: string;
|
||||
needsCleanup: boolean;
|
||||
};
|
||||
|
||||
export type PreFilledSlot = {
|
||||
uploadedImageId: string;
|
||||
templateId: string;
|
||||
role: UploadedImageRole;
|
||||
url: string;
|
||||
};
|
||||
|
||||
export type PackKind = 'patent' | 'production' | 'marketing' | 'accessories';
|
||||
|
||||
export type AssetStatus = 'draft' | 'selected' | 'needs_regen' | 'approved' | 'exported';
|
||||
@@ -188,6 +224,7 @@ export type CleanupCharacterRequest = {
|
||||
sessionId: string;
|
||||
imageId: string;
|
||||
force?: boolean;
|
||||
preserveLevel?: 'normal' | 'strict';
|
||||
};
|
||||
|
||||
export type CleanupCharacterResponse = {
|
||||
@@ -196,6 +233,35 @@ export type CleanupCharacterResponse = {
|
||||
provider: 'mock' | 'gpt';
|
||||
};
|
||||
|
||||
export type UploadImageResponse = {
|
||||
uploadedImage: UploadedImage;
|
||||
};
|
||||
|
||||
export type ProjectFromUploadRequest = {
|
||||
uploadedImages: UploadedImage[];
|
||||
mode: 'remix' | 'replicate' | 'extend';
|
||||
remixPrompt?: string;
|
||||
userHint?: string;
|
||||
styleId?: string;
|
||||
count?: number;
|
||||
};
|
||||
|
||||
export type ProjectFromUploadResponse = {
|
||||
sessionId: string;
|
||||
images?: GenImage[];
|
||||
characterSpec?: CharacterSpec;
|
||||
l1AnchorUrl?: string;
|
||||
preFilledSlots?: PreFilledSlot[];
|
||||
provider: 'mock' | 'gpt';
|
||||
};
|
||||
|
||||
export type LockCharacterFromUploadRequest = {
|
||||
sessionId: string;
|
||||
subjectImageId: string;
|
||||
userHint?: string;
|
||||
force?: boolean;
|
||||
};
|
||||
|
||||
export type RegenerateAssetRequest = {
|
||||
sessionId: string;
|
||||
userRefinement?: string;
|
||||
|
||||
Reference in New Issue
Block a user