docs: project storage architecture (en+zh), linked from README and docs index
This commit is contained in:
@@ -23,6 +23,11 @@ The binary can be downloaded here:
|
||||
|
||||
See [`docs/build.md`](docs/build.md) for build instructions on Windows (MSYS2), Linux (Debian/Ubuntu, Fedora, Arch Linux), and macOS.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Project Storage Architecture](docs/project-storage.md) ([中文](docs/zh/project-storage.md)) — database write-through persistence, node-granular journal, persistent undo
|
||||
- [Build guide](docs/build.md) · [工程文件格式](docs/zh/project-file-reference.md)
|
||||
|
||||
## Roadmap
|
||||
|
||||
| Version | Theme | Core Deliverables | Boundary Notes |
|
||||
|
||||
@@ -30,4 +30,5 @@ Oak Video Editor is a renamed fork of Olive, focused on delivering a polished, c
|
||||
|
||||
- Build from source on Windows, macOS, or Linux using the Build guide.
|
||||
- Learn how project data is stored in the Project File Reference.
|
||||
- Understand the database write-through persistence in the [Project Storage Architecture](project-storage.md).
|
||||
- Keep releases solid by following the Test Plan.
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
# Project Storage Architecture
|
||||
|
||||
[中文](zh/project-storage.md)
|
||||
|
||||
Oak persists projects in a **database** (SQLite by default, PostgreSQL
|
||||
supported), with **write-through persistence**: every edit is committed
|
||||
to the database as it happens — there is no Save button and nothing to
|
||||
lose. The `.ove` XML file, OTIO and FCPXML are import/export formats,
|
||||
not the working store.
|
||||
|
||||
## Design principles
|
||||
|
||||
- **Aggregate-granular persistence, not full relational mapping.** The
|
||||
database stores four things: project metadata, key/value settings,
|
||||
periodic full snapshots, and a per-node command journal. Domain
|
||||
semantics (timeline structure, node connections, keyframes, effect
|
||||
chains) stay inside each node's XML payload. A project is exactly
|
||||
"the node graph plus settings" — nothing else — so node granularity
|
||||
is a closed, complete model.
|
||||
- **One serialization truth.** The node XML in the database is the same
|
||||
document the `.ove` serializer produces (`oaknode::serializer`).
|
||||
New features (e.g. adjustment layers) extend the XML schema only —
|
||||
the database schema never changes.
|
||||
- **The journal is produced by diffing, not by instrumenting commands.**
|
||||
After every undoable command the in-memory project is re-serialized
|
||||
and compared node-by-node with the previous state; changed/added/
|
||||
removed nodes each become one journal row. Existing and future
|
||||
command types are covered automatically.
|
||||
- **The journal is the persistent undo history.** Replaying applies
|
||||
each node's newest image in command order; rewinding applies the
|
||||
previous images in reverse. Undo survives restarts.
|
||||
|
||||
## Schema (SQLite / PostgreSQL, sea-orm)
|
||||
|
||||
```sql
|
||||
projects(id PK, uuid UNIQUE, name, schema_ver, created_at, modified_at, command_seq)
|
||||
settings(project_id FK, key, value, PK(project_id, key))
|
||||
snapshots(project_id FK, command_seq, payload, written_at, PK(project_id, command_seq))
|
||||
journal(project_id FK, seq, node_identity, kind, old_xml, new_xml, at,
|
||||
PK(project_id, seq, node_identity))
|
||||
```
|
||||
|
||||
- `snapshots.payload` is the full project XML, written every
|
||||
`Storage/SnapshotIntervalSec` (default 600) while dirty, pruned to
|
||||
the newest 3. It only accelerates loading — the journal alone can
|
||||
rebuild the project from scratch.
|
||||
- `journal` rows are whole-node before/after images: `old_xml` is NULL
|
||||
for created nodes, `new_xml` is NULL for removed nodes. Loading takes
|
||||
the newest snapshot and replaces each node with its newest image in
|
||||
`seq` order; rewinding to command `N` applies `old_xml` backwards.
|
||||
`node_identity = 0` is the settings pseudo-node.
|
||||
- Every command is one synchronous transaction (journal rows +
|
||||
`command_seq + 1`), so a `kill -9` loses zero commands.
|
||||
- `Storage/JournalRetentionDays` (default 0 = keep forever) bounds the
|
||||
undo window; rows are kilobytes each.
|
||||
|
||||
## Timeline representation
|
||||
|
||||
The timeline is a chain of node references inside node XML:
|
||||
|
||||
```
|
||||
sequence ──<tracklists>──▶ tracklist ──<tracks>──▶ track ──<blocks>──▶ clip ──<footage>──▶ footage
|
||||
```
|
||||
|
||||
The clip row carries its timeline range (`<range in out/>`), media
|
||||
offset (`<media_in>`) and footage reference; effect chains are
|
||||
connection records inside the effect nodes' XML. Loading re-links
|
||||
these identities in two passes (see `oaknode::serializer`), so no
|
||||
join tables are needed. Timeline edits map to a handful of node rows:
|
||||
moving a clip touches its track and the clip; splitting adds one node
|
||||
and updates two; ripple edits touch the affected tracks and delete the
|
||||
removed clips.
|
||||
|
||||
## Write-through flow
|
||||
|
||||
1. A facade undo push (`oakengine_undo_push`, group end, undo/redo/
|
||||
jump) succeeds.
|
||||
2. The project is re-serialized in memory (microseconds to low
|
||||
milliseconds) and diffed against the last state.
|
||||
3. One transaction writes the changed node rows, bumps `command_seq`
|
||||
and `modified_at`.
|
||||
4. A background thread writes snapshots latest-wins; on exit the queue
|
||||
is flushed.
|
||||
|
||||
## Import / export
|
||||
|
||||
- Import: `.ove` / `.otio` / `.fcpxml` are parsed by their existing
|
||||
oakstorage backends and inserted as a new project row with
|
||||
`kind = 'import'` journal entries.
|
||||
- Export: the in-memory serialization is written through the ove-xml
|
||||
or otio backend; nothing is read from or written to the database
|
||||
beyond the current state.
|
||||
|
||||
## Multi-writer and platforms
|
||||
|
||||
v1 assumes a single writer per database (SQLite `busy_timeout`, PG row
|
||||
locks). Multi-writer collaboration is future work (M14). The default
|
||||
database is a single user-level SQLite file; PostgreSQL is selected
|
||||
with an `oakdb+pg://` URI.
|
||||
|
||||
See also: [M10 oakstorage manual](plans/riir/M10-oakstorage.md),
|
||||
[M13 write-through plan](plans/riir/M13-storage-live.md),
|
||||
[project file reference](project-file-reference.md).
|
||||
@@ -30,6 +30,7 @@ Oak 视频编辑器是 Olive 的重命名分支，目标是打造更完善、更
|
||||
|
||||
- 按《构建指南》在 Windows/macOS/Linux 上从源码构建。
|
||||
- 在《工程文件参考》了解项目数据结构。
|
||||
- 在《[工程存储架构](project-storage.md)》了解数据库写穿持久化与持久撤销历史。
|
||||
- 按《测试计划》确保发布质量。
|
||||
|
||||
## 下载
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
# 工程存储架构
|
||||
|
||||
[English](../project-storage.md)
|
||||
|
||||
Oak 的工程持久化在**数据库**（默认 SQLite，支持 PostgreSQL），采用
|
||||
**写穿持久化**：每一次编辑都即时落库——没有保存按钮，也没有可丢失
|
||||
的数据。.ove XML 文件、OTIO、FCPXML 是导入/导出格式，不是日常存储。
|
||||
|
||||
## 设计原则
|
||||
|
||||
- **按聚合粒度持久化，不做全关系型映射。** 数据库只存四样东西：
|
||||
工程元信息、KV 设置、周期全量快照、节点粒度的命令日志。领域语义
|
||||
（时间线结构、节点连接、关键帧、效果链）全部留在每个节点的 XML
|
||||
payload 里。工程 = 节点图 + settings，没有第三种东西——节点粒度
|
||||
因此是封闭全集。
|
||||
- **单一序列化事实。** 库里的节点 XML 与 .ove 序列化器
|
||||
（`oaknode::serializer`）产出的是同一份文档。新功能（比如调整图层）
|
||||
只需要扩展 XML schema，数据库 schema 永远不变。
|
||||
- **journal 由 diff 产生，不靠命令申报。** 每条 undoable 命令成功后，
|
||||
在内存里重新序列化工程并与上一状态逐节点比对；变化/新增/删除的
|
||||
节点各落一行。现有和未来的命令类型自动覆盖。
|
||||
- **journal 就是持久化撤销历史。** 回放按命令序应用各节点最新像；
|
||||
回退按逆序应用旧像。撤销跨会话存活。
|
||||
|
||||
## Schema（SQLite / PostgreSQL，sea-orm）
|
||||
|
||||
```sql
|
||||
projects(id PK, uuid UNIQUE, name, schema_ver, created_at, modified_at, command_seq)
|
||||
settings(project_id FK, key, value, PK(project_id, key))
|
||||
snapshots(project_id FK, command_seq, payload, written_at, PK(project_id, command_seq))
|
||||
journal(project_id FK, seq, node_identity, kind, old_xml, new_xml, at,
|
||||
PK(project_id, seq, node_identity))
|
||||
```
|
||||
|
||||
- `snapshots.payload` 是全工程 XML，脏状态下每
|
||||
`Storage/SnapshotIntervalSec`（默认 600 秒）写一份，只留最近 3 份。
|
||||
它只是加载加速器——journal 自己就能从零重建工程。
|
||||
- `journal` 行是整节点前后像：新增节点 `old_xml` 为 NULL，删除节点
|
||||
`new_xml` 为 NULL。加载 = 取最新快照，其后按 `seq` 顺序把每个节点
|
||||
替换为最新像；回退到命令 N = 逆序回写 `old_xml`。
|
||||
`node_identity = 0` 是 settings 伪节点。
|
||||
- 每条命令一个同步事务（journal 行 + `command_seq + 1`），
|
||||
`kill -9` 丢 0 条命令。
|
||||
- `Storage/JournalRetentionDays`（默认 0 = 全保留）限定撤销窗口；
|
||||
每行仅 KB 级。
|
||||
|
||||
## 时间线的表示
|
||||
|
||||
时间线是节点 XML 内部的引用链：
|
||||
|
||||
```
|
||||
sequence ──<tracklists>──▶ tracklist ──<tracks>──▶ track ──<blocks>──▶ clip ──<footage>──▶ footage
|
||||
```
|
||||
|
||||
clip 行自带时间线区间（`<range in out/>`）、媒体偏移（`<media_in>`）
|
||||
和素材引用；效果链是效果节点 XML 里的连接记录。加载时两阶段重连
|
||||
这些 identity（见 `oaknode::serializer`），所以不需要任何连接表。
|
||||
时间线编辑映射为少数节点行：移动 clip 触及它的 track 和 clip 本身；
|
||||
分割新增一个节点、更新两个；ripple 编辑触及受影响的 track 并删除
|
||||
被移除的 clip。
|
||||
|
||||
## 写穿流程
|
||||
|
||||
1. facade 的 undo 推送（`oakengine_undo_push`、group_end、
|
||||
undo/redo/jump）成功；
|
||||
2. 工程在内存里重新序列化（微秒到低毫秒级）并与上一状态 diff；
|
||||
3. 一个事务写入变化节点行、`command_seq + 1`、`modified_at`；
|
||||
4. 后台线程 latest-wins 写快照；退出前排空队列。
|
||||
|
||||
## 导入 / 导出
|
||||
|
||||
- 导入：.ove / .otio / .fcpxml 由既有 oakstorage 后端解析后，以
|
||||
`kind = 'import'` 的 journal 行写为新工程行。
|
||||
- 导出：内存序列化经 ove-xml 或 otio 后端写出；除当前状态外不读写
|
||||
数据库。
|
||||
|
||||
## 多写者与平台
|
||||
|
||||
v1 假设单写者（SQLite `busy_timeout`，PG 行锁）；多写者协作是后续
|
||||
工作（M14）。默认数据库是用户级单一 SQLite 文件；PostgreSQL 用
|
||||
`oakdb+pg://` 连接串选择。
|
||||
|
||||
另见：[M10 oakstorage 手册](plans/riir/M10-oakstorage.md)、
|
||||
[M13 写穿计划](plans/riir/M13-storage-live.md)、
|
||||
[工程文件格式参考](project-file-reference.md)。
|
||||
Reference in New Issue
Block a user