28.5 分区生命周期
如果数据天然按时间或租户整片到期,最好的 vacuum 往往是不要制造那些 dead tuple。
DELETE 500 million expired rows
-> row locks / WAL / dead tuples / index cleanup / vacuum / replica replay
DETACH one expired partition
-> catalog and lock operation
-> standalone table
-> archive / validate / drop分区不是免费的性能开关;它是把数据生命周期编码进物理边界。只有 partition key、 retention unit、query pruning 和发布流程一致时,整片退役才成立。
28.5.1 新分区预建、约束和父表显式 ANALYZE
从生命周期单位反推边界
先定义:
event_time_semantics: UTC timestamptz
retention: 400 days
retirement_unit: month
late_arrival: 7 days
future_precreate: 3 months
archive_retention: 7 years再决定:
partition key
range bounds
timezone
default partition policy
precreate horizon
detach cadence月分区不一定最好:
| 单位 | 优点 | 风险 |
|---|---|---|
| 日 | 退役粒度细 | partition 数、planning/catalog 开销 |
| 月 | 常见折中 | 大月仍可能过大 |
| 季/年 | 对象少 | 退役和维护粒度粗 |
| tenant hash/list | 隔离租户 | retention 可能仍需二级时间分区 |
目标不是最多 partition,而是让:
query predicate
retention cut
maintenance unit落在同一边界。
range 上界是排他的
CREATE TABLE app.events (
event_id bigint NOT NULL,
occurred_at timestamptz NOT NULL,
payload jsonb NOT NULL
) PARTITION BY RANGE (occurred_at);
CREATE TABLE app.events_2026_08
PARTITION OF app.events
FOR VALUES FROM ('2026-08-01 00:00:00+00')
TO ('2026-09-01 00:00:00+00');边界:
[2026-08-01 00:00Z, 2026-09-01 00:00Z)共享的 2026-09-01 属于下一个 partition。若应用按本地日历月保留,必须明确 DST 和
timezone;不要让 session TimeZone 隐式决定 DDL literal。
预建,不等 insert error 报警
写入没有匹配 partition 会失败。生产应提前:
generate future partitions
validate exact non-overlapping bounds
create local indexes
apply owner/grants/comments/storage parameters
ANALYZE when populated
alert on last future boundary例如维护表:
SELECT
parent.relname AS parent,
child.relname AS partition,
pg_get_expr(child.relpartbound, child.oid) AS bound
FROM pg_inherits AS i
JOIN pg_class AS parent ON parent.oid = i.inhparent
JOIN pg_class AS child ON child.oid = i.inhrelid
WHERE parent.oid = 'app.events'::regclass
ORDER BY child.relname;pg_partition_tree() 适合多层结构:
SELECT *
FROM pg_partition_tree('app.events');离线装载后 ATTACH
大分区可先作为普通表准备:
CREATE TABLE app.events_2026_08_stage
(LIKE app.events INCLUDING DEFAULTS INCLUDING CONSTRAINTS);
ALTER TABLE app.events_2026_08_stage
ADD CONSTRAINT events_2026_08_bound
CHECK (
occurred_at >= TIMESTAMPTZ '2026-08-01 00:00:00+00'
AND occurred_at < TIMESTAMPTZ '2026-09-01 00:00:00+00'
);
-- load, cleanse, build indexes, validate
ALTER TABLE app.events
ATTACH PARTITION app.events_2026_08_stage
FOR VALUES FROM ('2026-08-01 00:00:00+00')
TO ('2026-09-01 00:00:00+00');若已有一个有效且与 partition bound 匹配的 CHECK constraint,PostgreSQL 可避免
在持有 partition ACCESS EXCLUSIVE 时扫描全表验证。attach 完成后,这个重复
constraint 可在评审后删除。
若 parent 有 default partition,还应给 default 添加排除新范围的 CHECK;否则 attach
可能扫描 default,且持有其强锁。
注意:
- expression partition key 有额外限制;
- list partition 是否接受
NULL影响 constraint; - subpartition 可能递归锁/扫到 leaf;
- parent 是 virtual structure,实际 index 在 leaf;
- attach 前要验证 index/constraint 与 parent 模板一致。
default partition 是缓冲区,不是垃圾桶
default 可避免未知 key 直接失败,但会带来:
silent routing of bad/future data
attach scan/lock cost
DETACH CONCURRENTLY restriction on that parent
data migration before new range attach若使用 default:
- 监控 row count;
- bad key 立即告警;
- 定期清空到正确 partition;
- 在 attach/detach runbook 中显式处理;
- 不把它当永久无限分区。
parent 必须显式 ANALYZE
partition leaf 的变化不会触发 parent auto-analyze;partitioned table 自身不直接存 tuple,
autovacuum 不会在 parent 上运行 ANALYZE。当首次装载或分布显著变化时:
ANALYZE app.events_2026_08;
ANALYZE app.events;parent-level statistics 会影响引用 partitioned table 的 plan。生命周期动作完成而漏掉 parent analyze,可能导致:
row estimate drift
join order change
partition-wise plan quality loss这不是物理 bloat,却常被误归因成“分区太多”。
分区模板是 schema release
创建脚本应来自同一 desired state:
columns / generated expressions
constraints
indexes / INCLUDE / predicates
storage parameters / fillfactor
tablespace
owner / grants / RLS
publication policy
comments
autovacuum overrides不能靠:
CREATE TABLE child (LIKE parent);就假设复制了所有业务语义。LIKE 的 INCLUDING ... 选项、partitioned parent 的虚拟
对象、trigger/RLS/publication 行为都要按版本验证。
28.5.2 DETACH、归档、验证后删除
detach 不是 drop
ALTER TABLE app.events
DETACH PARTITION app.events_2024_01;结果:
parent no longer routes/scans it
child remains as standalone table
attached child indexes detach from parent indexes
cloned triggers are removed
data remains queryable by standalone name这是理想的 quarantine point:
online dataset
-> detached immutable dataset
-> archive
-> restore validation
-> deletion普通与 CONCURRENTLY
普通 detach 对 parent 取得 ACCESS EXCLUSIVE。
ALTER TABLE app.events
DETACH PARTITION app.events_2024_01 CONCURRENTLY;PostgreSQL 18 的 concurrent 形式不是“零锁”,而是内部两个 transaction:
- 对 parent 和 partition 取得
SHARE UPDATE EXCLUSIVE,标记 pending detach 并提交; - 等所有使用 partitioned table 的旧 transaction 离开;
- 再对 parent 取
SHARE UPDATE EXCLUSIVE、对 partition 取ACCESS EXCLUSIVE; - 完成 detach,并给 standalone table 添加等价
CHECKconstraint。
限制:
cannot run inside transaction block
not allowed when parent has a default partition
only one partition per parent may be pending detach
foreign-key related tables may acquire SHARE locks
old transactions can prolong the wait中断后:
ALTER TABLE app.events
DETACH PARTITION app.events_2024_01 FINALIZE;用于完成先前被取消/中断的 concurrent detach。自动化不能看到命令失败就直接重跑或
drop;先查 pending state,再决定 FINALIZE。
先关闭边界写入竞争
在 detach 前确认:
retention cutoff immutable
late-arrival window closed
backfill jobs stopped
application routes no new row to old range
timezone/cutoff reviewed
no open transaction still writes old partition否则 detach 后:
- 新写入可能失败;
- 被路由到 default;
- 被误写到 archive standalone table;
- 数据清单在导出期间变化。
一种做法是先把旧 partition 业务状态标成 sealed,再等 maximum transaction duration 过去,最后 detach。真正的控制点在应用与数据产品,不只在 DDL。
归档清单至少有四层
- 对象清单
database/schema/table
partition bound
owner/grants
columns/types/collations
constraints/indexes
row-level security- 逻辑清单
SELECT
count(*) AS rows,
min(event_id) AS min_id,
max(event_id) AS max_id,
min(occurred_at) AS min_time,
max(occurred_at) AS max_time,
sum(amount) AS amount_sum
FROM app.events_2024_01;- 归档文件清单
format/version
compression/encryption
object URI
bytes
cryptographic hash
created_at
retention/legal hold- 恢复清单
restore target
row/type/constraint verification
logical aggregates/digests
query spot checks
elapsed time
tool versions只有 file hash 相同,不能证明文件可以被当前工具恢复;只有 row count 相同,也不能证明 金额、范围和编码正确。
COPY 与 pg_dump 的选择
COPY
simple data stream
schema/privileges not included
explicit order and format needed
pg_dump table
schema/data options
dependency-aware archive
restore tooling and version policy needed
base backup
cluster physical recovery
not a per-partition logical archive对于大型表,不要用:
md5(string_agg(all_rows...))在 server 端聚合整个数据集;第 28 章夹具只有 10,000 行,才用它作为教学逻辑摘要。 生产可用有序 chunk hash、COPY/Parquet manifest、业务聚合和独立 restore 合并证明。
正式实验的顺序
events_2024 rows 10,000
parent total 15,000
manifest
rows 10,000
id range 1..10,000
amount sum 499,950.00
logical digest recorded
DETACH CONCURRENTLY pass
parent after detach 5,000
standalone 10,000
CSV bytes 547,894
CSV SHA-256 cd7d54e4...a9a41d0d
restore check rows 10,000
restore digest equal
drop standalone only after equality清理 validator 会拒绝:
drop before restore validation
row mismatch
digest mismatch
empty archive
parent count mismatch
force cleanup删除后还要验 backup policy
partition 从在线库删除后:
- PITR 仍可在 retention window 内恢复历史 cluster;
- 逻辑 archive 负责更长期访问;
- backup retention 和 archive legal retention 可能不同;
- GDPR/删除义务也可能要求从 archive 到期清除;
- catalog/monitoring 应记录在线与归档位置的转换。
生命周期不是 DROP TABLE 结束,而是 ownership 从 online service 转到 archive service。
28.5.3 用分区退役替代大批量 DELETE
大 DELETE 的债
DELETE FROM app.events
WHERE occurred_at < now() - interval '400 days';可能产生:
row locks
large transaction / long snapshot
WAL and archive volume
replica replay
dead heap tuples
dead index tuples
autovacuum backlog
relation growth before reuse
rollback/retry cost分批 delete 可控制 transaction:
WITH victim AS (
SELECT ctid
FROM app.events
WHERE occurred_at < $1
ORDER BY occurred_at
LIMIT 10000
FOR UPDATE SKIP LOCKED
)
DELETE FROM app.events AS e
USING victim AS v
WHERE e.ctid = v.ctid;但它仍逐行处理,且 ctid 只用于当次短事务。批处理适合选择性删除,不如整片
partition 退役。
detach 的收益来自事前设计
若 expired predicate 恰好覆盖完整 partition:
row-by-row physical change
-> partition metadata change它避免制造海量 obsolete tuples。随后 drop standalone table 删除 relation files, 也不需要 vacuum 每一行。
但不能夸大为 $O(1)$、瞬间、无 WAL、无锁:
- catalog 要更新;
- parent/partition/FK 有锁;
- concurrent 形式要等旧 transaction;
- replica 要 replay DDL;
- drop 仍要处理 dependency 和文件;
- archive copy 仍按数据量花费 I/O;
- planning/catalog 对 partition 数敏感。
准确说:
对生命周期与 partition bound 对齐的数据,detach/drop 把逐行淘汰的核心成本转换成 受控 DDL 与归档成本。
何时不能替代
predicate cuts through every partition
legal hold retains arbitrary rows
tenant records mixed in same partition
foreign keys prevent independent detach
late arrivals keep changing old ranges
application directly names leaf tables
default partition contains mixed data这时选择:
- 更合适的 partition key/subpartition;
- selective batch delete;
- logical archive+copy;
- tenant migration;
- schema redesign。
不要为了这次清理临时创建数千 partition;partitioning 是长期模型。
FK 与全局唯一性
partitioned table 的 unique/primary key 通常必须包含 partition key,才能由每个 leaf 的 局部索引共同保证全局逻辑唯一。跨 partition FK、引用 leaf、触发器和 publication 也会影响 detach。
设计前问:
Can an event_id be unique without time?
Who references expired rows?
Should archive preserve FK graph?
Can referenced partitions retire independently?若答案不清晰,retention 不是一个 DBA 单表任务。
DELETE 与 detach 的选择矩阵
| 条件 | batch DELETE | partition detach |
|---|---|---|
| 任意 predicate | 支持 | 不支持 |
| 整片时间范围 | 可做但昂贵 | 优先候选 |
| 在线逐步释放 | 支持 | 按 partition 粒度 |
| 归档后保留 standalone | 需 copy | 天然 |
| dead tuple/vacuum 债 | 有 | 不制造逐行债 |
| 前置建模 | 少 | 必须 |
| FK/依赖 | 行级处理 | DDL 级约束 |
28.5.4 将 ch04/ch07/ch11/ch16/ch28 串成能力索引
分区生命周期不是本章孤立技巧,而是贯穿设计、查询、发布和运维的能力。
第 4 章:数据表达决定边界
timestamptz vs timestamp
UTC and business timezone
NOT NULL / CHECK
identity and uniqueness
retention metadata若时间语义错,partition boundary 再精确也会错删。
能力交付:
partition_key:
column: occurred_at
type: timestamptz
canonical_zone: UTC
null_allowed: false
retention_cutoff_semantics: event_time第 7 章:统计与 pruning 证明查询受益
第 7 章:执行计划与统计信息 负责:
partition pruning
parameterized plan behavior
parent/leaf statistics
join estimate
EXPLAIN evidence能力交付:
representative predicates prune expected leaves
generic/custom plans both reviewed
parent ANALYZE after distribution change
planning time acceptable at partition count分区多但查询不带 partition key,可能同时扫描大量 leaf;那不是 vacuum 能修的。
第 11 章:生命周期 DDL 是安全发布
第 11 章:模式变更与安全发布 负责:
lock compatibility
lock_timeout
preflight holders
expand/contract
canary and rollback
DDL queue behavior能力交付:
change:
attach_or_detach: ...
lock_budget: ...
transaction_block: false
old_snapshot_gate: ...
interrupted_state: ...
finalize_or_rollback: ...DETACH CONCURRENTLY 仍是 production DDL。
第 16 章:时序/时空 workload 定义生命周期
time bucketing
late/out-of-order events
rollup/downsample
spatial/temporal retention
hot/warm/cold tiers能力交付:
late arrival horizon
immutable cutoff
rollup completion watermark
archive query contract只有 watermark 越过 partition end + late-arrival allowance,才可 seal/detach。
第 28 章:运营闭环
本章负责:
precreate
explicit analyze
seal
detach
archive manifest
restore validation
drop
monitor and audit合起来:
type/constraint semantics ch04
-> plan/pruning/statistics ch07
-> safe DDL release ch11
-> time-domain policy ch16
-> retirement SOP ch28可执行能力索引
| 能力 | 输入 | 证据 | 失败路由 |
|---|---|---|---|
| create future partition | calendar + schema | bound/tree diff | ch11 |
| attach loaded partition | valid CHECK + manifest | no scan/lock plan + row proof | ch11 |
| analyze hierarchy | distribution change | parent/leaf stats | ch07 |
| seal range | watermark + late window | no new writes | ch16 |
| detach | dependency + lock budget | topology diff | ch11/ch28 |
| archive | object + logical manifest | hash + restore | ch28/ch35 |
| drop | approved retention | archive proof + audit | ch28 |
| query cold data | archive contract | result/SLO test | ch16 |
生命周期控制表
可在 control plane 维护:
CREATE TABLE ops.partition_lifecycle (
parent_table regclass NOT NULL,
partition_table regclass,
range_start timestamptz NOT NULL,
range_end timestamptz NOT NULL,
state text NOT NULL CHECK (
state IN (
'planned','attached','sealed','detached',
'archived','restore_verified','dropped'
)
),
manifest_uri text,
manifest_sha256 text,
approved_by text,
changed_at timestamptz NOT NULL DEFAULT clock_timestamp(),
PRIMARY KEY (parent_table, range_start)
);但不要让这张表成为未经核验的自动 drop 开关。状态转换必须读取数据库 catalog、归档 系统和审批事实;control row 只是审计协调,不是单独真相。
本节检查清单
partition key and timezone semantics
retention and late-arrival horizon
future partitions precreated
bound gaps/overlaps checked
leaf schema/index/grant consistency
attach CHECK avoids scan where possible
default partition policy
parent explicit ANALYZE
dependency/FK inventory
old snapshot and lock budget
DETACH CONCURRENTLY restrictions
interrupted FINALIZE plan
object/logical/file/restore manifest
drop only after restore proof
online/archive ownership transition延伸阅读
- PostgreSQL 18:Table Partitioning
- PostgreSQL 18:ALTER TABLE ATTACH/DETACH
- PostgreSQL 18:Updating Planner Statistics
- PostgreSQL 18:
pg_partition_tree
上一节:膨胀与重建 · 返回本章目录 · 下一节:amcheck 与例行完整性检查 ·
查看全书目录 · 查看索引中心