跳至内容

28.5 分区生命周期

如果数据天然按时间或租户整片到期,最好的 vacuum 往往是不要制造那些 dead tuple

DELETE 500 million expired rows
  -> row locks / WAL / dead tuples / index cleanup / vacuum / replica replay

DETACH one expired partition
  -> catalog and lock operation
      -> standalone table
          -> archive / validate / drop

分区不是免费的性能开关;它是把数据生命周期编码进物理边界。只有 partition key、 retention unit、query pruning 和发布流程一致时,整片退役才成立。

28.5.1 新分区预建、约束和父表显式 ANALYZE

从生命周期单位反推边界

先定义:

event_time_semantics: UTC timestamptz
retention: 400 days
retirement_unit: month
late_arrival: 7 days
future_precreate: 3 months
archive_retention: 7 years

再决定:

partition key
range bounds
timezone
default partition policy
precreate horizon
detach cadence

月分区不一定最好:

单位优点风险
退役粒度细partition 数、planning/catalog 开销
常见折中大月仍可能过大
季/年对象少退役和维护粒度粗
tenant hash/list隔离租户retention 可能仍需二级时间分区

目标不是最多 partition,而是让:

query predicate
retention cut
maintenance unit

落在同一边界。

range 上界是排他的

CREATE TABLE app.events (
  event_id bigint NOT NULL,
  occurred_at timestamptz NOT NULL,
  payload jsonb NOT NULL
) PARTITION BY RANGE (occurred_at);

CREATE TABLE app.events_2026_08
  PARTITION OF app.events
  FOR VALUES FROM ('2026-08-01 00:00:00+00')
             TO   ('2026-09-01 00:00:00+00');

边界:

[2026-08-01 00:00Z, 2026-09-01 00:00Z)

共享的 2026-09-01 属于下一个 partition。若应用按本地日历月保留,必须明确 DST 和 timezone;不要让 session TimeZone 隐式决定 DDL literal。

预建,不等 insert error 报警

写入没有匹配 partition 会失败。生产应提前:

generate future partitions
validate exact non-overlapping bounds
create local indexes
apply owner/grants/comments/storage parameters
ANALYZE when populated
alert on last future boundary

例如维护表:

SELECT
  parent.relname AS parent,
  child.relname AS partition,
  pg_get_expr(child.relpartbound, child.oid) AS bound
FROM pg_inherits AS i
JOIN pg_class AS parent ON parent.oid = i.inhparent
JOIN pg_class AS child ON child.oid = i.inhrelid
WHERE parent.oid = 'app.events'::regclass
ORDER BY child.relname;

pg_partition_tree() 适合多层结构:

SELECT *
FROM pg_partition_tree('app.events');

离线装载后 ATTACH

大分区可先作为普通表准备:

CREATE TABLE app.events_2026_08_stage
  (LIKE app.events INCLUDING DEFAULTS INCLUDING CONSTRAINTS);

ALTER TABLE app.events_2026_08_stage
  ADD CONSTRAINT events_2026_08_bound
  CHECK (
    occurred_at >= TIMESTAMPTZ '2026-08-01 00:00:00+00'
    AND occurred_at < TIMESTAMPTZ '2026-09-01 00:00:00+00'
  );

-- load, cleanse, build indexes, validate

ALTER TABLE app.events
  ATTACH PARTITION app.events_2026_08_stage
  FOR VALUES FROM ('2026-08-01 00:00:00+00')
             TO   ('2026-09-01 00:00:00+00');

若已有一个有效且与 partition bound 匹配CHECK constraint,PostgreSQL 可避免 在持有 partition ACCESS EXCLUSIVE 时扫描全表验证。attach 完成后,这个重复 constraint 可在评审后删除。

若 parent 有 default partition,还应给 default 添加排除新范围的 CHECK;否则 attach 可能扫描 default,且持有其强锁。

注意:

  • expression partition key 有额外限制;
  • list partition 是否接受 NULL 影响 constraint;
  • subpartition 可能递归锁/扫到 leaf;
  • parent 是 virtual structure,实际 index 在 leaf;
  • attach 前要验证 index/constraint 与 parent 模板一致。

default partition 是缓冲区,不是垃圾桶

default 可避免未知 key 直接失败,但会带来:

silent routing of bad/future data
attach scan/lock cost
DETACH CONCURRENTLY restriction on that parent
data migration before new range attach

若使用 default:

  • 监控 row count;
  • bad key 立即告警;
  • 定期清空到正确 partition;
  • 在 attach/detach runbook 中显式处理;
  • 不把它当永久无限分区。

parent 必须显式 ANALYZE

partition leaf 的变化不会触发 parent auto-analyze;partitioned table 自身不直接存 tuple, autovacuum 不会在 parent 上运行 ANALYZE。当首次装载或分布显著变化时:

ANALYZE app.events_2026_08;
ANALYZE app.events;

parent-level statistics 会影响引用 partitioned table 的 plan。生命周期动作完成而漏掉 parent analyze,可能导致:

row estimate drift
join order change
partition-wise plan quality loss

这不是物理 bloat,却常被误归因成“分区太多”。

分区模板是 schema release

创建脚本应来自同一 desired state:

columns / generated expressions
constraints
indexes / INCLUDE / predicates
storage parameters / fillfactor
tablespace
owner / grants / RLS
publication policy
comments
autovacuum overrides

不能靠:

CREATE TABLE child (LIKE parent);

就假设复制了所有业务语义。LIKEINCLUDING ... 选项、partitioned parent 的虚拟 对象、trigger/RLS/publication 行为都要按版本验证。

28.5.2 DETACH、归档、验证后删除

detach 不是 drop

ALTER TABLE app.events
  DETACH PARTITION app.events_2024_01;

结果:

parent no longer routes/scans it
child remains as standalone table
attached child indexes detach from parent indexes
cloned triggers are removed
data remains queryable by standalone name

这是理想的 quarantine point:

online dataset
  -> detached immutable dataset
      -> archive
          -> restore validation
              -> deletion

普通与 CONCURRENTLY

普通 detach 对 parent 取得 ACCESS EXCLUSIVE

ALTER TABLE app.events
  DETACH PARTITION app.events_2024_01 CONCURRENTLY;

PostgreSQL 18 的 concurrent 形式不是“零锁”,而是内部两个 transaction:

  1. 对 parent 和 partition 取得 SHARE UPDATE EXCLUSIVE,标记 pending detach 并提交;
  2. 等所有使用 partitioned table 的旧 transaction 离开;
  3. 再对 parent 取 SHARE UPDATE EXCLUSIVE、对 partition 取 ACCESS EXCLUSIVE
  4. 完成 detach,并给 standalone table 添加等价 CHECK constraint。

限制:

cannot run inside transaction block
not allowed when parent has a default partition
only one partition per parent may be pending detach
foreign-key related tables may acquire SHARE locks
old transactions can prolong the wait

中断后:

ALTER TABLE app.events
  DETACH PARTITION app.events_2024_01 FINALIZE;

用于完成先前被取消/中断的 concurrent detach。自动化不能看到命令失败就直接重跑或 drop;先查 pending state,再决定 FINALIZE

先关闭边界写入竞争

在 detach 前确认:

retention cutoff immutable
late-arrival window closed
backfill jobs stopped
application routes no new row to old range
timezone/cutoff reviewed
no open transaction still writes old partition

否则 detach 后:

  • 新写入可能失败;
  • 被路由到 default;
  • 被误写到 archive standalone table;
  • 数据清单在导出期间变化。

一种做法是先把旧 partition 业务状态标成 sealed,再等 maximum transaction duration 过去,最后 detach。真正的控制点在应用与数据产品,不只在 DDL。

归档清单至少有四层

  1. 对象清单
database/schema/table
partition bound
owner/grants
columns/types/collations
constraints/indexes
row-level security
  1. 逻辑清单
SELECT
  count(*) AS rows,
  min(event_id) AS min_id,
  max(event_id) AS max_id,
  min(occurred_at) AS min_time,
  max(occurred_at) AS max_time,
  sum(amount) AS amount_sum
FROM app.events_2024_01;
  1. 归档文件清单
format/version
compression/encryption
object URI
bytes
cryptographic hash
created_at
retention/legal hold
  1. 恢复清单
restore target
row/type/constraint verification
logical aggregates/digests
query spot checks
elapsed time
tool versions

只有 file hash 相同,不能证明文件可以被当前工具恢复;只有 row count 相同,也不能证明 金额、范围和编码正确。

COPY 与 pg_dump 的选择

COPY
  simple data stream
  schema/privileges not included
  explicit order and format needed

pg_dump table
  schema/data options
  dependency-aware archive
  restore tooling and version policy needed

base backup
  cluster physical recovery
  not a per-partition logical archive

对于大型表,不要用:

md5(string_agg(all_rows...))

在 server 端聚合整个数据集;第 28 章夹具只有 10,000 行,才用它作为教学逻辑摘要。 生产可用有序 chunk hash、COPY/Parquet manifest、业务聚合和独立 restore 合并证明。

正式实验的顺序

events_2024 rows         10,000
parent total             15,000

manifest
  rows                   10,000
  id range               1..10,000
  amount sum             499,950.00
  logical digest         recorded

DETACH CONCURRENTLY      pass
parent after detach      5,000
standalone               10,000

CSV bytes                547,894
CSV SHA-256              cd7d54e4...a9a41d0d

restore check rows       10,000
restore digest           equal

drop standalone          only after equality

清理 validator 会拒绝:

drop before restore validation
row mismatch
digest mismatch
empty archive
parent count mismatch
force cleanup

删除后还要验 backup policy

partition 从在线库删除后:

  • PITR 仍可在 retention window 内恢复历史 cluster;
  • 逻辑 archive 负责更长期访问;
  • backup retention 和 archive legal retention 可能不同;
  • GDPR/删除义务也可能要求从 archive 到期清除;
  • catalog/monitoring 应记录在线与归档位置的转换。

生命周期不是 DROP TABLE 结束,而是 ownership 从 online service 转到 archive service。

28.5.3 用分区退役替代大批量 DELETE

大 DELETE 的债

DELETE FROM app.events
WHERE occurred_at < now() - interval '400 days';

可能产生:

row locks
large transaction / long snapshot
WAL and archive volume
replica replay
dead heap tuples
dead index tuples
autovacuum backlog
relation growth before reuse
rollback/retry cost

分批 delete 可控制 transaction:

WITH victim AS (
  SELECT ctid
  FROM app.events
  WHERE occurred_at < $1
  ORDER BY occurred_at
  LIMIT 10000
  FOR UPDATE SKIP LOCKED
)
DELETE FROM app.events AS e
USING victim AS v
WHERE e.ctid = v.ctid;

但它仍逐行处理,且 ctid 只用于当次短事务。批处理适合选择性删除,不如整片 partition 退役。

detach 的收益来自事前设计

若 expired predicate 恰好覆盖完整 partition:

row-by-row physical change
  -> partition metadata change

它避免制造海量 obsolete tuples。随后 drop standalone table 删除 relation files, 也不需要 vacuum 每一行。

但不能夸大为 $O(1)$、瞬间、无 WAL、无锁:

  • catalog 要更新;
  • parent/partition/FK 有锁;
  • concurrent 形式要等旧 transaction;
  • replica 要 replay DDL;
  • drop 仍要处理 dependency 和文件;
  • archive copy 仍按数据量花费 I/O;
  • planning/catalog 对 partition 数敏感。

准确说:

对生命周期与 partition bound 对齐的数据,detach/drop 把逐行淘汰的核心成本转换成 受控 DDL 与归档成本。

何时不能替代

predicate cuts through every partition
legal hold retains arbitrary rows
tenant records mixed in same partition
foreign keys prevent independent detach
late arrivals keep changing old ranges
application directly names leaf tables
default partition contains mixed data

这时选择:

  • 更合适的 partition key/subpartition;
  • selective batch delete;
  • logical archive+copy;
  • tenant migration;
  • schema redesign。

不要为了这次清理临时创建数千 partition;partitioning 是长期模型。

FK 与全局唯一性

partitioned table 的 unique/primary key 通常必须包含 partition key,才能由每个 leaf 的 局部索引共同保证全局逻辑唯一。跨 partition FK、引用 leaf、触发器和 publication 也会影响 detach。

设计前问:

Can an event_id be unique without time?
Who references expired rows?
Should archive preserve FK graph?
Can referenced partitions retire independently?

若答案不清晰,retention 不是一个 DBA 单表任务。

DELETE 与 detach 的选择矩阵

条件batch DELETEpartition detach
任意 predicate支持不支持
整片时间范围可做但昂贵优先候选
在线逐步释放支持按 partition 粒度
归档后保留 standalone需 copy天然
dead tuple/vacuum 债不制造逐行债
前置建模必须
FK/依赖行级处理DDL 级约束

28.5.4 将 ch04/ch07/ch11/ch16/ch28 串成能力索引

分区生命周期不是本章孤立技巧,而是贯穿设计、查询、发布和运维的能力。

第 4 章:数据表达决定边界

第 4 章:数据类型、约束与可靠数据表达 负责:

timestamptz vs timestamp
UTC and business timezone
NOT NULL / CHECK
identity and uniqueness
retention metadata

若时间语义错,partition boundary 再精确也会错删。

能力交付:

partition_key:
  column: occurred_at
  type: timestamptz
  canonical_zone: UTC
  null_allowed: false
retention_cutoff_semantics: event_time

第 7 章:统计与 pruning 证明查询受益

第 7 章:执行计划与统计信息 负责:

partition pruning
parameterized plan behavior
parent/leaf statistics
join estimate
EXPLAIN evidence

能力交付:

representative predicates prune expected leaves
generic/custom plans both reviewed
parent ANALYZE after distribution change
planning time acceptable at partition count

分区多但查询不带 partition key,可能同时扫描大量 leaf;那不是 vacuum 能修的。

第 11 章:生命周期 DDL 是安全发布

第 11 章:模式变更与安全发布 负责:

lock compatibility
lock_timeout
preflight holders
expand/contract
canary and rollback
DDL queue behavior

能力交付:

change:
  attach_or_detach: ...
  lock_budget: ...
  transaction_block: false
  old_snapshot_gate: ...
  interrupted_state: ...
  finalize_or_rollback: ...

DETACH CONCURRENTLY 仍是 production DDL。

第 16 章:时序/时空 workload 定义生命周期

第 16 章:时序、空间与时空查询 负责:

time bucketing
late/out-of-order events
rollup/downsample
spatial/temporal retention
hot/warm/cold tiers

能力交付:

late arrival horizon
immutable cutoff
rollup completion watermark
archive query contract

只有 watermark 越过 partition end + late-arrival allowance,才可 seal/detach。

第 28 章:运营闭环

本章负责:

precreate
explicit analyze
seal
detach
archive manifest
restore validation
drop
monitor and audit

合起来:

type/constraint semantics          ch04
  -> plan/pruning/statistics       ch07
      -> safe DDL release          ch11
          -> time-domain policy    ch16
              -> retirement SOP    ch28

可执行能力索引

能力输入证据失败路由
create future partitioncalendar + schemabound/tree diffch11
attach loaded partitionvalid CHECK + manifestno scan/lock plan + row proofch11
analyze hierarchydistribution changeparent/leaf statsch07
seal rangewatermark + late windowno new writesch16
detachdependency + lock budgettopology diffch11/ch28
archiveobject + logical manifesthash + restorech28/ch35
dropapproved retentionarchive proof + auditch28
query cold dataarchive contractresult/SLO testch16

生命周期控制表

可在 control plane 维护:

CREATE TABLE ops.partition_lifecycle (
  parent_table regclass NOT NULL,
  partition_table regclass,
  range_start timestamptz NOT NULL,
  range_end timestamptz NOT NULL,
  state text NOT NULL CHECK (
    state IN (
      'planned','attached','sealed','detached',
      'archived','restore_verified','dropped'
    )
  ),
  manifest_uri text,
  manifest_sha256 text,
  approved_by text,
  changed_at timestamptz NOT NULL DEFAULT clock_timestamp(),
  PRIMARY KEY (parent_table, range_start)
);

但不要让这张表成为未经核验的自动 drop 开关。状态转换必须读取数据库 catalog、归档 系统和审批事实;control row 只是审计协调,不是单独真相。

本节检查清单

partition key and timezone semantics
retention and late-arrival horizon
future partitions precreated
bound gaps/overlaps checked
leaf schema/index/grant consistency
attach CHECK avoids scan where possible
default partition policy
parent explicit ANALYZE
dependency/FK inventory
old snapshot and lock budget
DETACH CONCURRENTLY restrictions
interrupted FINALIZE plan
object/logical/file/restore manifest
drop only after restore proof
online/archive ownership transition

延伸阅读


上一节:膨胀与重建 · 返回本章目录 · 下一节:amcheck 与例行完整性检查 · 查看全书目录 · 查看索引中心

最后更新于