pg_stat_statements.max(5,000)限制不同语句的数量;默认值 track = top 只统计顶层语句。从 PostgreSQL 18 开始,计算 queryid 哈希时会按名称标识表,因此删除并重建表后仍会保留历史。反过来,不同模式中同名的表现在会共享一条记录;如果每个租户使用一个模式,那么每个租户的 select ... from orders 都会合并成一行。把 %Q 加入 log_line_prefix,日志就会带上 queryid。
过去,发送 in ($1, $2)、接着发送 in ($1, $2, $3) 的 ORM 会让视图为每种列表长度各生成一条记录。PostgreSQL 18 会把它们合并成一条,显示为 in ($1 /*, ... */)。使用数组的 = any($1) 仍是更好的习惯:在所有版本中都只有一种语句结构(经得住数据增长的查询)。
动手验证 · lab 30-observability
selectlab.prove('pg_stat_statements is preloaded, compute_query_id is auto, and it tracks top-level statements',current_setting('shared_preload_libraries')like'%pg_stat_statements%'andcurrent_setting('compute_query_id')='auto'andcurrent_setting('pg_stat_statements.track')='top');
selectcount(*)fromprojectswhereid=1;selectcount(*)fromprojectswhereid=2;selectlab.prove('whitespace and constants do not change the queryid',(selectcallsfromlab.stmtswherequery='select count(*) from projects where id = $1')=2);createtablescratch(idint);selectcount(*)fromscratchwhereid=1;droptablescratch;createtablescratch(idint);selectcount(*)fromscratchwhereid=1;selectlab.prove('PostgreSQL 18 hashes tables by name: a dropped and recreated table keeps its queryid',(selectcallsfromlab.stmtswherequery='select count(*) from scratch where id = $1')=2);
selectcount(*)fromuserswhereidin(1,2);selectcount(*)fromuserswhereidin(1,2,3,4,5);selectcount(*)fromuserswhereidin(7,8,9,10,11,12,13,14,15,16);selectcount(*)fromuserswhereidin($1,$2)\bind12\gselectcount(*)fromuserswhereidin($1,$2,$3)\bind123\gselectlab.prove('PostgreSQL 18 collapses IN lists of any length into one entry, shown as in ($1 /*, ... */)',(selectcallsfromlab.stmtswherequery='select count(*) from users where id in ($1 /*, ... */)')=5);
selectlab.prove('the report ran over 20 times less often than the lookup but took over 5 times more total time',(selectcallsfromwwherename='lookup')>20*(selectcallsfromwwherename='report')and(selecttotal_exec_timefromwwherename='report')>5*(selecttotal_exec_timefromwwherename='lookup'));selectlab.prove('the report touches the entire events table every call (blocks per call within 5% of its pages)',(select(shared_blks_hit+shared_blks_read)::numeric/callsfromwwherename='report')between0.95*(selectrelpagesfrompg_classwhererelname='events')and1.05*(selectrelpagesfrompg_classwhererelname='events'));selectlab.prove('the primary key lookup touches 3 pages: root and leaf of users_pkey, plus one heap page',(select(shared_blks_hit+shared_blks_read)::numeric/callsfromwwherename='lookup')between3and3.5and(selectlevelfrombt_metap('users_pkey'))=1);
selectlab.prove('the project list touches about 257 pages for 20 rows (between 200 and 320)',(select(shared_blks_hit+shared_blks_read)/callsfromwwherename='projects')between200and320);selectlab.prove('each insert writes about five WAL records: heap tuple, primary key entry, and three foreign key row locks',(selectwal_records::numeric/callsfromwwherename='insert')between4.9and5.6);-- one insert, read straight from the WAL with pg_walinspect
selectlab.prove('one insert transaction: heap INSERT, B-tree INSERT_LEAF, three heap LOCK records, then COMMIT',(selectarray_agg(resource_manager||'/'||record_typeorderbystart_lsn)frompg_get_wal_records_info(:'wal_start',:'wal_end')wherexid=:'insert_xid'::xid)=array['Heap/INSERT','Btree/INSERT_LEAF','Heap/LOCK','Heap/LOCK','Heap/LOCK','Transaction/COMMIT']);
pg_stat_statements 记录的是过去,pg_stat_activity 记录的是此刻:每个服务器进程一行,包括它正在运行什么、从什么时候开始运行,以及在等待什么。它的 state 可以是 active、idle(等待客户端的下一条语句,这是正常状态),或者 idle in transaction:位于一个打开的事务中,持有自己的锁,并可能持有快照,而应用正在做别的事。几毫秒属于正常情况;几分钟就是缺陷。
没有等待事件的活动会话正在占用 CPU
wait_event_type 和 wait_event 表示后端进程正在等待什么;这两列均为空的 active 行表示该会话正在 CPU 上运行。Lock 是重量级锁(可以锁表,也可以是另一个事务持有行锁时的 transactionid);LWLock 表示内部结构上的争用(解决 LWLock 等待要调整负载,而不是查询);IO 表示读取页面或刷新 WAL;Client 表示等待应用。pg_wait_events(PostgreSQL 17+)记录了每一种等待事件。每秒统计十次,你就得到了一个画像。
深入阅读:这些示例背后的会话
本节的示例来自账户 7 上的三个会话。会话 a 在事务中更新了该账户,然后运行 pg_sleep(60)。会话 b 打开一个事务,读取了一些用户,然后没有提交就进入空闲。会话 c 尝试更新同一个账户,于是被阻塞。睡眠中的会话状态是 active,而不是 idle:它正在运行一条语句,等待事件是 Timeout:PgSleep;应用步骤在数据库内运行时,慢就会呈现为这种样子。会话 a 还有一个 backend_xid,因为它执行过写入;会话 b 没有,因为事务要到第一次写入时才会取得 XID。
动手验证 · lab 30-observability
-- a: begin; update accounts set name = name where id = 7; select pg_sleep(60)
-- b: begin; select count(*) from users where account_id = 7; (then idle)
-- c: update accounts set plan = plan where id = 7; (blocks on a)
selectlab.prove('a session sleeping inside its transaction is active, waiting on Timeout/PgSleep, and has a backend_xid',(selectstate='active'andwait_event_type='Timeout'andwait_event='PgSleep'andbackend_xidisnotnullfrompg_stat_activitywherepid=:pid_a));selectlab.prove('a session waiting for a row waits on Lock/transactionid, and pg_blocking_pids names the holder',(selectwait_event_type='Lock'andwait_event='transactionid'andpg_blocking_pids(pid)=array[:pid_a]frompg_stat_activitywherepid=:pid_c));
selectlab.prove('an open read committed transaction that has not written shows idle in transaction, ClientRead, no xid, no xmin',(selectstate='idle in transaction'andwait_event='ClientRead'andbackend_xidisnullandbackend_xminisnullfrompg_stat_activitywherepid=:pid_b));selectlab.prove('a repeatable read transaction keeps its xmin while idle',(selectstate='idle in transaction'andbackend_xminisnotnullfrompg_stat_activitywherepid=:pid_d));
\watch 会每 100 ms 重新执行一次插入,共执行 600 次,也就是一分钟(count 是 psql 16+ 支持的参数)。每次执行都必须是独立事务:Postgres 每个事务只会为 pg_stat_activity 取一次快照,因此如果把采样循环写成一个 DO 块,它会把第一次采样记录 600 次。
深入阅读:SQL 中的采样循环必须清除快照
每次采样前调用 pg_stat_clear_snapshot(),并用 clock_timestamp() 给行加时间戳,因为 now() 和 statement_timestamp() 也会被冻结。(运行一分钟的 DO 块同样是一个运行一分钟的事务,这本身就是另一个小问题。)
动手验证 · lab 30-observability
begin;selectcount(*)frompg_stat_activitywhereapplication_name='ws_late'\g/dev/nullselectdblink_connect('ws_late',format('dbname=%s application_name=ws_late',current_database()))\g/dev/nullselectpg_sleep(0.3)\g/dev/nullselectlab.prove('inside one transaction pg_stat_activity repeats its first snapshot: a session that connected since is invisible',(selectcount(*)frompg_stat_activitywhereapplication_name='ws_late')=0);selectpg_stat_clear_snapshot()\g/dev/nullselectlab.prove('after pg_stat_clear_snapshot() the next read sees it',(selectcount(*)frompg_stat_activitywhereapplication_name='ws_late')=1);commit;
selectcoalesce(wait_event_type,'CPU')astype,coalesce(wait_event,'running')asevent,count(*)assamples,round(100.0*count(*)/sum(count(*))over(),1)aspct,round(count(*)/600.0,2)asavg_sessions-- / rounds taken
fromwait_sampleswherestate='active'groupby1,2orderbysamplesdesc;
-- ws_cpu spins in a PL/pgSQL loop, ws_sleep runs pg_sleep, ws_hold updates
-- account 9 and idles in its transaction, ws_wait updates the same row;
-- 20 samples, 100 ms apart, with \watch
selectlab.prove('a session running a CPU loop samples as active with no wait event (at least 16 of 20)',(selectcount(*)fromwait_sampleswhereapplication_name='ws_cpu'andstate='active'andwait_eventisnull)>=16);selectlab.prove('a session waiting for a row samples as Lock:transactionid every time',(selectcount(*)filter(where(wait_event_type,wait_event)=('Lock','transactionid'))=count(*)andcount(*)>=18fromwait_sampleswhereapplication_name='ws_wait'));
selectlab.prove('a session idle inside its transaction samples as idle in transaction, Client:ClientRead',(selectbool_and(state='idle in transaction'and(wait_event_type,wait_event)=('Client','ClientRead'))andcount(*)>=18fromwait_sampleswhereapplication_name='ws_hold'));
selectlab.prove('every report read the whole events table: about 2 million rows read per report',(selectseq_tup_read::numericfrompg_stat_user_tableswhererelname='events')/(selectcallsfromwwherename='report')between1900000and2200000);selectlab.prove('events counts more sequential scans than report runs, because each parallel worker''s share counts',(selectseq_scanfrompg_stat_user_tableswhererelname='events')>=1.5*(selectcallsfromwwherename='report'));selectlab.prove('events_pkey shows idx_scan = 0 after hundreds of inserts: uniqueness checks do not count as scans',(selectidx_scanfrompg_stat_user_indexeswhereindexrelname='events_pkey')=0and(selectn_tup_insfrompg_stat_user_tableswhererelname='events')>100);selectlab.prove('a database cloned from a template starts with empty statistics: n_live_tup is 0 for 100,000 users',(selectn_live_tupfrompg_stat_user_tableswhererelname='users')=0and(selectreltuplesfrompg_classwhererelname='users')=100000);
selectlab.prove('the events table is larger than a quarter of shared_buffers, so its scans use a bulkread ring',pg_relation_size('events')>pg_size_bytes(current_setting('shared_buffers'))/4);selectlab.prove('the report finds under 60% of its pages in shared buffers, however often it runs',(select100.0*shared_blks_hit/(shared_blks_hit+shared_blks_read)fromwwherename='report')<60);
deletefromeventswhereid%4=0;selectlab.prove('after deleting a quarter of events, pgstattuple_approx reports 20 to 30% dead tuples',(selectdead_tuple_percentbetween20and30frompgstattuple_approx('events')));selectlab.prove('with nothing vacuumed since, no page is all-visible, so it scans 100% of the table',(selectscanned_percent>99frompgstattuple_approx('events')));