×
技术社区 >  技术博客 >  OceanBase 回收站自动清理失效问题排查与解决方案

OceanBase 回收站自动清理失效问题排查与解决方案

问题现象

回收站配置的recyclebin_object_expire_time 7d,实际很久之前的还存在。

问题原因

1、原因是回收站的数据太多,导致清理任务的执行时间超时;

2、当前版本的超时时间计算方式是10*rpc_timeout =20s

关键信息

关键日志purge_recyclebin涉及的rs日志报错6212,合并的时候会触发一次清理,过滤rs日志需要过滤合并时间段的。

rootservice.log:[2026-aa-bb 10:48:55.438958] INFO [RS] ob_ddl_service.cpp:10584 [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] [It=11] [dc=0] start purge recyclebin except tenant(arg={tenant_id:1007, purge_num:10, expire_time:1777517294814806, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:49:15.375449] WARN [RS] purge_database_in_recyclebin (ob_ddl_operator.cpp:8291) [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] [lt=5] [dc=0] drop_table failed(ret=-6212) 
rootservice.log:[2026-aa-bb 10:49:15.375503] WARN [RS] purge_recyclebin_except_tenant (ob_ddl_service.cpp:10652) [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] purge recyclebin except_tenant(arg={tenant_id:1007, db_name:"__recycle_$_1661850897_1776073472155408"}, recycle_obj={tenant_id:1007, database_id:1124800395216xxx, table_id:1844674407370955xxxx, tablegroup_id:1844674407370955xxxx, object_name:"", object_type:4, tablegroup_name:"", database_name:"", ret=-6212) 
rootservice.log:[2026-aa-bb 10:49:15.376657] WARN [RS] purge_recyclebin_except_tenant (ob_ddl_service.cpp:10685) [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] [lt=9] [dc=0] trans end failed(is_commit=0, temp_ret=-6212) 
rootservice.log:[2026-aa-bb 10:49:15.376664] WARN [RS] purge_tenant_expire_recycle_objects (ob_ddl_service.cpp:10568) [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] [lt=4] [dc=0] failed to purge recyclebin except tenant(ret=-6212) 
rootservice.log:[2026-aa-bb 10:49:15.376745] WARN [RS] purge_expire_recycle_objects (ob_root_service.cpp:6231) [3055235][0][YB42150580D3-00064707DBECC3C8-0-0] [lt=6] [dc=0] failed to purge expire recyclebin objects(ret=-6212, arg={tenant_id:1007, purge_num:10, expire_time:1777517294814806, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:49:15.376878] WARN [RS] purge_recyclebin_objects (ob_root_service.cpp:17110) [3055316] [0][Y0-0000000000000000-0-0] [It=5] [dc=0] purge recyclebin objects failed(ret=-6212, ret="OB_TRANS_STMT_TIMEOUT", current_time=1778122094814806, expire_time:{1777517294814806}, affected_rows={0}, arg={tenant_id:1007, purge_num:10, expire_time:1777517294814806, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:49:16.190304] INFO [RS] ob_root_inspection.cpp:1812 [3055316][0][Y0-0000000000000000-0-0] [It=15] [dc=0] submit purge recyclebin task success(delay=600000000) 
rootservice.log:[2026-05-07 10:49:16.190311] INFO [RS] ob_root_inspection.cpp:1814 [3055316][0][Y0-0000000000000000-0-0] [It=6] [dc=0] purge recyclebin task end(delay=600000000) 
rootservice.log:[2026-aa-bb 10:59:16.190374] INFO [RS] ob_root_inspection.cpp:1796 [3055316][0][Y0-00064707DBECC867-0-0] [It=17] [dc=0] purge recyclebin task begin 
rootservice.log:[2026-aa-bb 10:59:16.431852] INFO [RS] ob_ddl_service.cpp:10584 [3055235][0][YB42150580D3-00064707DBECC869-0-0] [It=17] [dc=0] start purge recyclebin except tenant(arg={tenant_id:1002, purge_num:10, expire_time:1777517956190400, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:59:36.238681] WARN [RS] purge_database_in_recyclebin (ob_ddl_operator.cpp:8291) [3055235][0][YB42150580D3-00064707DBECC869-0-0] [lt=3] [dc=0] drop_table failed(ret=-6212) 
rootservice.log:[2026-aa-bb 10:59:36.238725] WARN [RS] purge_recyclebin_except_tenant (ob_ddl_service.cpp:10652) [3055235][0][YB42150580D3-00064707DBECC869-0-0] purge recyclebin except_tenant(arg={tenant_id:1002, db_name:"__recycle_$_1661850894_1702956738343144"}, recycle_obj={tenant_id:1002, database_id:1100611384xxx, table_id:1844674407370955xxxx, tablegroup_id:1844674407370955xxxx, object_name:"__recycle_$_1661850894_1702956738343144", original_name:"db_test", type:4, tablegroup_name:"", database_name:"", ret=-6212) 
rootservice.log:[2026-aa-bb 10:59:36.239931] WARN [RS] purge_recyclebin_except_tenant (ob_ddl_service.cpp:10685) [3055235][0][YB42150580D3-00064707DBECC869-0-0] [lt=6] [dc=0] trans end failed(is_commit=0, temp_ret=-6212) 
rootservice.log:[2026-aa-bb 10:59:36.239939] WARN [RS] purge_tenant_expire_recycle_objects (ob_ddl_service.cpp:10568) [3055235][0][YB42150580D3-00064707DBECC869-0-0] [lt=6] [dc=0] fail to purge recyclebin except tenant(ret=-6212) 
rootservice.log:[2026-aa-bb 10:59:36.254837] WARN [RS] purge_expire_recycle_objects (ob_root_service.cpp:6231) [3055235][0][YB42150580D3-00064707DBECC869-0-0] [lt=6] [dc=0] failed to purge expire recyclebin objects(ret=-6212, arg={tenant_id:1002, purge_num:10, expire_time:1777517956190400, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:59:36.254945] WARN [RS] purge_recyclebin_objects (ob_root_service.cpp:17110) [3055316] [0][Y0-0000000000000000-0-0] [It=6] [dc=0] purge recyclebin objects failed(ret=-6212, ret="OB_TRANS_STMT_TIMEOUT", current_time=1778122756190400, expire_time:{1777517956190400}, affected_rows={0}, arg={tenant_id:1002, purge_num:10, expire_time:1777517956190400, auto_purge:true}) 
rootservice.log:[2026-aa-bb 10:59:36.870505] INFO [RS] ob_ddl_service.cpp:10584 [3055235][0][YB42150580D3-00064707DBECC870-0-0] [It=10] [dc=0] start purge recyclebin except tenant(arg={tenant_id:1019, purge_num:10, expire_time:1777517956190400, auto_purge:true})

问题的风险及影响

回收站数据过多,占用较多的磁盘。

适用版本

本文问题及解决方案适用于OBServer 3.x版本

解决方法

方法一:手动执行purge database recycle_name;

方法二:业务低峰期,修改rpc_timeout,将超时调大,手动执行purge recyclein,将回收站清理;

①alter system set rpc_timeout = '30s'

②purge recyclebin;

如果此超时执行purge recyclebin仍然超时,则可以按需调大超时;

注意点:此超时时间是全局的rpc超时时间,回收站清理完成之后,需要调回去;

alter system set rpc_timeout = '2s';

如果在操作过程中遇到问题,欢迎在评论区留言交流,后续我们会持续分享更多实战运维干货,记得关注不迷路,下次见~

精选推荐