Remote threads 是僅當(dāng) Hive metastore 作為單獨(dú)的服務(wù)運(yùn)行是啟動(dòng)，請(qǐng)求需要開啟 compactor。

有以下幾種：

1. AcidOpenTxnsCounterService 統(tǒng)計(jì)當(dāng)前 open 的事務(wù)數(shù)

從表 TXNS 中統(tǒng)計(jì)狀態(tài)為 open 的事務(wù)。此事務(wù)數(shù)量可以再 hive metrics 中。

2. AcidHouseKeeperService

定期調(diào)用 txnHandler.performTimeOuts();
默認(rèn)的 txn 的 timeout是 300s。刪除300秒沒(méi)有心跳的事務(wù)。

TXN_TIMEOUT("metastore.txn.timeout", "hive.txn.timeout", 300, TimeUnit.SECONDS,"time after which transactions are declared aborted if the client has not sent a heartbeat."),

DumpDirCleanerTask

dump dir 是

REPLDIR("hive.repl.rootdir","/user/hive/repl/","HDFS root dir for all replication dumps."),

此目錄的 ttl 是

REPL_DUMPDIR_TTL("hive.repl.dumpdir.ttl", "7d",new TimeValidator(TimeUnit.DAYS),

AcidCompactionHistoryService

周期行調(diào)用 txnHandler.purgeCompactionHistory();
purgeCompactionHistory 的內(nèi)容如下。對(duì)于可以合并的 entity(分區(qū)或者表（非分區(qū)表）)，只保留最后幾次的history。

/*** For any given compactable entity (partition, table if not partitioned) the history of compactions* may look like "sssfffaaasffss", for example.  The idea is to retain the tail (most recent) of the* history such that a configurable number of each type of state is present.  Any other entries* can be purged.  This scheme has advantage of always retaining the last failure/success even if* it's not recent.* @throws MetaException*/@RetrySemantics.SafeToRetryvoid purgeCompactionHistory() throws MetaException;

RuntimeStatsCleanerTask

如果此參數(shù)設(shè)置true，則query reexecution 的時(shí)候會(huì)收集統(tǒng)計(jì)信息。

HIVE_QUERY_REEXECUTION_ALWAYS_COLLECT_OPERATOR_STATS("hive.query.reexecution.always.collect.operator.stats", false,"If sessionstats are enabled; this option can be used to collect statistics all the time"),

RawStore ms = HMSHandler.getMSForConf(conf);
int maxRetainSecs=(int) MetastoreConf.getTimeVar(conf, MetastoreConf.ConfVars.RUNTIME_STATS_MAX_AGE, TimeUnit.SECONDS);int deleteCnt = ms.deleteRuntimeStats(maxRetainSecs);

maxRetainSecs 默認(rèn) 3天。
刪除表 RUNTIME_STATS 中 createTime <= 3天前的記錄

RUNTIME_STATS_MAX_AGE("runtime.stats.max.age", "hive.metastore.runtime.stats.max.age", 86400 * 3, TimeUnit.SECONDS,"Stat entries which are older than this are removed.")

AcidWriteSetService

默認(rèn)每60秒的周期調(diào)用一次 txnHandler.performWriteSetGC(); .

執(zhí)行類似以下的語(yǔ)句。

select min(txn_id) commitHighWaterMark from TXNS where txn_state='OPEN';
delete from WRITE_SET where ws_commit_id < commitHighWaterMark;

EventCleanerTask

定期執(zhí)行以下語(yǔ)句。

RawStore ms = HMSHandler.getMSForConf(conf);
long deleteCnt = ms.cleanupEvents();

cleanupEvents 刪除 PARTITION_EVENTS表的過(guò)期數(shù)據(jù)。
PARTITION_EVENTS 表的數(shù)據(jù)，進(jìn)在 hcatalog 中使用

MaterializationsRebuildLockCleanerTask

從表 MATERIALIZATION_REBUILD_LOCKS 中找到過(guò)期數(shù)據(jù)，然后刪除。