【一个小发现】VictoriaMetrics中:vm-storage的备份文件,无法给单机版使用
阅读原文时间:2023年07月08日阅读:3

首先导入一个按天的备份:

vmrestore-prod \
  -configFilePath="/etc/cos/config.ini" \
  -credsFilePath="/etc/cos/creds.ini" \
  -customS3Endpoint="https://cos.ap-guangzhou.myqcloud.com" \
  -src="s3://xxxxxx/vm-storage/2021-12-27/" \
  -storageDataPath="/var/victoria-metrics/data/2021-12-27/" \
  -httpListenAddr="0.0.0.0:8420" \
  -memory.allowedBytes=2GB \
  -concurrency=10 \
  -maxBytesPerSecond=100MB

启动vm单机版:

victoria-metrics-prod \
  -httpListenAddr="0.0.0.0:8428" \
  -memory.allowedBytes=7GB \
  -retentionPeriod=1y \
  -search.logSlowQueryDuration=3s \
  -search.maxConcurrentRequests=500 \
  -search.maxUniqueTimeseries=3000000 \
  -storageDataPath="/var/victoria-metrics/data/2021-12-27/"

启动的时候出现如下错误:

2021-12-29T02:51:07.296Z        panic   VictoriaMetrics/lib/storage/index_db.go:3055    FATAL: cannot parse row starting with nsPrefix 1 during merge: cannot unmarshal tag from tag->metricIDs row "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x01flag\x01\x16\xc4\xe8'6FU\xec\x16\xc4\xe8'6FV\v\x16\xc4\xe8'6FV\x11\x16\xc4\xe8'6FV?\x16\xc4\xe8'6FVV\x16\xc4\xe8'6FVk\x16\xc4\xe8'6FVl\x16\xc4\xe8'6FVm\x16\xc4\xe8'6FVn\x16\xc4\xe8'6FVo\x16\xc4\xe8'6FVp\x16\xc4\xe8'6FVq\x16\xc4\xe8'6FVr\x16\xc4\xe8'6FVs\x16\xc4\xe8'6FV\xcb\x16\xc4\xe8'6FV\xcc\x16\xc4\xe8'6FV\xcd\x16\xc4\xe8'6FV\xce\x16\xc4\xe8'6FV\xcf\x16\xc4\xe8'6FV\xd0\x16\xc4\xe8'6FV\xd1\x16\xc4\xe8'6FV\xd2\x16\xc4\xe8'6FV\xd3\x16\xc4\xe8'6FV\xd4\x16\xc4\xe8'6FV\xd5\x16\xc4\xe8'6FV\xd6\x16\xc4\xe8'6FV\xd7\x16\xc4\xe8'6FV\xd8\x16\xc4\xe8'6FV\xd9\x16\xc4\xe8'6FV\xda\x16\xc4\xe8'6FW#\x16\xc4\xe8'6FW$\x16\xc4\xe8'6FW%\x16\xc4\xe8'6FW&\x16\xc4\xe8'6FW'\x16\xc4\xe8'6FW(\x16\xc4\xe8'6FW)\x16\xc4\xe8'6FW*\x16\xc4\xe8'6FW+\x16\xc4\xe8'6FW,\x16\xc4\xe8'6FW-\x16\xc4\xe8'6FW.\x16\xc4\xe8'6FW/": cannot unmarshal key: unsupported escaped char:
panic: FATAL: cannot parse row starting with nsPrefix 1 during merge: cannot unmarshal tag from tag->metricIDs row "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x01flag\x01\x16\xc4\xe8'6FU\xec\x16\xc4\xe8'6FV\v\x16\xc4\xe8'6FV\x11\x16\xc4\xe8'6FV?\x16\xc4\xe8'6FVV\x16\xc4\xe8'6FVk\x16\xc4\xe8'6FVl\x16\xc4\xe8'6FVm\x16\xc4\xe8'6FVn\x16\xc4\xe8'6FVo\x16\xc4\xe8'6FVp\x16\xc4\xe8'6FVq\x16\xc4\xe8'6FVr\x16\xc4\xe8'6FVs\x16\xc4\xe8'6FV\xcb\x16\xc4\xe8'6FV\xcc\x16\xc4\xe8'6FV\xcd\x16\xc4\xe8'6FV\xce\x16\xc4\xe8'6FV\xcf\x16\xc4\xe8'6FV\xd0\x16\xc4\xe8'6FV\xd1\x16\xc4\xe8'6FV\xd2\x16\xc4\xe8'6FV\xd3\x16\xc4\xe8'6FV\xd4\x16\xc4\xe8'6FV\xd5\x16\xc4\xe8'6FV\xd6\x16\xc4\xe8'6FV\xd7\x16\xc4\xe8'6FV\xd8\x16\xc4\xe8'6FV\xd9\x16\xc4\xe8'6FV\xda\x16\xc4\xe8'6FW#\x16\xc4\xe8'6FW$\x16\xc4\xe8'6FW%\x16\xc4\xe8'6FW&\x16\xc4\xe8'6FW'\x16\xc4\xe8'6FW(\x16\xc4\xe8'6FW)\x16\xc4\xe8'6FW*\x16\xc4\xe8'6FW+\x16\xc4\xe8'6FW,\x16\xc4\xe8'6FW-\x16\xc4\xe8'6FW.\x16\xc4\xe8'6FW/": cannot unmarshal key: unsupported escaped char: 

goroutine 43 [running]:
github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logMessage({0xac07e2, 0x42}, {0xc001318c00, 0x3e3}, 0x2)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:270 +0x994
github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logLevelSkipframes(0x1, {0xac07e2, 0x167}, {0xaef1c1, 0xc0013c3001}, {0xc000a85558, 0x0, 0xc000a85458})
        github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:138 +0x1a5
github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logLevel(...)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:130
github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.Panicf(...)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:126
github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.mergeTagToMetricIDsRowsInternal({0xc001294000, 0xc000a85628, 0x417de6}, {0xc0013c3000, 0x1e6, 0x2b61}, 0x1)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/index_db.go:3055 +0x61e
github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.mergeTagToMetricIDsRows({0xc001294000, 0x7f19c521c318, 0xc00099f000}, {0xc0013c3000, 0xc000a857a8, 0x7355e6})
        github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/index_db.go:3011 +0x29
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*blockStreamMerger).flushIB(0xc0006c0140, 0xc00126be1a, 0xc000a85d48, 0xc000082680)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/merge.go:169 +0x2a3
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*blockStreamMerger).Merge(0xc0006c0140, 0xc0000a68b8, 0x3, 0x1a0, 0xc94d08)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/merge.go:134 +0x42a
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.mergeBlockStreams(0xc000a72000, 0xc00008e120, {0xc0000a68b8, 0x3, 0x3}, 0x314, 0xc000053a28, 0x417de6)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/merge.go:36 +0x10c
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*Table).mergeParts(0xc0006a6e00, {0xc0000a67c8, 0x3, 0x3}, 0xc000098f01, 0x0)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/table.go:934 +0x854
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*Table).mergeExistingParts(0xc0006a6e00, 0xc0)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/table.go:806 +0x118
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*Table).partMerger(0xc0006a6e00)
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/table.go:820 +0x6d
github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*Table).startPartMergers.func1()
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/table.go:786 +0x26
created by github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset.(*Table).startPartMergers
        github.com/VictoriaMetrics/VictoriaMetrics/lib/mergeset/table.go:785 +0x2c

VM单机版看来和群集版是不太一致的,未来看能不能通过导入导出的方法来互通数据。