积少成多, 聚沙成塔

Kiritow's Homepage

Posted on Disqus:

PVE 8.2.4 ISO安装, 由于同一台机器上其他的磁盘上原来已经安装了 PVE 7.4 所以安装过程中提示是否要rename到pve-OLD 而且不能跳过.

安装之后原来磁盘上的PVE会被保留, 但这也造成了我们没法把原来的磁盘用作其他用途 (Disk Manage会报错占用了, 没法Wipe Disk)

参考以下步骤删除掉 pve-OLD

删除lv

1
2
3
lvdisplay
lvchange -an /dev/pve-OLD-0BFE9176
lvremove /dev/pve-OLD-0BFE9176

删除vg

1
2
vgdisplay
vgremove /pve-OLD-0BFE9176

删除pv

1
2
pvdisplay
pvremove /dev/nvme1n1p3

这样原来的磁盘就可以用作其他用途了.

参考

How to delete PVE Old

jordanhillis/pvekclean

Posted on Disqus:

PVE 8.2.4 版本iso安装之后 syslog里一直报错:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
...
Aug 31 22:29:31 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:29:41 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:29:51 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:01 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:11 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:21 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:31 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:41 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:30:51 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:31:01 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:31:11 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:31:21 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 31 22:31:31 pve-sg pve-firewall[1727]: status update error: iptables_restore_cmdlist: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
...

如果直接操作 ipset, iptables -vnL 命令会提示:

1
2
3
4
5
6
ipset v7.10: Cannot open session to kernel.
command 'ipset save' failed: exit code 1
...

iptables v1.8.9 (legacy): can't initialize iptables table...: Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

看起来是因为新版本启用了ebtables但是某些地方还是有问题, 没找到特别好的解决方案, 目前的解决办法是 Datacenter -> Firewall -> Options, 设置 ebtables 为 No. 重启后错误日志就消失了.

另外还有个奇怪的问题, 默认安装启用 ebtables 的情况下, CIFS mount会失败, 但是 dmesg 和 journalctl 里看不到错误. 目前也没有找到比较可靠的解决方案, 关闭 ebtables 之后正常了.

参考

status update error: iptables_restore_cmdlist

Posted on Disqus:

WebUI 上只有添加 Storage 的地方, 没有删除的方式, 需要手动进入Node Shell 进行删除

这里假设要删除的存储是 localssd

  1. 取消 mount 服务 (会自动umount)
1
2
systemctl status mnt-pve-localssd.mount
systemctl disable --now mnt-pve-localssd.mount
  1. 确定mount点已卸载并删除mount点
1
2
ls -al /mnt/pve/localssd
rmdir /mnt/pve/localssd
  1. 删除 mount 文件
1
rm /etc/systemd/system/mnt-pve-localssd.mount
  1. 删除存储配置
1
nano /etc/pve/storage.cfg

找到localssd这一段并删除

1
2
3
4
5
6
dir: localssd
        path /mnt/pve/localssd
        content images
        is_mountpoint 1
        nodes pve-sg
        shared 0

参考

[SOLVED] Removing old Storage from GUI

Proper way to remove old kernels from PVE 8.0.4 & which are safe to remove

Linux Quick Tip: How to Delete or Remove LVM volumes

Posted on Disqus:

  1. 制作一个Ubuntu Desktop启动盘, 使用这个U盘启动系统

  2. 备份全盘内容

1
dd if=/dev/nvme1n1 of=/mnt/download/diskbackup/diskc.raw bs=4M

其中 of 参数指定的输出文件位置可以在网络上, 例如SMB共享. 需要提前通过类似这样的命令 mount

1
2
sudo mkdir /mnt/download
sudo mount -t cifs <SMB地址> /mnt/download -o rw,uid=1000,gid=1000

如果中间不小心中断了的话, 也可以重新引导进入Ubuntu Desktop, 使用如下命令继续备份

1
dd if=/dev/nvme1n1 of=/mnt/download/diskbackup/diskc.raw bs=4M seek=123456789 skip=123456789 iflag=skip_bytes oflag=seek_bytes

其中 123456789 是已经备份生成出来的文件的长度, 建议稍微减小一些数值, 比如已经备份了157GiB (168577466368), 那就可以选择 150GiB作为继续点 (161061273600)

skip_bytes, seek_bytes 表示skip和seek的数值是字节, 而不是bs的数量. 否则实际上跳过的字节会变成 123456789 * 4M. iflag, oflag不要写反了

  1. (可选) 生成出来的文件可以转换成 qcow2 格式, 占用空间更小. 源文件越大转换时间越长.
1
qemu-img convert -O qcow2 diskc.raw diskc.qcow2
  1. 创建Proxmox VM

因为源物理机是Windows系统, 此处按照Windows系统创建.

不挂载ISO, BIOS选择OVMF (UEFI), EFI Disk 正常创建即可.

把文件放到Proxmox指定路径下. 如果该路径在SMB上则需要保证磁盘文件的owner正确, 而且权限正确, 否则vm无法启动报错Permission denied (此处跟stackexchange上遇到的情况不太一样, 我只需要chown+chmod 644就可以让vm正常启动了)

  1. 直接使用raw镜像启动, 进入Grub且退出后无法自动进入系统

虽然不知道是什么原因导致的, 但是可以通过在Grub下输入这些来引导windows. 其中 (hd0,gpt1) 是通过grub下 ls 命令得到的.

1
2
3
4
5
insmod part_gpt
insmod chain
set root=(hd0,gpt1)
chainloader /EFI/Microsoft/Boot/bootmgfw.efi
boot

参考

Translating bash to python; “dd” command “iflag=skip_bytes” how can be converted?

Determine the size of a block device

Resuming a DD of an entire disk

How to output file from the specified offset, but not “dd bs=1 skip=N”?

2.4. Converting Between RAW and QCOW2

How to use QEMU/KVM virtual machine disk image on SMB/CIFS network share: Permission denied

How to start a windows partition from the Grub command line

Posted on Edited on Disqus:

只讨论Linux VM

在PVE界面上操作Clone之后, 检查一下网卡MAC是不是不一样. 老版本PVE似乎有bug, clone VM的时候会把mac也复制. IP一样没问题, 只要都是DHCP等下重启就可以了.

  1. 打开新创建出来的VM, 修改hostname

sudo vim /etc/hostname

  1. 修改hosts文件, 把当前host的名字加上

sudo vim /etc/hosts

  1. 重置 machine-id
1
2
3
echo -n | sudo tee /etc/machine-id
sudo rm /var/lib/dbus/machine-id
sudo ln -s /etc/machine-id /var/lib/dbus/machine-id
  1. 重新配置 OpenSSH服务器 (复制出来的VM会有一样的服务器公钥,不安全)
1
2
sudo rm -rf /etc/ssh/ssh_host*
sudo dpkg-reconfigure openssh-server
  1. 重启机器即可

sudo reboot

Posted on Edited on Disqus:

在GCP上配置IPSec隧道

  1. 创建对等VPN网关 (Peering VPN Gateway)

  1. 创建Cloud VPN网关和隧道, 此时就是客户端的角色，这里要选择网卡数量，最多两个网卡，每个网卡分配一个公网IP

此时还不能配置BGP对话, 需要先建立起连接才可以

搭建IPSec链接

安装 Strongswan (IPSec VPN) 

1
sudo apt install strongswan strongswan-pki

配置ipsec, 配置文件路径 /etc/ipsec.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
config setup
    charondebug="all"
    uniqueids=yes
    strictcrlpolicy=no

conn %default
    ikelifetime=600m # 36,000 s
    keylife=180m # 10,800 s
    rekeymargin=3m
    keyingtries=3
    keyexchange=ikev2
    mobike=no
    ike=aes256gcm16-sha512-modp4096  # 这里的参数需要跟GCP平台匹配
    esp=aes256gcm16-sha512-modp8192  # 这里的参数需要跟GCP平台匹配
    authby=psk

conn net-net1
    leftupdown="/var/lib/strongswan/ipsec-vti.sh 0 169.254.232.77/32 169.254.232.78/32"  # 这里是p2p通道的GCP侧IP和本端IP
    left=10.0.8.4 # In case of NAT set to internal IP, e.x. 10.164.0.6
    leftid=10.0.8.4
    leftsubnet=0.0.0.0/0
    leftauth=psk
    right={GCP平台的公网IP}
    rightid=%any
    rightsubnet=0.0.0.0/0
    rightauth=psk
    type=tunnel
    # auto=add - means strongSwan won't try to initiate it
    # auto=start - means strongSwan will try to establish connection as well
    # Note that Google Cloud will also try to initiate the connection
    auto=start
    # dpdaction=restart - means strongSwan will try to reconnect if Dead Peer Detection spots
    #                  a problem. Change to 'clear' if needed
    dpdaction=restart
    mark=%unique
    # mark=1001
    # reqid=1001

conn net-net2
    leftupdown="/var/lib/strongswan/ipsec-vti.sh 1 169.254.155.53/32 169.254.155.54/32"  # 同上
    left=10.0.8.4 # In case of NAT set to internal IP, e.x. 10.164.0.6
    leftid=10.0.8.4
    leftsubnet=0.0.0.0/0
    leftauth=psk
    right={GCP平台的公网IP}
    rightid=%any
    rightsubnet=0.0.0.0/0
    rightauth=psk
    type=tunnel
    # auto=add - means strongSwan won't try to initiate it
    # auto=start - means strongSwan will try to establish connection as well
    # Note that Google Cloud will also try to initiate the connection
    auto=start
    # dpdaction=restart - means strongSwan will try to reconnect if Dead Peer Detection spots
    #                  a problem. Change to 'clear' if needed
    dpdaction=restart
    mark=%unique
    # mark=1002
    # reqid=1002

其中vti脚本内容如下, 路径 /var/lib/strongswan/ipsec-vti.sh (源自网络,见参考)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
#!/bin/bash
set -o nounset
set -o errexit

IP=$(which ip)

PLUTO_MARK_OUT_ARR=(${PLUTO_MARK_OUT//// })
PLUTO_MARK_IN_ARR=(${PLUTO_MARK_IN//// })

VTI_TUNNEL_ID=${1}
VTI_REMOTE=${2}
VTI_LOCAL=${3}

LOCAL_IF="${PLUTO_INTERFACE}"
VTI_IF="vti${VTI_TUNNEL_ID}"
# GCP's MTU is 1460, so it's hardcoded
GCP_MTU="1460"
# ipsec overhead is 73 bytes, we need to compute new mtu.
VTI_MTU=$((GCP_MTU-73))

case "${PLUTO_VERB}" in
    up-client)
        ${IP} link add ${VTI_IF} type vti local ${PLUTO_ME} remote ${PLUTO_PEER} okey ${PLUTO_MARK_OUT_ARR[0]} ikey ${PLUTO_MARK_IN_ARR[0]}
        ${IP} addr add ${VTI_LOCAL} remote ${VTI_REMOTE} dev "${VTI_IF}"
        ${IP} link set ${VTI_IF} up mtu ${VTI_MTU}

        # Disable IPSEC Policy
        sysctl -w net.ipv4.conf.${VTI_IF}.disable_policy=1

        # Enable loosy source validation, if possible. Otherwise disable validation.
        sysctl -w net.ipv4.conf.${VTI_IF}.rp_filter=2 || sysctl -w net.ipv4.conf.${VTI_IF}.rp_filter=0

        # If you would like to use VTI for policy-based you should take care of routing by yourselv, e.x.
        #if [[ "${PLUTO_PEER_CLIENT}" != "0.0.0.0/0" ]]; then
        #    ${IP} r add "${PLUTO_PEER_CLIENT}" dev "${VTI_IF}"
        #fi
        ;;
    down-client)
        ${IP} tunnel del "${VTI_IF}"
        ;;
esac

# Enable IPv4 forwarding
sysctl -w net.ipv4.ip_forward=1

# Disable IPSEC Encryption on local net
sysctl -w net.ipv4.conf.${LOCAL_IF}.disable_xfrm=1
sysctl -w net.ipv4.conf.${LOCAL_IF}.disable_policy=1

注: 此处VTI也适用于Route-based/Policy-based IPSec, 对应的vti local ip 写自己的eth0网卡ip, remote ip写GCP的公网IP, 跟right=那个参数保持一致即可, 不使用脚本也可以通过命令手动操作:

1
2
3
4
5
6
7
8
9
10
11
12
sudo ip tunnel add ipsec0 local <eth0 IP> remote <GCP IP> mode vti key 1  # 这里的key需要跟ipsec status里面提到的key一样. 也可以在配置ipsec隧道的时候加上key参数来指定.
sudo ip add add 10.10.0.1/30 dev ipsec0  # 这里随便加, 如果对端是Policy-based则加一个对端允许的IP, Route-based根据对方路由表规划加
sudo ip link set dev ipsec0 up
sudo ip route add 10.20.0.0/16 dev ipsec0 src 10.10.0.1  # 这里如果是Policy-based的则加对端的subnet, Route-based的根据路由表规划加.
# 这两条是用来关闭VTI的policy, 让VTI跟普通的interface行为一样.
sudo sysctl net.ipv4.conf.ipsec0.disable_policy=1
sudo sysctl net.ipv4.conf.ipsec0.rp_filter=2
# 这两条用来关闭eth0的XFRM和Policy, 让ipsec流量不再透明加解密.
sudo sysctl net.ipv4.eth0.disable_xfrm=1
sudo sysctl net.ipv4.eth0.disable_policy=1
# 帮助对端访问更多内网(可选)
sudo iptables -t nat -A POSTROUTING -o eth0 -s 10.20.0.0/16 -j MASQUERADE

调整 charon 配置, 不要安装路由 (下一步由bird2配置具体的路由, 否则会像WireGuard一样安装一个0.0.0.0/0的路由) /etc/strongswan.d/vti.conf 这里其实有点像wg的Table=off

1
2
3
4
charon {
    # We will handle routes by ourselves
    install_routes = no
}

配置PSK, 编辑 /etc/ipsec.secrets. PSK最长可以有63个字符 (strongswan的限制, RFC标准比这个长)

1
2
3
4
5
6
# This file holds shared secrets or RSA private keys for authentication.

# RSA private key for this host, authenticating it to any other host
# which knows the public part.

{GCP平台的公网IP} : PSK "pskhere"

配置防火墙/端口转发(如有需要), ipsec使用: 500/udp,4500/udp,4510/udp,4511/udp

通过命令启动/关闭/查看ipsec隧道

启动: sudo ipsec start

关闭: sudo ipsec stop

查看状态 sudo ipsec statusall (也可以使用 sudo ipsec status)

看到ESTABLISHED字样就表明已经建立连接了 (这里只能看到一个因为写文章的时候已经开始回收测试环境了)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Status of IKE charon daemon (strongSwan 5.9.5, Linux 5.15.0-107-generic, x86_64):
  uptime: 13 minutes, since Jul 25 14:04:56 2024
  malloc: sbrk 3100672, mmap 0, used 1427696, free 1672976
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 4
  loaded plugins: charon aesni aes rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm drbg attr kernel-netlink resolve socket-default connmark stroke updown eap-mschapv2 xauth-generic counters
Listening IP addresses:
  10.0.8.4
  169.254.232.78
Connections:
    net-net1:  10.0.8.4...34.128.44.254  IKEv2, dpddelay=30s
    net-net1:   local:  [10.0.8.4] uses pre-shared key authentication
    net-net1:   remote: uses pre-shared key authentication
    net-net1:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL, dpdaction=restart
    net-net2:  10.0.8.4...<reducted>  IKEv2, dpddelay=30s
    net-net2:   local:  [10.0.8.4] uses pre-shared key authentication
    net-net2:   remote: uses pre-shared key authentication
    net-net2:   child:  0.0.0.0/0 === 0.0.0.0/0 TUNNEL, dpdaction=restart
Security Associations (1 up, 0 connecting):
    net-net1[4]: ESTABLISHED 13 minutes ago, 10.0.8.4[10.0.8.4]...<reducted>
    net-net1[4]: IKEv2 SPIs: <reducted>, pre-shared key reauthentication in 9 hours
    net-net1[4]: IKE proposal: AES_GCM_16_256/PRF_HMAC_SHA2_512/MODP_4096
    net-net1{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: <reducted>
    net-net1{1}:  AES_GCM_16_256, 5986 bytes_i, 6007 bytes_o (96 pkts, 4s ago), rekeying in 2 hours
    net-net1{1}:   0.0.0.0/0 === 0.0.0.0/0

在GCP上配置BGP链路

创建高可用VPN, 本质是一个隧道组，要实现HA，最少需要 2 中的两个网卡每个网卡配置一条隧道到1，最多可以配置 n*m 个隧道

搭建BGP链路

安装BIRD2 sudo apt install bird2

编写bird2配置 /etc/bird/bird.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
log syslog all;
debug protocols all;

protocol device {
  scan time 10;
}

protocol direct {
  ipv4;       # Connect to default IPv4 table
}

protocol kernel {
  ipv4 {      # Connect protocol to IPv4 table by channel
        import none;
        export all;       # Export to protocol. default is export none
  };
}

protocol static {
  ipv4;
  route 192.168.48.0/24 via 10.181.0.2;
  route 169.254.232.77/32 via 192.254.232.78;
}

protocol bgp gcp_vpc_a_tun1 {
  local 169.254.232.78 as 65003;  # 本端ASN
  neighbor 169.254.232.77 as 64520;  # 对端ASN
  multihop;
  keepalive time 20;
  hold time 60;
  graceful restart aware;
  ipv4 {
    import filter {
      gw = 169.254.232.77;  # 不知道为什么一定要加这个
      accept;
    };
    import limit 10 action warn;
    export filter{
      if (net ~ 192.168.0.0/16) then accept;  # 这里是输出路由给GCP时的过滤
      else reject;
    };
    export limit 10 action warn;
  };
}

protocol bgp gcp_vpc_a_tun2 {
  local 169.254.155.54 as 65003;  # 本端ASN
  neighbor 169.254.155.53 as 64520;  # 对端ASN
  multihop;
  keepalive time 20;
  hold time 60;
  graceful restart aware;
  ipv4 {
    import filter {
      gw = 169.254.155.53;
      accept;
    };
    import limit 10 action warn;
    export filter{
      if (net ~ 192.168.0.0/16) then accept;
      else reject;
    };
    export limit 10 action warn;
  };
}

重加载bird配置 sudo birdc configure

查看bird协议状态 sudo birdc show protocol all

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
BIRD 2.0.8 ready.

<中间省略>

gcp_vpc_a_tun1 BGP        ---        up     14:04:57.581  Established
  BGP state:          Established
    Neighbor address: 169.254.232.77
    Neighbor AS:      64520
    Local AS:         65003
    Neighbor ID:      169.254.232.77
    Local capabilities
      Multiprotocol
        AF announced: ipv4
      Route refresh
      Graceful restart
      4-octet AS numbers
      Enhanced refresh
      Long-lived graceful restart
    Neighbor capabilities
      Multiprotocol
        AF announced: ipv4
      Route refresh
      Graceful restart
        Restart time: 60
        Restart recovery
        AF supported: ipv4
        AF preserved: ipv4
      4-octet AS numbers
    Session:          external multihop AS4
    Source address:   169.254.232.78
    Hold timer:       43.240/60
    Keepalive timer:  1.847/20
  Channel ipv4
    State:          UP
    Table:          master4
    Preference:     100
    Input filter:   (unnamed)
    Output filter:  (unnamed)
    Import limit:   10
      Action:       warn
    Export limit:   10
      Action:       warn
    Routes:         1 imported, 0 exported, 1 preferred
    Route change stats:     received   rejected   filtered    ignored   accepted
      Import updates:              1          0          0          0          1
      Import withdraws:            0          0        ---          0          0
      Export updates:              3          1          2        ---          0
      Export withdraws:            0        ---        ---        ---          0
    BGP Next hop:   169.254.232.78
    IGP IPv4 table: master4

gcp_vpc_a_tun2 BGP        ---        start  14:04:56.161  Active        Socket: Connection closed  同上,如果有两个对端且配置正确这里会是Established,但是对端已经被回收了
  BGP state:          Active
    Neighbor address: 169.254.155.53
    Neighbor AS:      64520
    Local AS:         65003
    Connect delay:    2.781/5
    Last error:       Socket: Connection closed
  Channel ipv4
    State:          DOWN
    Table:          master4
    Preference:     100
    Input filter:   (unnamed)
    Output filter:  (unnamed)
    Import limit:   10
      Action:       warn
    Export limit:   10
      Action:       warn
    IGP IPv4 table: master4

localnet   OSPF       master4    up     14:04:56.161  Alone
  Channel ipv4
    State:          UP
    Table:          master4
    Preference:     150
    Input filter:   (unnamed)
    Output filter:  (unnamed)
    Routes:         0 imported, 3 exported, 0 preferred
    Route change stats:     received   rejected   filtered    ignored   accepted
      Import updates:              0          0          0          0          0
      Import withdraws:            0          0        ---          0          0
      Export updates:              4          0          0        ---          4
      Export withdraws:            0        ---        ---        ---          0

没问题的话就能看到GCP推送过来的路由了 ip route

1
2
3
4
5
6
...
10.0.1.0/24 via 169.254.232.77 dev vti0 proto bird metric 32
...
169.254.232.77 dev vti0 proto kernel scope link src 169.254.232.78
169.254.232.77 dev vti0 proto bird scope link metric 32
...

参考

How to set up a VPN between strongSwan and Cloud VPN

Using Strongswan to setup site to site IPsec VPN between GCP and Digital Ocean

Configuring Site-to-Site IPSec VPN on Ubuntu using Strongswan

Route-based VPN - strongswan Documentation

Figuring out how ipsec transforms work in Linux

Establish VPN tunnel for in-house machine to access GCP network

KB: Connecting OpenWRT/LEDE router to Azure Virtual Network Gateway (IKEv2)

Secure site-to-site connection with Linux IPsec VPN

Google Cloud HA VPN interoperability guide for AWS

How does IPsec VPN really work?

BGP 对等会话

Google史一样的文档: Create two fully configured HA VPN gateways that connect to each other | Establish BGP sessions

GCP Networking: Part 2 Cloud Router

Foo over UDP

IPsec vs. WireGuard

The Noise Protocol Framework

rp_filter - Sysctl Explorer

howto/Bird2 - dn42 虽然是dn42的教程但是非常管用

Multiple connections with longer than 64 byte PSK keys fail with “MAC mismatched”

Posted on Edited on Disqus:

Gym Class Heroes - Make me your radio (Stereo Hearts) (Lyrics)

She Wolf Youtube上一大堆原神MMD的BGM

Sea of Tranquility - Bemax F1的小曲 《F1の小曲》《杆位の小曲》《最速の小曲》Sea of Tranquility-Bemax 应用: （无水印）F1赛车手出场meme报幕遮挡原版素材

Chinese 2024最火歌曲DJ ✔2024最火歌曲DJ Remix 抖音版 🎶 最好的音樂Chinese DJ remix 👍 Douyin Dj抖音版2024

往期优秀作品推荐

2024年1月-2月

Posted on Edited on Disqus:

不知道为啥缺氧没做这个功能, 创意工坊里也没有做这个的人, 只好自己写个脚本来做了. 特别感谢 @RoboPhred 提供的 oni-save-parser

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
const { readFileSync, writeFileSync } = require("fs");
const { parseSaveGame } = require('oni-save-parser');

const modsFile = JSON.parse(readFileSync('mods.json'))
console.log('parsing game... (this might take a while)')
const gameSave = parseSaveGame(readFileSync('world.sav').buffer, {
    versionStrictness: 'none'
})

const gameActiveMods = gameSave.world.active_mods
const steamMods = modsFile.mods

const gameActiveModIds = gameActiveMods.map(m => m.id)
const steamModIds = steamMods.map(m => m.label.id)

const missingMods = gameActiveModIds.filter(id => steamModIds.indexOf(id) == -1).map(id => gameActiveMods[gameActiveMods.findIndex(m => m.id == id)])
if (missingMods.length > 0) {
    console.log('The following mods are missing, please install/subscribe from Steam')
    console.log(missingMods)
    process.exit(1)
}

console.log(steamMods.map(m => m.label.title))

steamMods.sort((a, b) => {
    const modIDA = a.label.id
    const modIDB = b.label.id

    const indexA = gameActiveModIds.indexOf(modIDA)
    const indexB = gameActiveModIds.indexOf(modIDB)

    // put active mods before inactive mods
    if (indexA != -1 && indexB != -1) {
        return indexA - indexB
    }

    if (indexA != -1) {
        return -1
    }

    if (indexB != -1) {
        return 1
    }

    // sort inactive mods by name
    return a.label.title.toLowerCase().localeCompare(b.label.title.toLowerCase())
})

console.log(steamMods.map(m => m.label.title))

writeFileSync('mods.json', JSON.stringify(modsFile))

Posted on Edited on Disqus:

经手的一部分项目使用了rate.Limiter作为限流器, 但是使用了atomic.Value包裹. 猜测本意可能是为了多协程安全, 但实际上rate.Limiter本身就支持多协程访问. 最终导致了线上限流器行为与预期不符(大约超过限流2倍左右). 在更新限流器限流的时候, 应该使用 SetLimit 或 SetLimitAt 来更新限流配置.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
func main() {
	limiter := rate.NewLimiter(rate.Limit(50000), 10)
	limiterStore.Store(limiter)
	ctx := context.Background()

	go func() {
		for {
			time.Sleep(time.Second * 1)
			fmt.Printf("count: %v\n", count)
		}
	}()

	go func() {
		for {
			time.Sleep(time.Second * 20)
			newLimiter := rate.NewLimiter(rate.Limit(50000), 10)
			limiterStore.Store(newLimiter)
		}
	}()

	for i := 0; i < 1; i++ {
		go func() {
			for {
				beginTs := time.Now()
				for j := 0; j < 10000; j++ {
					_ = limiterStore.Load().(*rate.Limiter).Wait(ctx)
				}
				duration := time.Since(beginTs)
				fmt.Printf("dur: %v\n", duration.Nanoseconds())

				count++
			}
		}()
	}

	time.Sleep(time.Hour * 24)
}

Posted on Edited on Disqus:

现象: Ctrl+Alt+M / 多显示器菜单切换无效, 选择新布局模式后屏闪并且仍然只使用一块显示器.

修复方式: 客户机打开服务: 停止 vm3dservice (VMware SVGA Helper Service) 将启动设置改为禁用即可.

VMware workstation pro 17 验证有效.

补充: 全屏模式下 Ctrl+Alt, Alt+Space, N 即可最小化当前正在运行的VM

参考

Multi-monitor still broken, does anybody have a solution / workaround ?