<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>kubeadm on Chaney&#39;s MoonBook</title>
    <link>https://chaneyzorn.github.io/tags/kubeadm/</link>
    <description>Recent content in kubeadm on Chaney&#39;s MoonBook</description>
    <follow_challenge>
        <feedId>94402801171278848</feedId>
        <userId>60932538427591680</userId>
    </follow_challenge>
    <image>
      <title>Chaney&#39;s MoonBook</title>
      <url>https://chaneyzorn.github.io/cm/chaney-cover.jpg</url>
      <link>https://chaneyzorn.github.io/cm/chaney-cover.jpg</link>
    </image>
    <generator>Hugo(0.160.1) -- gohugo.io</generator>
    <language>zh</language>
    <managingEditor>chaneyzorn#gmail#com (ChaneyZorn)</managingEditor>
    <webMaster>chaneyzorn#gmail#com (ChaneyZorn)</webMaster>
    <copyright>Copyright © ChaneyZorn | CC BY-NC-ND 4.0 |</copyright>
    <lastBuildDate>Tue, 21 Jul 2026 15:27:16 +0800</lastBuildDate>
    <atom:link href="https://chaneyzorn.github.io/tags/kubeadm/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>使用 PVE 虚拟机部署 3 节点 Kubernetes 集群</title>
      <link>https://chaneyzorn.github.io/codes/deploy-k8s-on-pve/</link>
      <pubDate>Tue, 21 Jul 2026 15:27:16 +0800</pubDate><author>chaneyzorn#gmail#com (ChaneyZorn)</author>
      <guid>https://chaneyzorn.github.io/codes/deploy-k8s-on-pve/</guid>
      <description><![CDATA[<p>手头有一台 PVE（Proxmox VE）宿主机，希望在其上创建 3 台 Ubuntu 虚拟机，搭建一个 1 控制面 + 2 工作节点的 Kubernetes 集群，用于学习和测试。</p>
<p>国内环境的主要障碍是网络：k8s 官方 apt 源（pkgs.k8s.io）、镜像仓库（registry.k8s.io、quay.io、docker.io）以及 GitHub raw 均无法直接稳定访问，需要全程替换为国内镜像。本文使用的替代方案：</p>
<table>
  <thead>
      <tr>
          <th style="text-align: left">资源</th>
          <th style="text-align: left">官方地址（不可达）</th>
          <th style="text-align: left">国内替代</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td style="text-align: left">Ubuntu ISO / apt 源</td>
          <td style="text-align: left">archive.ubuntu.com</td>
          <td style="text-align: left">mirrors.tuna.tsinghua.edu.cn / mirrors.aliyun.com</td>
      </tr>
      <tr>
          <td style="text-align: left">k8s apt 源</td>
          <td style="text-align: left">pkgs.k8s.io</td>
          <td style="text-align: left">mirrors.aliyun.com/kubernetes-new</td>
      </tr>
      <tr>
          <td style="text-align: left">k8s 组件镜像</td>
          <td style="text-align: left">registry.k8s.io</td>
          <td style="text-align: left">registry.aliyuncs.com/google_containers</td>
      </tr>
      <tr>
          <td style="text-align: left">docker.io / quay.io / gcr.io 镜像</td>
          <td style="text-align: left">docker.io / quay.io / gcr.io</td>
          <td style="text-align: left">docker.m.daocloud.io / quay.m.daocloud.io / gcr.m.daocloud.io</td>
      </tr>
      <tr>
          <td style="text-align: left">GitHub raw / releases 文件</td>
          <td style="text-align: left">raw.githubusercontent.com / github.com</td>
          <td style="text-align: left">jsdelivr，或自备代理</td>
      </tr>
  </tbody>
</table>
<p>软件版本（本文撰写时验证过）：</p>
<ul>
<li>PVE 8.x</li>
<li>Ubuntu Server 24.04 LTS</li>
<li>containerd 2.2.x（使用 Ubuntu 官方仓库版本）</li>
<li>Kubernetes v1.36.x（kubeadm 部署）</li>
<li>Cilium 作为 CNI</li>
</ul>
<h2 id="1-总体规划">1. 总体规划</h2>
<h3 id="11-集群拓扑">1.1 集群拓扑</h3>
<table>
  <thead>
      <tr>
          <th style="text-align: left">主机名</th>
          <th style="text-align: left">IP</th>
          <th style="text-align: left">角色</th>
          <th style="text-align: left">最低规格</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td style="text-align: left">k8s-cp</td>
          <td style="text-align: left">10.8.8.180</td>
          <td style="text-align: left">control-plane</td>
          <td style="text-align: left">2C / 4G / 40G</td>
      </tr>
      <tr>
          <td style="text-align: left">k8s-node1</td>
          <td style="text-align: left">10.8.8.181</td>
          <td style="text-align: left">worker</td>
          <td style="text-align: left">2C / 4G / 40G</td>
      </tr>
      <tr>
          <td style="text-align: left">k8s-node2</td>
          <td style="text-align: left">10.8.8.182</td>
          <td style="text-align: left">worker</td>
          <td style="text-align: left">2C / 4G / 40G</td>
      </tr>
  </tbody>
</table>
<h3 id="12-网络规划">1.2 网络规划</h3>
<table>
  <thead>
      <tr>
          <th style="text-align: left">网络</th>
          <th style="text-align: left">网段</th>
          <th style="text-align: left">说明</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td style="text-align: left">节点网络</td>
          <td style="text-align: left">10.8.8.0/24</td>
          <td style="text-align: left">虚拟机桥接 vmbr0，与 PVE 同网段；网关 10.8.8.1，DNS 10.8.8.66 / 8.8.8.8</td>
      </tr>
      <tr>
          <td style="text-align: left">Pod 网络</td>
          <td style="text-align: left">10.244.0.0/16</td>
          <td style="text-align: left">由 Cilium 分配使用，不得与节点网络重叠</td>
      </tr>
      <tr>
          <td style="text-align: left">Service 网络</td>
          <td style="text-align: left">10.96.0.0/12</td>
          <td style="text-align: left">不得与节点/Pod 网络重叠</td>
      </tr>
  </tbody>
</table>
<h2 id="2-pve-上准备-ubuntu-虚拟机">2. PVE 上准备 Ubuntu 虚拟机</h2>
<h3 id="21-下载-ubuntu-server-镜像">2.1 下载 Ubuntu Server 镜像</h3>
<p>在 PVE 节点上下载 ISO（用清华镜像站，国内访问较快）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nb">cd</span> /var/lib/vz/template/iso/
</span></span><span class="line"><span class="cl">wget https://mirrors.tuna.tsinghua.edu.cn/ubuntu-releases/24.04/ubuntu-24.04.4-live-server-amd64.iso
</span></span></code></pre></td></tr></table>
</div>
</div><p>具体文件名以 <a href="https://mirrors.tuna.tsinghua.edu.cn/ubuntu-releases/24.04/">https://mirrors.tuna.tsinghua.edu.cn/ubuntu-releases/24.04/</a> 列表为准。</p>
<h3 id="22-创建第一台虚拟机">2.2 创建第一台虚拟机</h3>
<p>PVE Web 界面「创建 VM」，主要选项：</p>
<ul>
<li>常规：名称 <code>ubuntu-k8s-tpl</code>（之后转为模板）</li>
<li>操作系统：选择刚下载的 ISO，Guest OS 类型 Linux 6.x</li>
<li>系统：机器类型 q35，SCSI 控制器 VirtIO SCSI single，BIOS 默认 SeaBIOS</li>
<li>磁盘：VirtIO Block，40 GB</li>
<li>CPU：2 核以上，类型 host</li>
<li>内存：4096 MB 以上</li>
<li>网络：桥接 vmbr0，模型 VirtIO（半虚拟化）</li>
</ul>
<h3 id="23-安装-ubuntu-server">2.3 安装 Ubuntu Server</h3>
<p>启动虚拟机进入安装程序，注意：</p>
<ul>
<li>语言建议选 English（避免控制台中文乱码）</li>
<li>网络先保持 DHCP，静态 IP 留到克隆后逐台配置</li>
<li>存储使用整块磁盘即可（默认 LVM 方案无妨）</li>
<li>勾选 <code>Install OpenSSH server</code></li>
<li>Ubuntu Server 默认不创建 swap 分区，无需额外处理</li>
</ul>
<p>安装完成后重启，SSH 登录进行基础配置。</p>
<h3 id="24-安装后基础配置在模板机内执行">2.4 安装后基础配置（在模板机内执行）</h3>
<p>Ubuntu 24.04 的 apt 源为 DEB822 格式，替换为国内镜像：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo sed -i -e <span class="s1">&#39;s@//.*archive.ubuntu.com@//mirrors.tuna.tsinghua.edu.cn@g&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">            -e <span class="s1">&#39;s@//security.ubuntu.com@//mirrors.tuna.tsinghua.edu.cn@g&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">            /etc/apt/sources.list.d/ubuntu.sources
</span></span><span class="line"><span class="cl">sudo apt-get update <span class="o">&amp;&amp;</span> sudo apt-get full-upgrade -y
</span></span></code></pre></td></tr></table>
</div>
</div><p>安装常用工具和 qemu-guest-agent（PVE 感知虚拟机状态、安全关机所必需）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo apt-get install -y qemu-guest-agent curl vim net-tools
</span></span></code></pre></td></tr></table>
</div>
</div><p>同时在 PVE 端该虚拟机的「选项 → QEMU Guest Agent」勾选启用。</p>
<p>注意 qemu-guest-agent 不需要手动 <code>systemctl enable</code>：它的 unit 没有 <code>[Install]</code> 段（执行 enable 会提示 &ldquo;no installation config&rdquo;，属正常现象），PVE 端勾选后虚拟机内会出现 <code>/dev/virtio-ports/org.qemu.guest_agent.0</code> 设备，udev 检测到后会自动拉起服务。验证：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">systemctl status qemu-guest-agent          <span class="c1"># 应为 active (running)</span>
</span></span><span class="line"><span class="cl">ls /dev/virtio-ports/                      <span class="c1"># 应看到 org.qemu.guest_agent.0</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>服务正常后，PVE 的虚拟机摘要页面会显示该虚拟机的 IP 地址。</p>
<p>清理模板，避免克隆出的机器互相冲突：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span><span class="lnt">12
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 清空 machine-id，首次启动时会自动重新生成</span>
</span></span><span class="line"><span class="cl">sudo truncate -s <span class="m">0</span> /etc/machine-id
</span></span><span class="line"><span class="cl">sudo rm -f /var/lib/dbus/machine-id
</span></span><span class="line"><span class="cl">sudo ln -s /etc/machine-id /var/lib/dbus/machine-id
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 禁用 cloud-init，防止它按旧配置覆盖网络</span>
</span></span><span class="line"><span class="cl">sudo touch /etc/cloud/cloud-init.disabled
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 检查 netplan 配置，确保没有 macaddress 绑定字段（克隆后网卡 MAC 会变）</span>
</span></span><span class="line"><span class="cl">sudo cat /etc/netplan/*.yaml
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">sudo poweroff
</span></span></code></pre></td></tr></table>
</div>
</div><h3 id="25-转为模板并克隆">2.5 转为模板并克隆</h3>
<p>在 PVE Web 界面右键该虚拟机「转换成模板」，然后从模板 <strong>完整克隆（Full Clone）</strong> 出 3 台：</p>
<ul>
<li><code>k8s-cp</code></li>
<li><code>k8s-node1</code></li>
<li><code>k8s-node2</code></li>
</ul>
<h3 id="26-克隆后逐台修改主机名和静态-ip">2.6 克隆后逐台修改主机名和静态 IP</h3>
<p>以 k8s-cp 为例（k8s-node1 用 10.8.8.181、k8s-node2 用 10.8.8.182，其余相同）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo hostnamectl set-hostname k8s-cp
</span></span></code></pre></td></tr></table>
</div>
</div><p>编辑 netplan（文件名以实际为准，通常是 <code>/etc/netplan/00-installer-config.yaml</code>）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">network</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">ethernets</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">ens18</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">dhcp4</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">addresses</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="m">10.8.8.180</span><span class="l">/24]</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">routes</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="nt">to</span><span class="p">:</span><span class="w"> </span><span class="l">default</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">via</span><span class="p">:</span><span class="w"> </span><span class="m">10.8.8.1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">nameservers</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">addresses</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="m">10.8.8.66</span><span class="p">,</span><span class="w"> </span><span class="m">8.8.8.8</span><span class="p">]</span><span class="w">
</span></span></span></code></pre></td></tr></table>
</div>
</div><div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 修正权限，否则 netplan 会警告 &#34;Permissions ... are too open&#34;</span>
</span></span><span class="line"><span class="cl">sudo chmod <span class="m">600</span> /etc/netplan/00-installer-config.yaml
</span></span><span class="line"><span class="cl">sudo netplan apply
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="3-所有节点系统初始化">3. 所有节点：系统初始化</h2>
<p>以下操作在 <strong>3 台节点上都要执行</strong>。</p>
<p>配置 hosts 互相解析（可选但推荐）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">cat <span class="s">&lt;&lt;&#39;EOF&#39; | sudo tee -a /etc/hosts
</span></span></span><span class="line"><span class="cl"><span class="s">10.8.8.180 k8s-cp
</span></span></span><span class="line"><span class="cl"><span class="s">10.8.8.181 k8s-node1
</span></span></span><span class="line"><span class="cl"><span class="s">10.8.8.182 k8s-node2
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>时区与时间同步（节点间时间偏差会导致证书校验失败）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo timedatectl set-timezone Asia/Shanghai
</span></span><span class="line"><span class="cl">sudo sed -i <span class="s1">&#39;s/^#NTP=.*/NTP=ntp.aliyun.com/&#39;</span> /etc/systemd/timesyncd.conf
</span></span><span class="line"><span class="cl">sudo systemctl restart systemd-timesyncd
</span></span></code></pre></td></tr></table>
</div>
</div><p>关闭 swap（kubelet 要求）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo swapoff -a
</span></span><span class="line"><span class="cl">sudo sed -i <span class="s1">&#39;/\bswap\b/s/^/#/&#39;</span> /etc/fstab
</span></span></code></pre></td></tr></table>
</div>
</div><p>加载内核模块并开启 iptables 桥接转发：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span><span class="lnt">12
</span><span class="lnt">13
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">cat <span class="s">&lt;&lt;&#39;EOF&#39; | sudo tee /etc/modules-load.d/k8s.conf
</span></span></span><span class="line"><span class="cl"><span class="s">overlay
</span></span></span><span class="line"><span class="cl"><span class="s">br_netfilter
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">sudo modprobe overlay
</span></span><span class="line"><span class="cl">sudo modprobe br_netfilter
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat <span class="s">&lt;&lt;&#39;EOF&#39; | sudo tee /etc/sysctl.d/k8s.conf
</span></span></span><span class="line"><span class="cl"><span class="s">net.bridge.bridge-nf-call-ip6tables = 1
</span></span></span><span class="line"><span class="cl"><span class="s">net.bridge.bridge-nf-call-iptables = 1
</span></span></span><span class="line"><span class="cl"><span class="s">net.ipv4.ip_forward = 1
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span><span class="line"><span class="cl">sudo sysctl --system
</span></span></code></pre></td></tr></table>
</div>
</div><p>防火墙：Ubuntu 默认 ufw 未启用、PVE 虚拟机防火墙默认关闭，则无需处理。如果启用过防火墙，控制面需放行 6443、2379-2380、10250、10251、10252、10257、10259，工作节点需放行 10250、30000-32767。</p>
<h2 id="4-所有节点安装-containerd">4. 所有节点：安装 containerd</h2>
<p>直接使用 Ubuntu 官方仓库的 containerd（24.04 仓库当前为 2.2.x），省去额外配置第三方源：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo apt-get install -y containerd
</span></span></code></pre></td></tr></table>
</div>
</div><p>生成完整默认配置并修改：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo mkdir -p /etc/containerd
</span></span><span class="line"><span class="cl">containerd config default <span class="p">|</span> sudo tee /etc/containerd/config.toml
</span></span></code></pre></td></tr></table>
</div>
</div><p>需要改四处：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 1. 使用 systemd cgroup driver（与 kubelet 保持一致）</span>
</span></span><span class="line"><span class="cl">sudo sed -i <span class="s1">&#39;s/SystemdCgroup = false/SystemdCgroup = true/&#39;</span> /etc/containerd/config.toml
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 2. pause 沙箱镜像改为阿里云镜像（tag 保持不变）</span>
</span></span><span class="line"><span class="cl">sudo sed -i <span class="s1">&#39;s#registry.k8s.io/pause#registry.aliyuncs.com/google_containers/pause#&#39;</span> /etc/containerd/config.toml
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 3. 置空 cri registry 段的 config_path：Ubuntu 的 containerd 2.2 默认配置里它</span>
</span></span><span class="line"><span class="cl"><span class="c1">#    非空（如 &#39;/etc/containerd/certs.d:/etc/docker/certs.d&#39;），与下一步的内联</span>
</span></span><span class="line"><span class="cl"><span class="c1">#    mirrors 互斥，不置空会导致 CRI 插件加载失败（值以实际文件为准）</span>
</span></span><span class="line"><span class="cl">sudo sed -i <span class="s2">&#34;s#config_path = &#39;/etc/containerd/certs.d:/etc/docker/certs.d&#39;#config_path = &#39;&#39;#&#34;</span> /etc/containerd/config.toml
</span></span><span class="line"><span class="cl">grep -n config_path /etc/containerd/config.toml   <span class="c1"># 确认 cri registry 段的值已为空</span>
</span></span></code></pre></td></tr></table>
</div>
</div><div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span><span class="lnt">12
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 4. 为 docker.io / quay.io / gcr.io 等配置镜像加速，追加到配置文件末尾</span>
</span></span><span class="line"><span class="cl">cat <span class="s">&lt;&lt;&#39;EOF&#39; | sudo tee -a /etc/containerd/config.toml
</span></span></span><span class="line"><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="cl"><span class="s">[plugins.&#39;io.containerd.cri.v1.images&#39;.registry.mirrors.&#39;docker.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">  endpoint = [&#39;https://docker.m.daocloud.io&#39;, &#39;https://registry-1.docker.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">[plugins.&#39;io.containerd.cri.v1.images&#39;.registry.mirrors.&#39;quay.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">  endpoint = [&#39;https://quay.m.daocloud.io&#39;, &#39;https://quay.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">[plugins.&#39;io.containerd.cri.v1.images&#39;.registry.mirrors.&#39;gcr.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">  endpoint = [&#39;https://gcr.m.daocloud.io&#39;, &#39;https://gcr.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">[plugins.&#39;io.containerd.cri.v1.images&#39;.registry.mirrors.&#39;registry.k8s.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">  endpoint = [&#39;https://k8s.m.daocloud.io&#39;, &#39;https://registry.k8s.io&#39;]
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span></code></pre></td></tr></table>
</div>
</div><blockquote>
<p>注意：以上写法对应 containerd 2.x 的 v3 配置格式（CRI 插件路径为 <code>io.containerd.cri.v1.images</code>，配置文件开头 <code>version = 3</code>），已在 Ubuntu 24.04 仓库的 containerd 2.2.1 上实际验证。如果用的是 containerd 1.7，插件路径是 <code>io.containerd.grpc.v1.cri</code>，且没有 config_path 互斥问题。如果之前按不匹配的版本改过配置，直接重新执行 <code>containerd config default | sudo tee /etc/containerd/config.toml</code> 从头再来即可。</p>
</blockquote>
<p>配置 crictl（排查容器问题时用）。crictl 由 <code>cri-tools</code> 包提供：Ubuntu 仓库里它在 universe 组件中，第 5 节的 kubernetes-new 源也会提供，第 5 节会显式安装。所以这里只写配置文件即可，crictl 验证留到第 5 节之后：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">cat <span class="s">&lt;&lt;&#39;EOF&#39; | sudo tee /etc/crictl.yaml
</span></span></span><span class="line"><span class="cl"><span class="s">runtime-endpoint: unix:///var/run/containerd/containerd.sock
</span></span></span><span class="line"><span class="cl"><span class="s">image-endpoint: unix:///var/run/containerd/containerd.sock
</span></span></span><span class="line"><span class="cl"><span class="s">timeout: 10
</span></span></span><span class="line"><span class="cl"><span class="s">EOF</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>重启并验证（containerd 没有 config check 子命令，用 <code>config dump</code> 输出解析后的实际配置来检查，语法错误会直接报出来）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span><span class="lnt">6
</span><span class="lnt">7
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo systemctl restart containerd
</span></span><span class="line"><span class="cl">sudo systemctl <span class="nb">enable</span> containerd
</span></span><span class="line"><span class="cl">systemctl is-active containerd                    <span class="c1"># 应为 active</span>
</span></span><span class="line"><span class="cl">containerd config dump <span class="p">|</span> grep -i systemdCgroup    <span class="c1"># 应为 true</span>
</span></span><span class="line"><span class="cl">containerd config dump <span class="p">|</span> grep daocloud            <span class="c1"># 应看到 mirrors 已生效</span>
</span></span><span class="line"><span class="cl">containerd config dump <span class="p">|</span> grep disabled            <span class="c1"># 不应看到 disabled_plugins 包含 &#34;cri&#34;</span>
</span></span><span class="line"><span class="cl">sudo journalctl -u containerd -b --no-pager <span class="p">|</span> grep <span class="s1">&#39;failed to load plugin&#39;</span>   <span class="c1"># 应无输出</span>
</span></span></code></pre></td></tr></table>
</div>
</div><blockquote>
<p>如果最后一条 grep 有输出（常见为 <code>mirrors cannot be set when config_path is provided</code> 或 cri 被 disabled_plugins 禁用），说明 CRI 插件没有加载，kubeadm 会报 <code>unknown service runtime.v1.RuntimeService</code>，按输出提示回头检查配置。</p>
</blockquote>
<p>待第 5 节装好 cri-tools 后，还可以实测拉取：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">crictl pull docker.io/library/nginx:1.27-alpine     <span class="c1"># 实测加速拉取</span>
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="5-所有节点安装-kubeadm--kubelet--kubectl">5. 所有节点：安装 kubeadm / kubelet / kubectl</h2>
<p>使用阿里云的 kubernetes-new 源（社区 pkgs.k8s.io 的镜像。注意旧源 <code>mirrors.aliyun.com/kubernetes/apt</code> 停留在 1.28，不要再用）。</p>
<p>如需其他小版本，把下面 URL 中的 <code>v1.36</code> 换掉即可（可用版本列表见 <a href="https://mirrors.aliyun.com/kubernetes-new/core/stable/">https://mirrors.aliyun.com/kubernetes-new/core/stable/</a>）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo apt-get install -y apt-transport-https ca-certificates gpg
</span></span><span class="line"><span class="cl">sudo install -m <span class="m">0755</span> -d /etc/apt/keyrings
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">curl -fsSL https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.36/deb/Release.key <span class="p">|</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.36/deb/ /&#34;</span> <span class="p">|</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  sudo tee /etc/apt/sources.list.d/kubernetes.list
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">sudo apt-get update
</span></span></code></pre></td></tr></table>
</div>
</div><p>查看可用版本并安装（3 台节点版本必须一致）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">apt-cache madison kubeadm <span class="p">|</span> head -5     <span class="c1"># 记下版本号，如 1.36.2-1.1</span>
</span></span><span class="line"><span class="cl"><span class="c1"># cri-tools（crictl）在新版打包中不一定是 kubeadm 的强依赖，显式安装</span>
</span></span><span class="line"><span class="cl">sudo apt-get install -y kubelet kubeadm kubectl cri-tools
</span></span><span class="line"><span class="cl">sudo apt-mark hold kubelet kubeadm kubectl
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="6-控制面kubeadm-init">6. 控制面：kubeadm init</h2>
<p>在 <strong>k8s-cp</strong> 上编写初始化配置 <code>kubeadm-init.yaml</code>：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt"> 1
</span><span class="lnt"> 2
</span><span class="lnt"> 3
</span><span class="lnt"> 4
</span><span class="lnt"> 5
</span><span class="lnt"> 6
</span><span class="lnt"> 7
</span><span class="lnt"> 8
</span><span class="lnt"> 9
</span><span class="lnt">10
</span><span class="lnt">11
</span><span class="lnt">12
</span><span class="lnt">13
</span><span class="lnt">14
</span><span class="lnt">15
</span><span class="lnt">16
</span><span class="lnt">17
</span><span class="lnt">18
</span><span class="lnt">19
</span><span class="lnt">20
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubeadm.k8s.io/v1beta4</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">InitConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">localAPIEndpoint</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">advertiseAddress</span><span class="p">:</span><span class="w"> </span><span class="m">10.8.8.180</span><span class="w">           </span><span class="c"># 控制面节点 IP</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">bindPort</span><span class="p">:</span><span class="w"> </span><span class="m">6443</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">nodeRegistration</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">criSocket</span><span class="p">:</span><span class="w"> </span><span class="l">unix:///var/run/containerd/containerd.sock</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">imagePullPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">IfNotPresent</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nn">---</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubeadm.k8s.io/v1beta4</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">ClusterConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kubernetesVersion</span><span class="p">:</span><span class="w"> </span><span class="l">v1.36.2             </span><span class="w"> </span><span class="c"># 与 apt-cache madison 查到的版本一致（去掉 -1.1 后缀）</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">imageRepository</span><span class="p">:</span><span class="w"> </span><span class="l">registry.aliyuncs.com/google_containers</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">networking</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">podSubnet</span><span class="p">:</span><span class="w"> </span><span class="m">10.244.0.0</span><span class="l">/16             </span><span class="w"> </span><span class="c"># 与 Cilium 的 clusterPoolIPv4PodCIDRList 一致</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">serviceSubnet</span><span class="p">:</span><span class="w"> </span><span class="m">10.96.0.0</span><span class="l">/12</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nn">---</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubelet.config.k8s.io/v1beta1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">KubeletConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">cgroupDriver</span><span class="p">:</span><span class="w"> </span><span class="l">systemd</span><span class="w">
</span></span></span></code></pre></td></tr></table>
</div>
</div><p>说明：</p>
<ul>
<li>必须显式写 <code>kubernetesVersion</code>，否则 kubeadm 会去 dl.k8s.io 查询最新版本，国内可能超时。</li>
<li>必须显式配置 <code>imageRepository</code>，否则默认从 registry.k8s.io 拉取组件镜像，国内无法直接拉取。</li>
<li>v1beta3 已在 kubeadm 1.34 中移除，1.34 及以上必须使用 v1beta4；如果安装的是 1.33 及更早版本，把 apiVersion 改回 <code>kubeadm.k8s.io/v1beta3</code> 即可，本例用到的字段在两个版本中位置相同。也可执行 <code>kubeadm config print init-defaults</code> 查看当前 kubeadm 的默认配置格式。</li>
</ul>
<p>先预拉镜像验证镜像源可用，再执行初始化：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo kubeadm config images pull --config kubeadm-init.yaml
</span></span><span class="line"><span class="cl">sudo kubeadm init --config kubeadm-init.yaml
</span></span></code></pre></td></tr></table>
</div>
</div><p>成功后按输出提示配置 kubectl：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">mkdir -p <span class="nv">$HOME</span>/.kube
</span></span><span class="line"><span class="cl">sudo cp -i /etc/kubernetes/admin.conf <span class="nv">$HOME</span>/.kube/config
</span></span><span class="line"><span class="cl">sudo chown <span class="k">$(</span>id -u<span class="k">)</span>:<span class="k">$(</span>id -g<span class="k">)</span> <span class="nv">$HOME</span>/.kube/config
</span></span></code></pre></td></tr></table>
</div>
</div><p><strong>保存 init 输出末尾的 <code>kubeadm join ...</code> 命令</strong>，第 8 步要用。如果丢失可重新生成：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubeadm token create --print-join-command
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="7-控制面安装-cni-插件cilium">7. 控制面：安装 CNI 插件（Cilium）</h2>
<p>不装 CNI 的话节点会一直停留在 NotReady。目前主流 CNI 是 <strong>Cilium、Calico、Flannel</strong> 三家。Cilium 基于 eBPF，2023 年从 CNCF 毕业，被 GKE、Azure 等托管 k8s 服务采用，支持 L3-L7 NetworkPolicy、Hubble 流量可观测，还可完全替代 kube-proxy；Calico 以 BGP 路由和网络策略见长，企业集群常见；Flannel 最轻量，但不支持 NetworkPolicy。本文选用 Cilium。</p>
<h3 id="71-安装-cilium">7.1 安装 Cilium</h3>
<p>Cilium 对内核的要求是 4.19+（5.10+ 可解锁全部特性），Ubuntu 24.04 自带的 6.8 内核完全满足。</p>
<p>下载 cilium-cli（GitHub releases 国内不可直达，请自行通过代理下载，或找能出网的机器下载后 scp 过来）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">wget https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
</span></span><span class="line"><span class="cl">sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin
</span></span></code></pre></td></tr></table>
</div>
</div><p>安装：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">cilium install --set ipam.operator.clusterPoolIPv4PodCIDRList<span class="o">=</span>10.244.0.0/16
</span></span><span class="line"><span class="cl">cilium status --wait
</span></span></code></pre></td></tr></table>
</div>
</div><p>说明：</p>
<ul>
<li><code>clusterPoolIPv4PodCIDRList</code> 必须与 <code>kubeadm-init.yaml</code> 中的 <code>podSubnet</code> 一致。</li>
<li>默认保留 kube-proxy，对 kubeadm 集群开箱即用。</li>
<li>Cilium 镜像托管在 quay.io，第 4 步配置的 quay.m.daocloud.io 镜像加速自动生效，无需改任何镜像地址。</li>
</ul>
<p>可选：验证集群网络连通性（会额外拉取 quay.io 上的测试镜像，同样走镜像加速）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">cilium connectivity <span class="nb">test</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>注意其中的外网连通用例（访问 1.1.1.1）在国内无法连通，失败属预期，不影响对集群内网络功能的判断。个别用例（如 L7 负载均衡的 l7-lb）的镜像若拉取失败（ImagePullBackOff），可用 <code>kubectl -n cilium-test-1 describe pod &lt;pod&gt;</code> 在 Events 里看具体镜像和原因，一般是该镜像在镜像加速服务上不可用，可忽略该用例。测试资源都创建在 <code>cilium-test-1</code> namespace 中，验证完删除该 namespace 即清理干净：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl delete namespace cilium-test-1
</span></span></code></pre></td></tr></table>
</div>
</div><p>进阶：Cilium 可以完全替代 kube-proxy（<code>kubeProxyReplacement=true</code>，需在 <code>kubeadm init</code> 时跳过 kube-proxy 插件或事后移除），并可启用 Hubble Relay/UI 做流量可观测，按需参考官方文档开启。</p>
<h2 id="8-worker-节点加入集群">8. worker 节点：加入集群</h2>
<p>在 <strong>k8s-node1、k8s-node2</strong> 上分别执行第 6 步保存的 join 命令，形如：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo kubeadm join 10.8.8.180:6443 <span class="se">\
</span></span></span><span class="line"><span class="cl">  --token abcdef.0123456789abcdef <span class="se">\
</span></span></span><span class="line"><span class="cl">  --discovery-token-ca-cert-hash sha256:xxxxxxxx
</span></span></code></pre></td></tr></table>
</div>
</div><p>注意 init 输出的 join 命令原文不带 <code>sudo</code>，直接复制执行会报 <code>ERROR IsPrivilegedUser</code>，记得补上。</p>
<p>worker 的组件镜像同样来自 registry.aliyuncs.com/google_containers（join 时 kubeadm 只拉 pause 和 kube-proxy， kubeadm 会沿用镜像仓库设置；如拉取失败，可先 <code>sudo kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers</code>）。</p>
<p>worker 加入后，Cilium agent 会以 DaemonSet 形式自动在新节点上启动，无需额外操作。</p>
<h2 id="9-验证集群">9. 验证集群</h2>
<p>在控制面上：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl get nodes -o wide        <span class="c1"># 3 个节点均为 Ready</span>
</span></span><span class="line"><span class="cl">kubectl get pods -A              <span class="c1"># 系统组件全部 Running（加 -w 可持续观察状态变化）</span>
</span></span><span class="line"><span class="cl">cilium status                    <span class="c1"># Cilium 各节点 agent 正常</span>
</span></span></code></pre></td></tr></table>
</div>
</div><p>部署一个测试应用（nginx 镜像经 docker.m.daocloud.io 加速拉取）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span><span class="lnt">5
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl create deployment web --image<span class="o">=</span>nginx:1.27-alpine --replicas<span class="o">=</span><span class="m">3</span>
</span></span><span class="line"><span class="cl">kubectl expose deployment web --port<span class="o">=</span><span class="m">80</span> --type<span class="o">=</span>ClusterIP
</span></span><span class="line"><span class="cl">kubectl get pods -o wide
</span></span><span class="line"><span class="cl">kubectl get svc web
</span></span><span class="line"><span class="cl">curl &lt;web 的 ClusterIP&gt;
</span></span></code></pre></td></tr></table>
</div>
</div><p>如需在控制面节点上调度普通 Pod（测试集群常见需求）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl taint nodes k8s-cp node-role.kubernetes.io/control-plane:NoSchedule-
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="10-安装-helm可选">10. 安装 helm（可选）</h2>
<p>helm 是 k8s 的包管理工具，后续安装 Hubble UI、ingress-controller 等 chart 会用到。它是纯客户端工具，装在操作集群的机器上即可（如 k8s-cp），直接使用 <code>~/.kube/config</code>。</p>
<p>方式一：官方安装脚本（自动安装最新版）：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># raw.githubusercontent.com 不可直达，脚本本身走 jsdelivr 下载</span>
</span></span><span class="line"><span class="cl">curl -fsSL https://cdn.jsdelivr.net/gh/helm/helm@main/scripts/get-helm-3 -o get-helm-3.sh
</span></span><span class="line"><span class="cl">bash get-helm-3.sh
</span></span></code></pre></td></tr></table>
</div>
</div><p>脚本会从 get.helm.sh（Azure CDN，国内一般可直连）下载 helm 二进制。</p>
<p>方式二：手动下载指定版本：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span><span class="lnt">4
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 版本号到 https://github.com/helm/helm/releases 查看（页面打不开就走代理）</span>
</span></span><span class="line"><span class="cl">curl -fLO https://get.helm.sh/helm-v3.18.4-linux-amd64.tar.gz
</span></span><span class="line"><span class="cl">tar xzf helm-v3.18.4-linux-amd64.tar.gz
</span></span><span class="line"><span class="cl">sudo install -m <span class="m">0755</span> linux-amd64/helm /usr/local/bin/helm
</span></span></code></pre></td></tr></table>
</div>
</div><p>验证：</p>
<div class="highlight"><div class="chroma">
<table class="lntable"><tr><td class="lntd">
<pre tabindex="0" class="chroma"><code><span class="lnt">1
</span><span class="lnt">2
</span><span class="lnt">3
</span></code></pre></td>
<td class="lntd">
<pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">helm version
</span></span><span class="line"><span class="cl"><span class="c1"># 可选：验证 repo 可用（helm.cilium.io 托管在 GitHub Pages，国内一般可达）</span>
</span></span><span class="line"><span class="cl">helm repo add cilium https://helm.cilium.io <span class="o">&amp;&amp;</span> helm repo update
</span></span></code></pre></td></tr></table>
</div>
</div><h2 id="11-常见问题">11. 常见问题</h2>
<ul>
<li><strong>kubeadm init 卡在拉镜像</strong>：检查是否忘了 <code>imageRepository</code>；用 <code>kubeadm config images pull --config kubeadm-init.yaml</code> 单独验证拉取。</li>
<li><strong>kubeadm 报 <code>unknown service runtime.v1.RuntimeService</code></strong>：containerd 的 CRI 插件没有加载。用 <code>sudo journalctl -u containerd -b --no-pager | grep 'failed to load plugin'</code> 看具体原因——最常见两种：一是 cri registry 段的 <code>config_path</code> 非空与内联 <code>mirrors</code> 互斥（Ubuntu containerd 2.2 的默认配置即如此，按第 4 节置空即可）；二是配置里 <code>disabled_plugins</code> 包含了 &ldquo;cri&rdquo;。修复并 <code>systemctl restart containerd</code> 后，<code>sudo crictl version</code> 能列出 RuntimeVersion 即恢复。</li>
<li><strong>init 报 cgroup driver 不匹配</strong>：确认 containerd 的 <code>SystemdCgroup = true</code> 且 init 配置里 <code>cgroupDriver: systemd</code>，两边一致后 <code>sudo systemctl restart containerd</code> 重试。</li>
<li><strong>节点一直 NotReady</strong>：多半是 CNI 插件未就绪。<code>kubectl describe node &lt;节点&gt;</code> 看 Conditions；Cilium 用 <code>cilium status</code> 和 <code>kubectl -n kube-system logs -l k8s-app=cilium</code> 排查。镜像拉取失败时检查 quay 加速配置。</li>
<li><strong>kubelet 启动失败 / 反复重启</strong>：<code>journalctl -u kubelet -f</code> 看日志，常见原因是 swap 未完全关闭或配置文件错误。</li>
<li><strong>克隆的虚拟机注册冲突</strong>：machine-id 重复会导致 kubelet 无法正常注册，重新生成： <code>sudo rm /etc/machine-id &amp;&amp; sudo systemd-machine-id-setup</code>，再重启 kubelet。</li>
<li><strong>join token 过期（默认 24 小时）</strong>：在控制面执行 <code>kubeadm token create --print-join-command</code> 重新生成。</li>
<li><strong>GitHub 直连失败</strong>：raw 文件和 releases 下载在国内不可直达时，请自行使用代理，或找一台能出网的机器下载好文件再 scp 过来。</li>
</ul>
<h2 id="12-附录镜像代理的配置位置与流程">12. 附录：镜像代理的配置位置与流程</h2>
<p>本文涉及&quot;拉镜像&quot;的代理一共分布在 3 个位置，分两类机制，另有 1 类容易混淆的非镜像代理。</p>
<h3 id="配置位置">配置位置</h3>
<p><strong>1. containerd registry mirrors（<code>/etc/containerd/config.toml</code>，第 4 节）—— 透明代理，覆盖名字固定的第三方镜像</strong></p>
<table>
  <thead>
      <tr>
          <th style="text-align: left">原镜像仓库</th>
          <th style="text-align: left">代理 endpoint</th>
          <th style="text-align: left">兜底</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td style="text-align: left">docker.io</td>
          <td style="text-align: left">docker.m.daocloud.io</td>
          <td style="text-align: left">registry-1.docker.io</td>
      </tr>
      <tr>
          <td style="text-align: left">quay.io</td>
          <td style="text-align: left">quay.m.daocloud.io</td>
          <td style="text-align: left">quay.io</td>
      </tr>
      <tr>
          <td style="text-align: left">gcr.io</td>
          <td style="text-align: left">gcr.m.daocloud.io</td>
          <td style="text-align: left">gcr.io</td>
      </tr>
      <tr>
          <td style="text-align: left">registry.k8s.io</td>
          <td style="text-align: left">k8s.m.daocloud.io</td>
          <td style="text-align: left">registry.k8s.io</td>
      </tr>
  </tbody>
</table>
<p>生效对象：cilium 等组件清单里写死的 quay.io 镜像、手动部署的 nginx （docker.io）、connectivity test 的测试镜像（gcr.io）等。好处是无需修改镜像清单本身。前提是把 <code>config_path</code> 置空（containerd 2.2 默认启用的 certs.d 目录方式与内联 mirrors 互斥，否则 CRI 插件加载失败，见第 4 节）。</p>
<p><strong>2. containerd pause 沙箱镜像（同文件 <code>pinned_images</code> 的 <code>sandbox</code>）—— 改名换源</strong></p>
<p>把 <code>registry.k8s.io/pause</code> 改为 <code>registry.aliyuncs.com/google_containers/pause</code>。这不是代理，是直接把镜像名指到国内可达的地址。</p>
<p><strong>3. kubeadm 的 <code>imageRepository</code>（<code>kubeadm-init.yaml</code>，第 6 节）——改名换源</strong></p>
<p><code>registry.aliyuncs.com/google_containers</code> 作用于 kubeadm 负责拉取的全部控制面组件：kube-apiserver、kube-controller-manager、kube-scheduler、kube-proxy、 etcd、coredns、pause。kubeadm 拼镜像名时用它替换默认的 <code>registry.k8s.io</code> 前缀。</p>
<p><strong>（区分）非镜像代理</strong>：apt 包走清华/阿里云源、cilium-cli 等 GitHub 文件下载需自备代理——这些是文件/包下载，与 containerd 镜像代理是两条独立的线。</p>
<h3 id="代理流程">代理流程</h3>
<p>以 kubelet 在某节点启动一个 cilium Pod（镜像 <code>quay.io/cilium/cilium:vX.Y.Z</code>）为例：</p>
<ol>
<li>kubelet 通过 CRI 调 containerd 的 <code>PullImage</code></li>
<li>containerd 的 CRI 插件解析镜像引用，提取 registry 主机名 <code>quay.io</code></li>
<li>查 registry 配置：命中 mirrors，endpoint 列表第一个是 <code>https://quay.m.daocloud.io</code></li>
<li>向该 endpoint 发标准 OCI registry 请求（先 Head/GET manifest，再 GET blob），镜像路径保持原样（<code>/cilium/cilium/...</code>）；daocloud 作为缓存代理回源 quay.io 取数返回</li>
<li>第一个 endpoint 失败则按列表顺序回退到下一个</li>
<li>成功则 unpack 到 snapshotter，镜像入库，Pod 创建</li>
</ol>
<p>而 <code>kubeadm init</code> 拉控制面镜像走的是另一条路径：镜像名已被拼成 <code>registry.aliyuncs.com/google_containers/kube-apiserver:vX.Y.Z</code>，主机名是 <code>registry.aliyuncs.com</code>，不命中任何 mirrors 规则，直连阿里云（国内可达，无需代理）。</p>
<p>两者的分工：<strong>mirrors 透明代理管第三方镜像（yaml 里名字固定的）， imageRepository / pause 改名换源管 k8s 官方组件</strong>——前者在 resolver 层替换 endpoint，后者在生成镜像名时直接换掉前缀。</p>
<h2 id="参考链接">参考链接</h2>
<ul>
<li><a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/">Installing kubeadm - Kubernetes</a></li>
<li><a href="https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/">kubeadm init - Kubernetes</a></li>
<li><a href="https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/">kubeadm 配置 API（v1beta4）- Kubernetes</a></li>
<li><a href="https://kubernetes.io/docs/setup/production-environment/container-runtimes/">Container runtimes - Kubernetes</a></li>
<li><a href="https://mirrors.aliyun.com/kubernetes-new/core/stable/">阿里云镜像站 kubernetes-new 目录</a></li>
<li><a href="https://github.com/DaoCloud/public-image-mirror">DaoCloud 公开镜像加速说明</a></li>
<li><a href="https://docs.cilium.io/">Cilium 官方文档</a></li>
<li><a href="https://github.com/cilium/cilium">Cilium - GitHub</a></li>
<li><a href="https://mirrors.tuna.tsinghua.edu.cn/ubuntu-releases/">清华镜像站 Ubuntu releases</a></li>
<li><a href="https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines">Qemu/KVM Virtual Machines - Proxmox VE Wiki</a></li>
<li><a href="https://helm.sh/docs/">Helm 官方文档</a></li>
</ul>
]]></description>
    </item>
  </channel>
</rss>
