[root@node1 ~]# kubectl get node NAME STATUS ROLES AGE VERSION master Ready control-plane 82d v1.24.1 node1 Ready control-plane 82d v1.24.1 node2 Ready control-plane 42m v1.24.1 wulaoer Ready <none> 82d v1.24.1 [root@node1 ~]# kubeadm certs check-expiration #查看证书到期时间 [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Dec 14, 2024 08:00 UTC 282d ca no apiserver Dec 14, 2024 08:00 UTC 282d ca no apiserver-etcd-client Dec 14, 2024 08:00 UTC 282d etcd-ca no apiserver-kubelet-client Dec 14, 2024 08:00 UTC 282d ca no controller-manager.conf Dec 14, 2024 08:00 UTC 282d ca no etcd-healthcheck-client Dec 14, 2024 08:00 UTC 282d etcd-ca no etcd-peer Dec 14, 2024 08:00 UTC 282d etcd-ca no etcd-server Dec 14, 2024 08:00 UTC 282d etcd-ca no front-proxy-client Dec 14, 2024 08:00 UTC 282d front-proxy-ca no scheduler.conf Dec 14, 2024 08:00 UTC 282d ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Dec 12, 2033 06:21 UTC 9y no etcd-ca Dec 12, 2033 06:21 UTC 9y no front-proxy-ca Dec 12, 2033 06:21 UTC 9y no #也可以使用for i in $(ls /etc/kubernetes/pki/*.crt /etc/kubernetes/pki/etcd/*.crt);do echo 不过太复杂,建议使用上面的方法。
我这里使用的是集群方式,因为是我自己实验的环境,所以每个节点的证书时间都不一样,如果生产环境建议每个节点都时间要一直,总不能差别太多吧,那样更新的时候也不好更新。因为证书时间还有很久,我这里修改一下系统时间,看一下会报什么错。
[root@node1 ~]# date -s "20250101" Thu Jan 1 00:00:00 CST 2025 [root@node1 ~]# date Thu Jan 1 00:00:07 CST 2025exit [root@node1 ~]# kubectl get node Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2026-01-01T00:00:10+08:00 is after 2025-03-07T03:15:39Z
更新一年证书
这里使用命令久已经出现提示,证书时间过期了,那么我们先看一下更新一年,在更新证书之前先备份一下,方便后期会滚。
[root@node1 ~]# cp -a /etc/kubernetes{,.bak} [root@node1 ~]# cp -a /var/lib/kubelet{,.bak} [root@node1 ~]# cp -a /var/lib/etcd /var/lib/etcd.bak [root@node1 ~]# kubeadm certs renew all [renew] Reading configuration from the cluster... [renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [renew] Error reading configuration from the Cluster. Falling back to default configuration certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed certificate for serving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificate for the API server to connect to kubelet renewed certificate embedded in the kubeconfig file for the controller manager to use renewed certificate for liveness probes to healthcheck etcd renewed certificate for etcd nodes to communicate with each other renewed certificate for serving etcd renewed certificate for the front proxy client renewed certificate embedded in the kubeconfig file for the scheduler manager to use renewed Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates. [root@node1 ~]# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [check-expiration] Error reading configuration from the Cluster. Falling back to default configuration CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Dec 31, 2026 16:02 UTC 364d ca no apiserver Dec 31, 2026 16:02 UTC 364d ca no apiserver-etcd-client Dec 31, 2026 16:02 UTC 364d etcd-ca no apiserver-kubelet-client Dec 31, 2026 16:02 UTC 364d ca no controller-manager.conf Dec 31, 2026 16:02 UTC 364d ca no etcd-healthcheck-client Dec 31, 2026 16:02 UTC 364d etcd-ca no etcd-peer Dec 31, 2026 16:02 UTC 364d etcd-ca no etcd-server Dec 31, 2026 16:02 UTC 364d etcd-ca no front-proxy-client Dec 31, 2026 16:02 UTC 364d front-proxy-ca no scheduler.conf Dec 31, 2026 16:02 UTC 364d ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Dec 12, 2033 06:21 UTC 7y no etcd-ca Dec 12, 2033 06:21 UTC 7y no front-proxy-ca Dec 12, 2033 06:21 UTC 7y no
更新后,重启一下etcd,api,controller-manager,scheduler四个组件或者重启节点也是一样的,如果你的是高可用的集群环境。这里已经更新了1年。
更新100年证书
下面看一下更新100年的方式。因为上个节点的时间问题,这里换个节点。另外建议在生产环境中不建议使用,主要是因为版本升级后,证书的日志会默认改成1年有效期,看一下手动更新方式。先准备一下环境:
[root@master ~]# yum -y install git [root@master ~]# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:24:38Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"} [root@master ~]# git clone -b v1.24.1 --depth=1 https://github.com/kubernetes/kubernetes.git Cloning into 'kubernetes'... remote: Enumerating objects: 24819, done. remote: Counting objects: 100% (24819/24819), done. remote: Compressing objects: 100% (18564/18564), done. remote: Total 24819 (delta 7289), reused 13817 (delta 4556), pack-reused 0 Receiving objects: 100% (24819/24819), 36.14 MiB | 8.89 MiB/s, done. Resolving deltas: 100% (7289/7289), done. Note: switching to '3ddd0f45aa91e2f30c70734b175631bec5b5825a'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false Updating files: 100% (23394/23394), done. [root@master ~]# sed -ri 's#time.Hour \* 24 \* 365#time.Hour \* 24 \* 365 \* 100#' ~/kubernetes/cmd/kubeadm/app/constants/constants.go [root@master ~]# vim ~/kubernetes/staging/src/k8s.io/client-go/util/cert/cert.go #这里修改的是ca,etcdca,front-proxy-ca三个组件的证书 ......................................... const duration365d = time.Hour * 24 * 365 // Config contains the basic fields required for creating a certificate type Config struct { CommonName string Organization []string AltNames AltNames Usages []x509.ExtKeyUsage } // AltNames contains the domain names and IP addresses that will be added // to the API Server's x509 certificate SubAltNames field. The values will // be passed directly to the x509.Certificate object. type AltNames struct { DNSNames []string IPs []net.IP } // NewSelfSignedCACert creates a CA certificate func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, error) { now := time.Now() tmpl := x509.Certificate{ SerialNumber: new(big.Int).SetInt64(0), Subject: pkix.Name{ CommonName: cfg.CommonName, Organization: cfg.Organization, }, DNSNames: []string{cfg.CommonName}, NotBefore: now.UTC(), //NotAfter: now.Add(duration365d * 10).UTC(), #ca证书默认是10年,修改成100年 NotAfter: now.Add(duration365d * 100).UTC(), KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign, BasicConstraintsValid: true, IsCA: true, } ......................................... [root@master ~]# wget https://studygolang.com/dl/golang/go1.18.2.linux-amd64.tar.gz [root@master ~]# yum -y install tar [root@master ~]# tar xf go1.18.2.linux-amd64.tar.gz -C /usr/local/ [root@master ~]# echo "PATH=/usr/local/go/bin:$PATH" >> /etc/profile [root@master ~]# source /etc/profile [root@master ~]# go version go version go1.18.2 linux/amd64 [root@master ~]# yum -y install gcc [root@master ~]# cd ~/kubernetes [root@master kubernetes]# yum -y install make && rsync [root@master kubernetes-1.24.1]# make WHAT=cmd/kubeadm +++ [0307 14:36:26] Building go targets for linux/amd64 k8s.io/kubernetes/hack/make-rules/helpers/go2make (non-static) +++ [0307 14:36:32] Generating deepcopy code for 57 targets +++ [0307 14:36:39] Building go targets for linux/amd64 k8s.io/kubernetes/vendor/k8s.io/code-generator/cmd/defaulter-gen (non-static) +++ [0307 14:36:41] Generating defaulter code for 92 targets +++ [0307 14:37:20] Building go targets for linux/amd64 k8s.io/kubernetes/vendor/k8s.io/code-generator/cmd/conversion-gen (non-static) +++ [0307 14:37:24] Generating conversion code for 129 targets +++ [0307 14:44:21] Building go targets for linux/amd64 k8s.io/kubernetes/vendor/k8s.io/kube-openapi/cmd/openapi-gen (non-static) +++ [0307 14:44:42] Generating openapi code for KUBE +++ [0307 14:45:09] Generating openapi code for AGGREGATOR +++ [0307 14:45:11] Generating openapi code for APIEXTENSIONS +++ [0307 14:45:13] Generating openapi code for CODEGEN +++ [0307 14:45:15] Generating openapi code for SAMPLEAPISERVER +++ [0307 14:45:16] Building go targets for linux/amd64 k8s.io/kubernetes/cmd/kubeadm (static) [root@master kubernetes-1.24.1]# cp /usr/bin/kubeadm{,.bak} [root@master kubernetes-1.24.1]# cp _output/bin/kubeadm /usr/bin/ cp: overwrite '/usr/bin/kubeadm'? y [root@master kubernetes-1.24.1]# cp -r /etc/kubernetes/pki{,.bak}
这里注意在编译时make WHAT=cmd/kubeadm
,总是会提示失败,或者其他的问题,首先看一下CPU是否够,如果失败可以把_output
目录删除后重新编译。
[root@master kubernetes-1.24.1]# kubeadm certs check-expiration #检查证书时间 [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Mar 07, 2025 03:15 UTC 364d ca no apiserver Mar 07, 2025 03:15 UTC 364d ca no apiserver-etcd-client Mar 07, 2025 03:15 UTC 364d etcd-ca no apiserver-kubelet-client Mar 07, 2025 03:15 UTC 364d ca no controller-manager.conf Mar 07, 2025 03:15 UTC 364d ca no etcd-healthcheck-client Mar 07, 2025 03:15 UTC 364d etcd-ca no etcd-peer Mar 07, 2025 03:15 UTC 364d etcd-ca no etcd-server Mar 07, 2025 03:15 UTC 364d etcd-ca no front-proxy-client Mar 07, 2025 03:15 UTC 364d front-proxy-ca no scheduler.conf Mar 07, 2025 03:15 UTC 364d ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Dec 12, 2033 06:21 UTC 9y no etcd-ca Dec 12, 2033 06:21 UTC 9y no front-proxy-ca Dec 12, 2033 06:21 UTC 9y no [root@master kubernetes-1.24.1]# kubeadm certs renew all #更新证书 [renew] Reading configuration from the cluster... [renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed certificate for serving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificate for the API server to connect to kubelet renewed certificate embedded in the kubeconfig file for the controller manager to use renewed certificate for liveness probes to healthcheck etcd renewed certificate for etcd nodes to communicate with each other renewed certificate for serving etcd renewed certificate for the front proxy client renewed certificate embedded in the kubeconfig file for the scheduler manager to use renewed Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates. [root@master kubernetes-1.24.1]# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Feb 12, 2124 06:57 UTC 99y ca no apiserver Feb 12, 2124 06:57 UTC 99y ca no apiserver-etcd-client Feb 12, 2124 06:57 UTC 99y etcd-ca no apiserver-kubelet-client Feb 12, 2124 06:57 UTC 99y ca no controller-manager.conf Feb 12, 2124 06:57 UTC 99y ca no etcd-healthcheck-client Feb 12, 2124 06:57 UTC 99y etcd-ca no etcd-peer Feb 12, 2124 06:57 UTC 99y etcd-ca no etcd-server Feb 12, 2124 06:57 UTC 99y etcd-ca no front-proxy-client Feb 12, 2124 06:57 UTC 99y front-proxy-ca no scheduler.conf Feb 12, 2124 06:57 UTC 99y ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Dec 12, 2033 06:21 UTC 9y no etcd-ca Dec 12, 2033 06:21 UTC 9y no front-proxy-ca Dec 12, 2033 06:21 UTC 9y no
这里编译后更新证书时,发现只ca
,etcd-ca
,front-proxy-ca
三个组件的证书没有更新,并不是设置,我们前面已经修改了,主要是因为这三个证书的更新必须是kubeadm init
才可以,下面看一下更新ca
,etcd-ca
,front-proxy-ca
三个证书时间。
[root@master kubernetes-1.24.1]# kubeadm init phase certs all W0307 15:20:27.144914 5548 version.go:103] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://cdn.dl.k8s.io/release/stable-1.txt": context deadline exceeded (Client.Timeout exceeded while awaiting headers) W0307 15:20:27.144980 5548 version.go:104] falling back to the local client version: v1.24.1 [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 10.211.55.31] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [10.211.55.31 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [10.211.55.31 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [root@master kubernetes-1.24.1]# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Feb 12, 2124 06:57 UTC 99y ca no apiserver Feb 12, 2124 07:20 UTC 99y ca no apiserver-etcd-client Feb 12, 2124 07:20 UTC 99y etcd-ca no apiserver-kubelet-client Feb 12, 2124 07:20 UTC 99y ca no controller-manager.conf Feb 12, 2124 06:57 UTC 99y ca no etcd-healthcheck-client Feb 12, 2124 07:20 UTC 99y etcd-ca no etcd-peer Feb 12, 2124 07:20 UTC 99y etcd-ca no etcd-server Feb 12, 2124 07:20 UTC 99y etcd-ca no front-proxy-client Feb 12, 2124 07:20 UTC 99y front-proxy-ca no scheduler.conf Feb 12, 2124 06:57 UTC 99y ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Feb 12, 2124 07:20 UTC 99y no etcd-ca Feb 12, 2124 07:20 UTC 99y no front-proxy-ca Feb 12, 2124 07:20 UTC 99y no
证书更新后,其他节点可以把编译好的kubeadm
拷贝到其他节点上,如果直接使用kubeadm init phase certs all
命令更新你会发现会有admin.conf
,controller-manager.conf
,scheduler.conf
三个文件的证书只更新了1年,其他的都正常。
[root@node2 kubernetes]# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Mar 31, 2026 16:00 UTC 2y ca no apiserver Feb 12, 2124 07:18 UTC 99y ca no apiserver-etcd-client Feb 12, 2124 07:18 UTC 99y etcd-ca no apiserver-kubelet-client Feb 12, 2124 07:18 UTC 99y ca no controller-manager.conf Mar 31, 2026 16:00 UTC 2y ca no etcd-healthcheck-client Feb 12, 2124 07:18 UTC 99y etcd-ca no etcd-peer Feb 12, 2124 07:18 UTC 99y etcd-ca no etcd-server Feb 12, 2124 07:18 UTC 99y etcd-ca no front-proxy-client Feb 12, 2124 07:18 UTC 99y front-proxy-ca no scheduler.conf Mar 31, 2026 16:00 UTC 2y ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Feb 12, 2124 07:18 UTC 99y no etcd-ca Feb 12, 2124 07:18 UTC 99y no front-proxy-ca Feb 12, 2124 07:18 UTC 99y no
如果先更新上面的证书,然后更新下面的ca证书,会发现所有的证书都可以更新100年了,所以把kubeadm
拷贝到其他节点后,需要先更新api证书,后更新ca证书,才会都更新。
[root@node2 kubernetes]# kubeadm certs renew all [renew] Reading configuration from the cluster... [renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed certificate for serving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificate for the API server to connect to kubelet renewed certificate embedded in the kubeconfig file for the controller manager to use renewed certificate for liveness probes to healthcheck etcd renewed certificate for etcd nodes to communicate with each other renewed certificate for serving etcd renewed certificate for the front proxy client renewed certificate embedded in the kubeconfig file for the scheduler manager to use renewed Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates. [root@node2 kubernetes]# mv pki pki.bal [root@node2 kubernetes]# kubeadm init phase certs all I0307 15:23:47.037766 67544 version.go:255] remote version is much newer: v1.29.2; falling back to: stable-1.24 [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local node2] and IPs [10.96.0.1 10.211.55.33] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost node2] and IPs [10.211.55.33 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost node2] and IPs [10.211.55.33 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [root@node2 kubernetes]# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Feb 12, 2124 07:22 UTC 99y ca no apiserver Feb 12, 2124 07:23 UTC 99y ca no apiserver-etcd-client Feb 12, 2124 07:23 UTC 99y etcd-ca no apiserver-kubelet-client Feb 12, 2124 07:23 UTC 99y ca no controller-manager.conf Feb 12, 2124 07:22 UTC 99y ca no etcd-healthcheck-client Feb 12, 2124 07:23 UTC 99y etcd-ca no etcd-peer Feb 12, 2124 07:23 UTC 99y etcd-ca no etcd-server Feb 12, 2124 07:23 UTC 99y etcd-ca no front-proxy-client Feb 12, 2124 07:23 UTC 99y front-proxy-ca no scheduler.conf Feb 12, 2124 07:22 UTC 99y ca no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Feb 12, 2124 07:23 UTC 99y no etcd-ca Feb 12, 2124 07:23 UTC 99y no front-proxy-ca Feb 12, 2124 07:23 UTC 99y no
证书更新成功后,把新的证书更新到配置文件中,方便后期继续使用。
[root@node2 kubernetes]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config cp: overwrite '/root/.kube/config'? y
至此,kubernetes集群高可用的证书更新算是结束了,后期有其他的在继续更新,没有了,看些其他的吧。
您可以选择一种方式赞助本站
支付宝扫一扫赞助
微信钱包扫描赞助
赏