Skip to main content

Troubleshooting

Common Issues​

Calico/Flannel CrashLoopBackOff​

Symptom​

Network pods crash with error:

Failed to check br_netfilter: stat /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory

Solution​

Load the kernel module on the affected node:

modprobe br_netfilter

Then restart the pods:

kubectl delete pod -n kube-system <pod-name> --force

Prevention​

Add modprobe br_netfilter to your node provisioning steps.

MetalLB CRDs OutOfSync​

Symptom​

ArgoCD shows MetalLB CRDs as OutOfSync.

Cause​

MetalLB v0.14+ uses CRDs with webhook conversions. The webhook CA bundle is dynamically generated.

Solution​

The metallb.yaml template includes ignoreDifferences to ignore the caBundle field. This is already configured in the template.

ArgoCD Repo-Server OOM​

Symptom​

ArgoCD repo-server pods crash with OOM errors.

Solution​

Resources are already configured in cluster/dev/argocd-values.yaml:

repoServer:
  resources:
    requests:
      cpu: 250m
      memory: 256Mi
    limits:
      cpu: 1000m
      memory: 1Gi

If issues persist, increase memory limits.

Node Not Ready​

Common Causes​

  1. Kubelet not running:

    systemctl status kubelet
    systemctl restart kubelet

  2. Certificates expired:

    kubeadm certs check-expiration
    kubeadm certs renew all
    systemctl restart kubelet

  3. Network issues: Check that br_netfilter module is loaded and sysctl settings are applied.

Deploy Not Working​

Check 1: ArgoCD Applications​

kubectl get applications -n argocd

Check 2: Application Status​

kubectl get application <app-name> -n argocd -o wide

Check 3: Sync Status​

argocd app get <app-name>

Check 4: ArgoCD Logs​

kubectl logs -n argocd deployment/argocd-application-controller
kubectl logs -n argocd deployment/argocd-repo-server

Certificate Issues​

Check Certificate Status​

kubectl get certificates -A
kubectl describe certificate <name> -n <namespace>

Check Challenge Status​

kubectl get challenges -A

Common Causes​

  1. Cloudflare API token invalid - Verify token permissions
  2. DNS propagation delay - Wait a few minutes
  3. Rate limiting - Let's Encrypt has rate limits

PVC Pending​

Check Storage Class​

kubectl get storageclass
kubectl describe storageclass local-path

Check Provisioner​

kubectl get pods -n local-path-storage
kubectl logs -n local-path-storage -l app=local-path-provisioner

Common Causes​

  1. No nodes available - Ensure nodes are Ready
  2. Insufficient disk space - Check node disk usage
  3. Wrong storage class - Use storageClassName: local-path

Ingress Not Working​

Check Ingress Status​

kubectl get ingress -A
kubectl describe ingress <name> -n <namespace>

Check Controller​

kubectl get pods -n ingress-nginx
kubectl logs -n ingress-nginx -l app.kubernetes.io/component=controller

Check TLS Secret​

kubectl get secrets -A | grep tls
kubectl describe secret <tls-secret> -n <namespace>

DNS Not Resolving​

Check External-DNS​

kubectl get pods -n external-dns
kubectl logs -n external-dns -l app.kubernetes.io/name=external-dns

Verify DNS Records​

dig myapp.ssdk8s.xyz
nslookup myapp.ssdk8s.xyz

Check Cloudflare​

Verify records in Cloudflare dashboard.

High Resource Usage​

Check Pod Resource Usage​

kubectl top pods -A
kubectl top nodes

Check Node Resources​

kubectl describe node <node-name>

Common Solutions​

  1. Increase node resources - Add more nodes
  2. Reduce retention - Lower Prometheus/Loki retention
  3. Scale down - Reduce replica counts