Skip to main content

Prometheus Server Pod with high load is frecuently evicted (how to fix)

Prometheus Server Pod with high load is frecuently evicted (how to fix)

Recently I deployed Prometheus, Grafana, Alertmanager and PushGateway using the official helm chart. In this case the k8s cluster is production, so that tls is required. By the same way I have many clusters with Helm 2.x in production and some modifications was required to use differents private keys and certificates. At last I rewrite my .bashrc and others files to include something like this (only relevant sections are showed):

```
# ~/.bashrc
....
# https://github.com/ahmetb/kubectl-aliases
[ -f ~/.kubectl_aliases ] && source ~/.kubectl_aliases
....
# kubernetes specific
export KUBE_EDITOR="nano"
source <(kubectl completion bash) # setup autocomplete in bash into the current shell, bash-completion package should be installed first.
....
```

Other file:
```
# ~/.bash_aliases

# note for cluster AKS
# Error: incompatible versions client[v2.16.1] server[v2.13.1]
# https://github.com/helm/helm/r…
Post a Comment

Starting

Iniciando...


 Bueno, en algún momento debía de empezar....

 Después de darle vuelta un buen rato a la idea de escribir, compartir y sembrar la semilla de todo aquello que, a Dios gracias, he tenido la bendición de aprender, siento que es el momento de plasmar esas ideas...

 Ahora estoy aprendiendo a usar este sistema de blog, aunque si se me pone reacio, mejor me hago uno...

bueno, continúo con el respaldo de mi laptop, que ya el fan-cooler le suena que parece una rolinera de las turbinas del Guri a punto de explotar... y debo apresurarme en pasar la data al disco externo de respaldo que me hice con un disco de estado sólido (SDD) y una cajita USB3 --simplemente vuela !!!

 Vale, ahí nos vemos cuando ya termine de respaldar (Y)

Comments

Post a Comment

Popular posts from this blog

How fix full disk for /data in Prometheus Server deployed with Helm chart

How fix full disk for /data in Prometheus Server deployed with Helm chart

The prometheus-server pod has two containers: prometheus-server-configmap-reload and prometheus-server.

Currently the prometheus-server has one disk of 20GiB and in was full in sixty days. We need to resize or change this pvc for at least 60GiB.

In order to do this, we need:

the prometheus-server need to be stoppedbackup current volume data in another pvcdelete current prometheus-server pvcrecreate the previous prometheus pvcrestore previous backup on this new prometheus-server pvcstart the previous stopped deployment of prometheus-sever
There are the details: (the "k" is one alias for "kubectl")

1. the prometheus-server need to be stopped ---
we need to get information for current prometheus-server deployment

```
$ k -n monitoring get deployment
NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
grafana                                 1/1     1            1         …
Post a Comment
Read more

Prometheus Server Pod with high load is frecuently evicted (how to fix)

Prometheus Server Pod with high load is frecuently evicted (how to fix)

Recently I deployed Prometheus, Grafana, Alertmanager and PushGateway using the official helm chart. In this case the k8s cluster is production, so that tls is required. By the same way I have many clusters with Helm 2.x in production and some modifications was required to use differents private keys and certificates. At last I rewrite my .bashrc and others files to include something like this (only relevant sections are showed):

```
# ~/.bashrc
....
# https://github.com/ahmetb/kubectl-aliases
[ -f ~/.kubectl_aliases ] && source ~/.kubectl_aliases
....
# kubernetes specific
export KUBE_EDITOR="nano"
source <(kubectl completion bash) # setup autocomplete in bash into the current shell, bash-completion package should be installed first.
....
```

Other file:
```
# ~/.bash_aliases

# note for cluster AKS
# Error: incompatible versions client[v2.16.1] server[v2.13.1]
# https://github.com/helm/helm/r…
Post a Comment
Read more