173 lines
5.2 KiB
Markdown
173 lines
5.2 KiB
Markdown
Forgejo instances running in K8S
|
|
|
|
## Deployments
|
|
|
|
The [webhooks of this repository](https://code.forgejo.org/infrastructure/k8s/settings/hooks) will update existing deployments by [sending a POST request](https://code.forgejo.org/infrastructure/wakeup-on-logs). For instance https://v200.next.forgejo.org/.well-known/wakeup-on-logs/forgejo-v200 will upgrade v200.next.forgejo.org.
|
|
|
|
### Current
|
|
|
|
* https://v9.next.forgejo.org
|
|
* https://v8.next.forgejo.org
|
|
* https://v7.next.forgejo.org (referenced here for completness but not deployed using Helm, it updated hourly using an ad-hoc shell script)
|
|
|
|
### Disabled
|
|
|
|
They are online because they contain information referenced from the Forgejo issue tracker to demonstrate a problem. But they can no longer be used for testing because they are either obsolete or running a Forgejo instance that is EOL.
|
|
|
|
```
|
|
-e FORGEJO__service__DISABLE_REGISTRATION=true \
|
|
-e FORGEJO__actions__ENABLED=false \
|
|
-e FORGEJO__mirror__ENABLED=false \
|
|
```
|
|
|
|
* https://next.forgejo.org
|
|
|
|
### Offline
|
|
|
|
These instances are offline but archived and can be booted for forensice analysis if neeeded.
|
|
|
|
### LXC container
|
|
|
|
```sh
|
|
version=9
|
|
name=forgejo-v$version
|
|
```
|
|
|
|
```sh
|
|
lxc-helpers.sh lxc_container_create --config "k8s" $name
|
|
echo "lxc.start.auto = 1" | sudo tee -a /var/lib/lxc/$name/config
|
|
lxc-helpers.sh lxc_container_start $name
|
|
lxc-helpers.sh lxc_container_user_install $name $(id -u) $USER
|
|
```
|
|
|
|
### K3S installation
|
|
|
|
```sh
|
|
lxc-helpers.sh lxc_container_run $name -- sudo --user debian bash
|
|
echo 'export TERM=xterm-256color' >> .bashrc
|
|
echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> .bashrc
|
|
exit
|
|
```
|
|
|
|
```
|
|
lxc-helpers.sh lxc_container_run $name -- sudo --user debian bash
|
|
sudo apt-get install curl
|
|
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE=0644 sh -
|
|
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -
|
|
```
|
|
### Setup the domain
|
|
|
|
* In [the registrar](https://www.ovh.com/manager/#/web/domain/forgejo.org/zone) `v$version.next.forgejo.org CNAME hetzner04.forgejo.org.`
|
|
|
|
```sh
|
|
ip=$(lxc-helpers.sh lxc_container_run $name -- hostname -I | cut -f1 -d' ')
|
|
sudo tee /etc/nginx/sites-available/v$version.next.forgejo.org <<'EOF'
|
|
server {
|
|
listen 80;
|
|
listen [::]:80;
|
|
|
|
server_name v{VERSION}.next.forgejo.org;
|
|
|
|
location / {
|
|
proxy_pass http://{IP}:3000;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto https;
|
|
client_max_body_size 2G;
|
|
}
|
|
}
|
|
EOF
|
|
sudo sed -i -e "s/{IP}/$ip/" -e "s/{VERSION}/$version/" /etc/nginx/sites-available/v$version.next.forgejo.org
|
|
```
|
|
|
|
```sh
|
|
( cd /etc/nginx/sites-enabled ; ln -s /etc/nginx/sites-available/v$version.next.forgejo.org )
|
|
sudo certbot -n --agree-tos --email contact@forgejo.org -d v$version.next.forgejo.org --nginx
|
|
```
|
|
|
|
### Forward SSH
|
|
|
|
```sh
|
|
port=$(printf "2%02d0" $version)
|
|
cat > /home/debian/v$version.nftables <<EOF
|
|
add table ip v$version;
|
|
flush table ip v$version;
|
|
add chain ip v$version prerouting {
|
|
type nat hook prerouting priority 0;
|
|
policy accept;
|
|
dnat ip addr . port to tcp dport map { $port : $ip . 2222 };
|
|
};
|
|
EOF
|
|
```
|
|
|
|
- Add to `iface enp4s0 inet static` in `/etc/network/interfaces`
|
|
```
|
|
up nft -f /home/debian/v$version.nftables
|
|
```
|
|
|
|
## Define the wakeup-on-logs script
|
|
|
|
```
|
|
cd /etc/wakeup-on-logs
|
|
sudo ln -s forgejo-v8 forgejo-v9
|
|
sudo systemctl restart wakeup-on-logs-run
|
|
```
|
|
|
|
```sh
|
|
#!/bin/bash
|
|
|
|
set -x
|
|
|
|
self="${BASH_SOURCE[0]}"
|
|
name=$(basename $self)
|
|
# keep it lower than https://code.forgejo.org/infrastructure/wakeup-on-logs
|
|
# otherwise it will get killed by it
|
|
timeout=4m
|
|
|
|
function lxc_run() {
|
|
lxc-attach $name -- sudo --user debian KUBECONFIG=/etc/rancher/k3s/k3s.yaml "$@" |& tee -a /var/log/$name.log
|
|
}
|
|
|
|
image=codeberg.org/forgejo-experimental/forgejo
|
|
major=${name##*v}
|
|
digest=$(skopeo inspect --format "{{.Digest}}" docker://$image:$major-rootless)
|
|
values=https://code.forgejo.org/infrastructure/k8s/raw/branch/main/forgejo-v$major/values.yml
|
|
lxc_run helm upgrade forgejo -f $values -f /home/debian/secrets.yml oci://code.forgejo.org/forgejo-helm/forgejo --atomic --wait --timeout $timeout --install --set image.digest=$digest
|
|
```
|
|
|
|
## Define Forgejo Helm values
|
|
|
|
* https://code.forgejo.org/infrastructure/k8s/src/branch/main/forgejo-v$version/values.yml
|
|
* Add https://v$version.next.forgejo.org/.well-known/wakeup-on-logs/forgejo-v$version as a webhook https://code.forgejo.org/infrastructure/k8s/settings/hooks
|
|
|
|
Note that it requires the $version.0-test release to be published before it can successfully deploy. Otherwise it will timeout because the pod fails:
|
|
|
|
```
|
|
debian@forgejo-v9:~$ kubectl get pods
|
|
NAME READY STATUS RESTARTS AGE
|
|
forgejo-ff4fb8767-hhdnk 0/1 Init:ErrImagePull 0 3m25s
|
|
```
|
|
|
|
## Define Forgejo Helm secrets
|
|
|
|
```
|
|
cat secrets.yml
|
|
gitea:
|
|
admin:
|
|
password: "XXX"
|
|
config:
|
|
mailer:
|
|
PASSWD: "YYY"
|
|
( cd /var/lib/lxc ; cp -a forgejo-v8/rootfs/home/debian/secrets.yml forgejo-v$version/rootfs/home/debian/secrets.yml )
|
|
```
|
|
|
|
## Move the container to replicated storage
|
|
|
|
```
|
|
lxc-helpers.sh lxc_container_stop $name
|
|
sudo mv /var/lib/lxc/$name /precious/lxc/$name
|
|
sudo ln -s /precious/lxc/$name /var/lib/lxc/$name
|
|
lxc-helpers.sh lxc_container_start $name
|
|
```
|