Linux

LXC Container - Requirements & Basics

Requirements

Install requirements for debian

apt-get install lxc libvirt0 libpam-cgfs bridge-utils uidmap
Basics

Start Container

lxc-start -n %NAME%

Stop Container

lxc-stop -n %NAME%

Attach Container (get access to cli inside container)

lxc-attach -n %NAME%

List info/state of containers

lxc-ls --fancy

Create container (template downloaded for unprivileged container)

lxc-create -n %NAME% -t download -- -r bookworm

Destroy (delete) container - container have to be stopped before

lxc-destroy -n %NAME%
Creating unprivileged containers

When creating unprivileged containers as root with shared UID and GID the files /etc/subuid and /etc/subgid need some entries. Check content of both files before appending stuff to it! Furthermore the default lcx configuration file /etc/lxc/default.conf has to be extened.

echo "root:100000:65536" >>/etc/subuid
echo "root:100000:65536" >>/etc/subgid
echo "lxc.idmap = u 0 100000 65536" >>/etc/lxc/default.conf
echo "lxc.idmap = g 0 100000 65536" >>/etc/lxc/default.conf
Autostart for containers

Add the following line to configuration file (/var/lib/lxc/%%NAME%%/config) of each container, which should be start automatically.

lxc.start.auto = 1
Basic default configuration
xc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx

lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1

lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

Jekyll - Install & Basics

#Install requirements with apt-get
apt-get install ruby ruby-dev build-essential

Do not install gems as root user

Login as user and execute following commands:

echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc
echo 'export GEM_HOME=$HOME/gems' >> ~/.bashrc
echo 'export PATH=$HOME/gems/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

Install Jekyll and dependencies:

gem install jekyll bundler

#update jekyll or any other gem
gem update jekyll

#update Rubygems
gem update --system

Working with Jekyll

#start a new website, named 'the_blog'
jekyll new the_blog

#build new created website
jekyll build
# => The current folder will be generated into ./_site

jekyll build --destination <destination>
# => The current folder will be generated into <destination>

jekyll build --source <source> --destination <destination>
# => The <source> folder will be generated into <destination>

jekyll build --watch
# => The current folder will be generated into ./_site,
#    watched for changes, and regenerated automatically.

 

nftables - Firewall

Basics

Using nftables as the successor to iptables makes life much more convenient. So I switched to nftables at my projects.

For debian (bookworm) it's necessary to enable the nftables service:

systemctl enable nftables.service

Default configuration file for nftables is located at: /etc/nftables.conf. I don't want to make big modifications at preinstalled files, so I just add an include statement for customized rules:

include "/etc/custom/nftables.rules" 

Here are some basic commands for nftables

List loaded ruleset

nft list ruleset

List sets 
The following command lists the set WHITELIST of table CUSTOM

nft list set inet CUSTOM WHITELIST
Basic ruleset for a server providing http/https services
#!/usr/sbin/nft -f

table inet CUSTOM {
    # empty set WHITELIST - will be filled dynamically by scripts
	set WHITELIST {
		type ipv4_addr
	}
	chain INPUT {
		type filter hook input priority 0; policy drop;

		# allow established/related connections
		ct state {established, related} accept

		# early drop of invalid connections
		ct state invalid drop

		# allow from loopback
		iifname lo accept

		# allow icmp (ipv4) - only from IP set WHITELIST
		ip protocol icmp ip saddr @WHITELIST accept

        # allow only ssh connections from IP set WHITELIST
		tcp dport {22} ip saddr @WHITELIST accept

		# allow web services
		tcp dport {80, 443} accept

		# allow internal container interface
		iifname "lxcbr0" ip daddr 10.0.3.1 accept

		# everything else drop
		drop
	}

	chain OUTPUT {
		type filter hook output priority 0; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority 0; policy drop;
		oifname "lxcbr0" accept
        iifname "lxcbr0" accept
	}
}

Updating nftables set

The following bash script updates the set WHITELIST with a dynamic IP address. The script should be executed periodically by a cron  job.

The variables 

have to be modified.

#!/bin/bash
# Update ipset to let my dynamic IP in
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# nftables table and set
ipset=WHITELIST
table=CUSTOM

# dynamical IP for resolving via DNS
host=myurl.example.com

#IP for whitelisting
ips="127.0.0.1 127.0.0.2"

# hostname for logging
me=$(basename "$0")

ip=$(dig +short $host | tail -1)
if [ -z "${ip}" ]; then
    logger -t "${me}" "IP for ${host} not found"
    exit 1
fi

ips="${ips} ${ip}"

ret=`nft get element inet ${table} ${ipset} { ${ip} } > /dev/null 2>&1; echo $?`

if [[ ${ret} == "1" ]] then 
	nft flush set inet ${table} ${ipset}
	logger -t "${me}"  "Adding IP ${ip} to set ${ipset}."
	for i in ${ips}
	do
		logger -t "${me}"  "Adding IP ${i} to set ${ipset}."
		nft add element inet ${table} ${ipset} { ${i} }
	done
fi

exit 0

Debian - Tasks for new installs

Install of common packages
apt install net-tools mc git vim bind9-dnsutils ripgrep fzf
VIM Configuration
cd /etc/vim ; wget https://wuideweidewelt.de/vim/vim_config_updater.sh ; sh vim_config_updater.sh
Setup fzf

Editing file /etc/bash.bashrc

# append to bash.bashrc
echo "source /usr/share/bash-completion/completions/fzf" >> /etc/bash.bashrc
echo "source /usr/share/doc/fzf/examples/key-bindings.bash" >> /etc/bash.bashrc

mdadm - Software Raid

Summary of commands dealing with software raid at linux machines

 

Create an array

mdadm --create /dev/md/md_name --level=<RAID-Level> --raid-devices=<amount of physical partitions at array> /dev/sdX1 /dev/sdY1

Check raid array

mdadm --detail /dev/md/<LABEL>

# or

cat /proc/mdstat

Remove raid array

umount /dev/md/md_name
mdadm --stop /dev/md/md_name
mdadm --zero-superblock /dev/sdX1
mdadm --zero-superblock /dev/sdY1

Restart sync of array

mdadm --readwrite /dev/md/md_name

Update raid config

/usr/share/mdadm/mkconfig > /etc/mdadm/mdamd.conf

When device is needed for boot

update-initramfs -u
update-grub2
grub-install /dev/sdX

 

LUKS - Encryption

Summary of basic commands

Create encrypted partition

cryptsetup -c aes-xts-plain64 -s 512 -h sha512 luksFormat /dev/sdX2

Open encrypted partition as crypted_sdX2

cryptsetup luksOpen /dev/sdX2 crypted_sdX2

Add encryption key

cryptsetup luksAddKey /dev/sdX2

Remove encryption key

Attention !!!
When you don’t have antoher key, you aren’t able to decrypt the partition afterwards

cryptsetup luksRemoveKey /dev/sdX2

Caddy webserver - valid certificates for local services

  1. Setup a debian machine or use an existing machine

  2. Install xcaddy  (add xcaddy apt repository)

    apt install -y debian-keyring debian-archive-keyring apt-transport-https
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-xcaddy-archive-keyring.gpg
    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-xcaddy.list
    sudo apt update
    sudo apt install xcaddy
  3. Install prebuild go language from go website

    mkdir golang
    cd golang
    
    #remove existing go version
    rm -rf /usr/local/go
    
    # check for latest release and download
    wget https://go.dev/dl/go1.22.2.linux-amd64.tar.gz
    tar -xvf go1.22.2.linux-amd64.tar.gz -C /usr/local
    
    # set environment variables for go / e.g. edit bash.bashrc
    export GOROOT=/usr/local/go
    export GOPATH=$HOME/go
    export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
    

  4. Build your own caddy binary with needed plugins - in this case include dns-netcup module

    xcaddy build --with github.com/caddy-dns/netcup
    
    # move binary to /usr/bin
    # rename it to caddy_custom to mark it as custom build
    
    mv caddy /usr/bin/caddy_custom
    
    # create default config Directory fpr Caddyfile
    mkdir /etc/caddy
    touch /etc/caddy/Caddyfile



  5. Create systemd service file and enable it

    # create caddy group
    groupadd --system caddy
    
    #create caddy user
    useradd --system \
        --gid caddy \
        --create-home \
        --home-dir /var/lib/caddy \
        --shell /usr/sbin/nologin \
        --comment "Caddy web server" \
        caddy
    
    #create default working folder
    mkdir /var/lib/caddy
    chown -R caddy:caddy /var/lib/caddy
    
    # create empty service file
    touch /lib/systemd/system/caddy.service

    This example service file includes env vars for the netcup dns module

    # caddy.service
    #
    # For using Caddy with a config file.
    #
    # Make sure the ExecStart and ExecReload commands are correct
    # for your installation.
    #
    # See https://caddyserver.com/docs/install for instructions.
    #
    # WARNING: This service does not use the --resume flag, so if you
    # use the API to make changes, they will be overwritten by the
    # Caddyfile next time the service is restarted. If you intend to
    # use Caddy's API to configure it, add the --resume flag to the
    # `caddy run` command or use the caddy-api.service file instead.
    
    [Unit]
    Description=Caddy
    Documentation=https://caddyserver.com/docs/
    After=network.target network-online.target
    Requires=network-online.target
    
    [Service]
    Environment="NETCUP_CUSTOMER_NUMBER=XXXXX"
    Environment="NETCUP_API_KEY=XXXXX"
    Environment="NETCUP_API_PASSWORD=XXXXX"
    Type=notify
    User=caddy
    Group=caddy
    ExecStart=/usr/bin/caddy_custom run --environ --config /etc/caddy/Caddyfile
    ExecReload=/usr/bin/caddy_custom reload --config /etc/caddy/Caddyfile --force
    TimeoutStopSec=5s
    LimitNOFILE=1048576
    LimitNPROC=512
    PrivateTmp=true
    ProtectSystem=full
    AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
    
    [Install]
    WantedBy=multi-user.target
    

    Enable caddy as service

    systemctl daemon-reload
    systemctl enable caddy.service
    systemctl start caddy.service

  6. Sample Caddyfile

    # The Caddyfile is an easy way to configure your Caddy web server.
    #
    # Unless the file starts with a global options block, the first
    # uncommented line is always the address of your site.
    #
    # To use your own domain name (with automatic HTTPS), first make
    # sure your domain's A/AAAA DNS records are properly pointed to
    # this machine's public IP, then replace ":80" below with your
    # domain name.
    
    #global options
    {
            # E-Mail for ACME
            email info@mydomain.de
    }
    
    # caddy directive for import
    (insecure) {
            transport http {
                    tls_insecure_skip_verify
            }
    }
    
    # reverse proxy for websockets
    wss01.mydomain.de:8884 {
            @websockets {
                    header Connection *Upgrade*
                    header Upgrade websocket
            }
            tls {
                    dns netcup {
                            customer_number {env.NETCUP_CUSTOMER_NUMBER}
                            api_key {env.NETCUP_API_KEY}
                            api_password {env.NETCUP_API_PASSWORD}
                    }
                    propagation_timeout 900s
                    propagation_delay 600s
                    resolvers 1.1.1.1
            }
            reverse_proxy @websockets 192.168.2.91:8884 {
                    import insecure
            }
    }
    
    # wildcard domain/certificate for internal services
    *.mydomain.de {
            tls {
                    dns netcup {
                            customer_number {env.NETCUP_CUSTOMER_NUMBER}
                            api_key {env.NETCUP_API_KEY}
                            api_password {env.NETCUP_API_PASSWORD}
                    }
                    # netcup dns is slow
                    propagation_timeout 900s
                    propagation_delay 600s
                    resolvers 1.1.1.1
            }
    
            @ilo01 host ilo01.mydomain.de
            handle @ilo01 {
                    reverse_proxy 192.168.2.13 {
                            import insecure
                    }
            }
    
            @pve01 host pve01.mydomain.de
            handle @pve01 {
                    reverse_proxy 192.168.2.10:8006 {
                            import insecure
                    }
            }
    
            @pbs01 host pbs01.mydomain.de
            handle @pbs01 {
                    reverse_proxy 192.168.2.15:8007 {
                            import insecure
                    }
            }
    
            @home host home.mydomain.de
            handle @home {
                    reverse_proxy https://192.168.2.94 {
                            import insecure
                    }
            }
    
            handle {
                    abort
            }
    
    }

Fail2Ban

1. Unban IP

List banned IP via iptables

iptables -L -n

Get configured jails

fail2ban-client status

Unban IP

fail2ban-client set JAIL-NAME unbanip XXX.XXX.XXX.XXX
2. Configuration for caddy reverse proxy

Enable access logging at caddy

(custom_log) {
	log {
		format json {
			time_format iso8601
		}
		output file /var/log/caddy/{args[0]}.access.log {
			roll_size 10mb
			roll_keep 20
			roll_keep_for 720h
		}
	}
}

mydomain.com {
	import custom_log mydomain.com
    reverse_proxy XXX.XXX.XXX.XXX
}

Create caddy filter file (regex) for fail2ban - /etc/fail2ban/filter.d/caddy-status.conf

[Definition]
failregex = ^.*"remote_ip":"<HOST>",.*?"status":(?:401|403|500),.*$
ignoreregex =
datepattern = "ts":"%%Y-%%m-%%dT%%H:%%M:%%S.

Enable jail in fail2ban config (/etc/fail2ban/jail.local)

[caddy-status]
backend		= auto
enabled     = true
port        = http,https
filter      = caddy-status
logpath     = /var/log/caddy/*.access.log
maxretry    = 10

Docker @debian

Uninstall old (pre-installed) versions
for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
Set up Docker's apt repository
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
Install the Docker packages
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

 

 

 

 

LVM2

Create/add physical disk to LVM
pvcreate /dev/sdb
Create volume group
vgcreate vg00 /dev/sdb1 /dev/sdc1
Create volume
# create by given size in GB
lvcreate -n data -L1G vg00
# create by given size of vg in %
lvcreate -n data -l100%VG vg00
Angabe in Prozent des freien Speichers in der VG:
# create by given free size of vg in %
lvcreate -n data -l100%FREE vg00

 

 

 

iSCSI

Search for iSCSI targets

iscsiadm --mode discovery --portal XXX.XXX.XXX.XXX --type sendtargets

Login

iscsiadm -m node -T iqn.2024-11.local.... -p XXX.XXX.XXX.XXX:3260 --login

Check iSCSI sessions

iscsiadm -m session

Logout of iSCSI session

iscsiadm -m node -T iqn.2024-05.local.... -p XXX.XXX.XXX.XXX:3260 -u

Remove iSCSI target

iscsiadm -m node -o delete -T iqn.2024-05.local......

 

Proxmox BackupServer - pbs

Create datastore at sshfs mount
  1. mount path via sshfs e.g. /mnt/backup
  2. use bindfs to ignore missing chown and chgrp for sshfs mount 

    bindfs --chown-ignore --chgrp-ignore /mnt/backup /mnt/backup-bindfs

  3. Create datastore via pbs UI with path /mnt/backup-bindfs
  4. Modify /etc/proxmox-backup/datastore.cfg and replace /mnt/backup-bindfs with /mnt/backup