Can't get container running with quadlet/podman

[XenGi]

Ohai,
I/m currently switching from a podman based setup to quadlet. quadlet drops pretty much all privileges by default and runs rootless containers. So exactly how it should be imho.

My setup works when I start the container with podman as root. Now with quadlet it seems that it needs more rights then it gets.

This is my quadlet container under /etc/containers/systemd/transmission.container:

[Unit]
Description=transmission container
Wants=network.target
After=network-online.target
RequiresMountsFor=/data

[Container]
Image=docker.io/haugene/transmission-openvpn:latest
Volume=/data:/data
PublishPort=9091:9091
User=900
Group=900
Environment=PUID=900
Environment=PGID=900
Environment=TZ=Europe/Berlin
Environment=OPENVPN_PROVIDER=MULLVAD
Environment=OPENVPN_CONFIG=somecountry_all
Environment=OPENVPN_USERNAME=my-mullvad-id
Environment=OPENVPN_PASSWORD=m
Environment=HEALTH_CHECK_HOST=am.i.mullvad.net
Environment=LOG_TO_STDOUT=True
Environment=LOCAL_NETWORK=192.168.178.0/24
Environment=TRANSMISSION_ENCRYPTION=2
Environment=TRANSMISSION_PEER_PORT=12345
Environment=TRANSMISSION_IDLE_SEEDING_LIMIT=30
Environment=TRANSMISSION_IDLE_SEEDING_LIMIT_ENABLED=True
Environment=TRANSMISSION_RATIO_LIMIT=2
Environment=TRANSMISSION_RATIO_LIMIT_ENABLED=True
# don't try to create tun and avoid these errors:
# mknod: /dev/net/tun: Permission denied
# chmod: cannot access '/dev/net/tun': Permission denied
Environment=CREATE_TUN_DEVICE=False
Timezone=Europe/Berlin
AddCapability=NET_ADMIN
PodmanArgs=--sysctl=net.ipv6.conf.all.disable_ipv6=0 --device=/dev/net/tun

[Service]
Restart=always

This creates a systemd unit with the following podman call:

podman run \
  --name=systemd-%N \
  --cidfile=%t/%N.cid \
  --replace \
  --rm \
  -d \
  --log-driver journald \
  --pull=never \
  --runtime /usr/bin/crun \
  --cgroups=split \
  --tz=Europe/Berlin \
  --init \
  --sdnotify=conmon \
  --security-opt=no-new-privileges \
  --cap-drop=all \
  --cap-add=net_admin \
  --mount type=tmpfs,tmpfs-size=512M,destination=/tmp \
  --user 900:900 \
  --uidmap 900:900:1 \
  --uidmap 0:0:1 \
  --uidmap 1:1879048192:899 \
  --uidmap 901:1879049091:64637 \
  --gidmap 900:900:1 \
  --gidmap 0:0:1 \
  --gidmap 1:1879048192:899 \
  --gidmap 901:1879049091:64637 \
  -v /data:/data \
  -p=9091:9091 \
  --env CREATE_TUN_DEVICE=False \
  --env HEALTH_CHECK_HOST=am.i.mullvad.net \
  --env LOCAL_NETWORK=192.168.178.0/24 \
  --env LOG_TO_STDOUT=True \
  --env OPENVPN_CONFIG=somecountry_all \
  --env OPENVPN_PASSWORD=m \
  --env OPENVPN_PROVIDER=MULLVAD \
  --env OPENVPN_USERNAME=my-mullvad-id \
  --env PGID=900 \
  --env PUID=900 \
  --env TRANSMISSION_ENCRYPTION=2 \
  --env TRANSMISSION_IDLE_SEEDING_LIMIT=30 \
  --env TRANSMISSION_IDLE_SEEDING_LIMIT_ENABLED=True \
  --env TRANSMISSION_PEER_PORT=12345 \
  --env TRANSMISSION_RATIO_LIMIT=2 \
  --env TRANSMISSION_RATIO_LIMIT_ENABLED=True \
  --env TZ=Europe/Berlin \
  --sysctl=net.ipv6.conf.all.disable_ipv6=0 \
  --device=/dev/net/tun \
  docker.io/haugene/transmission-openvpn:latest

I already fixed these errors by supplying CREATE_TUN_DEVICE=False:

Nov 19 00:59:53 downloader conmon[3873526]: mknod: /dev/net/tun: Permission denied
Nov 19 00:59:53 downloader conmon[3873526]: chmod: cannot access '/dev/net/tun': Permission denied

Now the container tries to create the /etc/openvpn/mullvad folder and fails with these errors:

Nov 19 01:06:46 downloader conmon[3874327]: mkdir: cannot create directory '/etc/openvpn/mullvad': Permission denied
Nov 19 01:06:46 downloader conmon[3874327]: Using OpenVPN provider: MULLVAD
Nov 19 01:06:46 downloader conmon[3874327]: Running with VPN_CONFIG_SOURCE auto
Nov 19 01:06:46 downloader conmon[3874327]: No bundled config script found for MULLVAD. Defaulting to external config
Nov 19 01:06:46 downloader conmon[3874327]: Downloading configs from https://github.com/haugene/vpn-configs-contrib/archive/main.zip into /tmp/tmp.aLFxZJzdVy
Nov 19 01:06:47 downloader conmon[3874327]: Extracting configs to /tmp/tmp.pDPakAnyc1
Nov 19 01:06:48 downloader conmon[3874327]: Found configs for MULLVAD in /tmp/tmp.pDPakAnyc1/vpn-configs-contrib-main/openvpn/mullvad, will replace current content in /etc/openvpn/mullvad
Nov 19 01:06:48 downloader conmon[3874327]: mv: 
Nov 19 01:06:48 downloader conmon[3874327]: cannot create directory '/etc/openvpn/mullvad'
Nov 19 01:06:48 downloader conmon[3874327]: : Permission denied
Nov 19 01:06:48 downloader conmon[3874327]: 
Nov 19 01:06:48 downloader conmon[3874327]: Cleanup: deleting /tmp/tmp.aLFxZJzdVy and /tmp/tmp.pDPakAnyc1
Nov 19 01:06:48 downloader conmon[3874327]: Supplied config se_all.ovpn could not be found.
Nov 19 01:06:48 downloader conmon[3874327]: Your options for this provider are:
Nov 19 01:06:48 downloader conmon[3874327]: ls: cannot access '/etc/openvpn/mullvad': No such file or directory
Nov 19 01:06:48 downloader conmon[3874327]: NB: Remember to not specify .ovpn as part of the config name.

I haven't looked into the Dockerfile and entrypoint scripts yet what is actually going on here. I guess the script assumes to be root at this point and therefore having the rights to create that folder. But I start my container as uid 900:900. Does it actually need to start as root and I have to trust that it reduces it's rights to uid 900 by itself? Would be nice to run this container rootless.

---

If I run the container as root, removing the User=900 and Group=900 lines from the quadlet file, the generated podman command looks like this:

podman run \
  --name=systemd-%N \
  --cidfile=%t/%N.cid \
  --replace \
  --rm \
  -d \
  --log-driver journald \
  --pull=never \
  --runtime /usr/bin/crun \
  --cgroups=split \
  --tz=Europe/Berlin \
  --init \
  --sdnotify=conmon \
  --security-opt=no-new-privileges \
  --cap-drop=all \
  --cap-add=net_admin \
  --mount type=tmpfs,tmpfs-size=512M,destination=/tmp \
  --uidmap 0:0:1 \
  --uidmap 1:1879048192:65536 \
  --gidmap 0:0:1 \
  --gidmap 1:1879048192:65536 \
  -v /data:/data \
  -p=9091:9091 \
  --env CREATE_TUN_DEVICE=False \
  --env HEALTH_CHECK_HOST=am.i.mullvad.net \
  --env LOCAL_NETWORK=192.168.178.0/24 \
  --env LOG_TO_STDOUT=True \
  --env OPENVPN_CONFIG=somecountry_all \
  --env OPENVPN_PASSWORD=m \
  --env OPENVPN_PROVIDER=MULLVAD \
  --env OPENVPN_USERNAME=my-mullvad-id \
  --env PGID=900 \
  --env PUID=900 \
  --env TRANSMISSION_ENCRYPTION=2 \
  --env TRANSMISSION_IDLE_SEEDING_LIMIT=30 \
  --env TRANSMISSION_IDLE_SEEDING_LIMIT_ENABLED=True \
  --env TRANSMISSION_PEER_PORT=12345 \
  --env TRANSMISSION_RATIO_LIMIT=2 \
  --env TRANSMISSION_RATIO_LIMIT_ENABLED=True \
  --env TZ=Europe/Berlin \
  --sysctl=net.ipv6.conf.all.disable_ipv6=0 \
  --device=/dev/net/tun \
  docker.io/haugene/transmission-openvpn:latest

This seems to work and bring up the tunnel but fails to start transmission becvause of this:

Nov 19 01:42:02 downloader conmon[3973133]: Updating Transmission settings.json with values from env variables
Nov 19 01:42:02 downloader conmon[3973133]: Using existing settings.json for Transmission /data/transmission-home/settings.json
Nov 19 01:42:02 downloader conmon[3973133]: Traceback (most recent call last):
Nov 19 01:42:02 downloader conmon[3973133]:   File "/etc/transmission/updateSettings.py", line 44, in <module>
Nov 19 01:42:02 downloader conmon[3973133]:     with open(configuration_baseline, 'r') as input_file:
Nov 19 01:42:02 downloader conmon[3973133]: PermissionError: [Errno 13] Permission denied: '/data/transmission-home/settings.json'

The folder should have proper permissions:

ls -la /data/transmission-home/
total 293420
drwxrwxr-x  5   900   900         7 Nov 19 01:37 .
drwxr-xr-x 15  1000   969        13 Sep 15 18:12 ..
drwxr-xr-x  2   900   900         0 Jul  2 00:02 blocklists
-rw-------  1   900   900      1583 Nov 11 21:37 dht.dat
drwxr-xr-x  2   900   900         0 Nov 11 08:42 resume
-rw-------  1   900   900      2302 Nov 11 21:37 settings.json
-rw-------  1   900   900       161 Nov 11 08:43 stats.json
drwxr-xr-x  2   900   900         0 Nov 11 08:42 torrents
-rw-r--r--  1   900   900 300456455 Nov 11 21:37 transmission.log

I'm not sure what goes wrong here. Does anyone have a clue?

docker-transmission-openvpn/transmission/updateSettings.py

Line 44 in d1ece1c

with open(configuration_baseline, 'r') as input_file:

I already checked the code around the error, but that looks fine. It only opens the file read only. Also at this point it should open the file as root so it should definitely have enough access rights.

I tried changing the permissions on that file to -rw-rw-r-- and that worked but then it failed a bit later when writing the settings back:

Nov 19 01:58:00 downloader conmon[3974579]: Traceback (most recent call last):
Nov 19 01:58:00 downloader conmon[3974579]:   File "/etc/transmission/updateSettings.py", line 90, in <module>
Nov 19 01:58:00 downloader conmon[3974579]:     with open(transmission_settings, 'w') as fp:
Nov 19 01:58:00 downloader conmon[3974579]: PermissionError: [Errno 13] Permission denied: '/data/transmission-home/settings.json'

I'm not sure which user it is using at this point. It seems to be not root and not the one I provided.

---

On a different note. When I stopped the container I get this:

Nov 19 02:01:20 downloader conmon[3974579]: Sending kill signal to transmission-daemon
Nov 19 02:01:20 downloader conmon[3974579]: /etc/transmission/stop.sh: line 13: kill: `': not a pid or valid job spec

Not sure why this wouldn't work from the stop.sh script:

PID=$(pidof transmission-daemon)
kill "$PID"

[ignaciolg]

First of all, sorry for bringing this discussion alive, but is the closest to the issue that I had.

After some testing and search on google I was able to run the image on a "rootless" container (with some extra capabilities)
Biggest changes

• added the NET_RAW capability
• Modified the group_range allowed to execute ping with the sudo sysctl -w "net.ipv4.ping_group_range=0 2147483647" command

After applying this two changes, the container is running.

All the info is from

• https://superuser.com/questions/1702163/ping-socket-operation-not-permitted-ubuntu-docker-image/1702188#1702188
• https://superuser.com/questions/1746588/ping-does-not-work-on-a-rootless-ubuntu-podman-container-on-fedora

version: '3.3'
services:
    transmission-openvpn:
        container_name: transmission-openvpn
        image: haugene/transmission-openvpn:dev
        devices:
            - /dev/net/tun

        dns:
            - 1.1.1.1
            - 8.8.8.8


        ports:
            - '9091:9091'

        cap_add:
            - NET_ADMIN
            - NET_RAW

        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /mnt/mfs_media:/data
            - ./config/:/config


        environment:
            - OPENVPN_PROVIDER=NORDVPN
            - OPENVPN_USERNAME=${OPENVPN_USERNAME}
            - OPENVPN_PASSWORD=${OPENVPN_PASSWORD}
            - NORDVPN_CATEGORY=legacy_p2p
            - LOCAL_NETWORK=192.168.10.0/24
            - OPENVPN_OPTS=--inactive 3600 --ping 10 --ping-exit 60
            - CREATE_TUN_DEVICE=false

        logging:
            driver: json-file
            options:
                max-size: 10m

        restart: always

Downloads are working as expected