explain how to run locally on macOS

This reverts commit aaf706bf77.

env.template

@@ -1,5 +1,15 @@
+# copy this to .env and it will be sourced by the appropriate services 
+
+# domain your services will be running on
 DOMAIN=localhost
 
-SMTP_USER=admin
-SMTP_HOST=localhost
+# admin user for auth
+ADMIN_USER=
+ADMIN_PASS=
+
+# used for sending notifications and reset passwords
+# only supports smtp+starttls
+SMTP_ADDR=
 SMTP_PORT=587
+SMTP_USER=
+SMTP_PASS=

readme.md

@@ -4,58 +4,133 @@ Experiment in digital autonomy
 
 Latest code is hosted on https://git.woodbine.nyc/micro/woodbine.nyc
 
-In general, everything is orchestrated by the compose files.
+If you are new to running your own websites, welcome!
 
-Sometimes, you will see a -setup service in the compose file.
-This usually runs a script that checks or generates secrets, and does initial configuration if needed.
+Note that a "service" is a fuzzy name for software that is expected to be always running.
+
+A simple web server (`python3 -m http.server`) could be a service, as could something like Gmail.
 
 ## Goals
 
-We hope this is understandable by a single individual, after learning a bit about docker compose and caddy.
+Understandable
+
+- a person should be able to adapt this to their community while learning the least amount of new concepts and technology
+- the person who set it up should not be needed to maintain the services
+
+Resiliant
+
+- services should work even when other parts of the web are not accessible
+
+Lean
+
+- we prefer lightweight software, which usually require less long-term maintenance
+
+## Decisions
+
+There are many other kinds of digital autonomy, but most people are used to the web.
+
+We hope to share our decision making here, so you can follow our thought process.
+
+### Decisions made for you
+
+These needs are required for anyone who wants to deploy **web-based** services.
+
+#### Auth
+
+We need a way for people to either register an account or sign in with an external account to use the services.
+
+After trying authelia, zitadel, authentik, and keycloak, got the furthest with zitadel.
+
+#### Web
+
+To host a webpage, you need some software that listens for http requests. We chose Caddy.
+
+If you would like to edit the webpage, either change the files in `./data/web/site/` directly, or you can connect via WebDAV and edit the file remotely via https://web.localhost.
+
+#### Backup
+
+If you will be helping a community, its important to have backups and restore. We have two helper services, `backup-files` and `backup-database`.
+
+These use duplicity to backup to a backblaze instance, so you will need to setup that beforehand.
 
-## setup
+#### Secrets
+
+We have two helper services for making sure secrets exist (`check-secrets`), or generating unique secrets for other services that need them (`generate-secrets`).
+
+---
+
+## getting started
+
+### setup
 
 Make a backblaze B2 account for backups. Add the secrets to ./secrets/backup/.
 
 Fill out env.template and make sure to pass it in the next command
 
-## running
+### running
+
+Helper scripts can be found in [the scripts directory](./scripts)
 
-We have two scripts in the `scripts/` directory - up and down
+To start 
 
     ./scripts/up
 
-To stop all the containers, you can ctrl+c, or 
+To stop, you can press ctrl+c, or in another terminal run
 
     ./scripts/down
 
-To generate secrets for all services
+To generate secrets for all services ahead-of-time
 
-    ./scripts/secrets
+    ./scripts/generate-secrets
 
-## port forwarding
+### port forwarding
 
 The caddy service expects to be able to bind to ports 80 and 443
 
 One simple way is to allow unprivileged users access to these low ports
 
-    echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/sysctl.conf
-    sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
+If you are on linux, you can run
 
-## alpha
+    $ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
+    $ echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/sysctl.conf
+
+The first command will set privileges until reboot. The second will make those privileges permanent.
+
+If you are on macOS, using podman, you will want to run those commands in the linux virtual machine
+
+    $ podman machine ssh
+    core@localhost:~$ echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/systctl.conf
+    core@localhost:~$ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
+
+---
+
+## design
+
+All the services are defined by docker compose files.
+
+We provide `backup-files`, `backup-database`, `check-secrets`, and `generate-secrets` helper services.
+
+We have configured Caddy to import all files found in /etc/caddy.d/, so if you want to add a new service, you will need to make a small `Proxyfile` to tell caddy what subdomain to forward to what port.
+
+See [the services readme](./services/readme.md) for a guide on adding a new service.
+
+--- 
+
+## roadmap
+
+### alpha
 
-- [~] single sign-on (authelia)
-    - [ ] per-user webdav folders via authelia
-    - [ ] any OIDC service setup
-- [~] file backup (duplicity) 
-- [ ] postgres backup (duplicity)
 - [ ] decide on single postgres instance or multiple
+- [ ] postgres backup (duplicity)
+- [ ] single sign-on for webdav (one user per folder)
+- [ ] single sign-on for one more service
+- [x] identity provider (zitadel)
+- [x] file backup (duplicity)
 - [x] reverse proxy (caddy)
 - [x] personal home pages (caddy-webdav)
-- [x] migrate from yaml to env for authelia config
 - [x] setup notifications via smtp
 
-## beta
+### beta
 
 - [ ] file restore
 - [ ] postgres restore
@@ -65,10 +140,10 @@ One simple way is to allow unprivileged users access to these low ports
 - [ ] mailing list (listmonk)
 - [ ] code forge (gitea or forgejo)
 
-## 0.1
+### 0.1
 
 - [ ] only expose 443, 587, 993
-- [ ] running on betabasement-woodbine.nyc
+- [ ] running on beta.woodbine.nyc
 - [ ] audit on secrets management
 - [ ] audit on mail server
 - [ ] audit on general architecture

scripts/down

@@ -1,7 +1,8 @@
-podman compose --env-file env.production \
+podman compose --env-file ${ENV_FILE:-.env} \
   --file services/secrets.yaml \
   --file services/backup.yaml  \
   --file services/proxy.yaml \
   --file services/auth.yaml \
   --file services/web.yaml \
+  --file services/git.yaml \
   down --volumes

scripts/exec

@@ -0,0 +1,8 @@
+podman compose --env-file ${ENV_FILE:-.env} \
+  --file services/secrets.yaml \
+  --file services/backup.yaml  \
+  --file services/proxy.yaml \
+  --file services/auth.yaml \
+  --file services/web.yaml \
+  --file services/git.yaml \
+  exec "$@"

scripts/ps

@@ -0,0 +1,8 @@
+podman compose --env-file ${ENV_FILE:-.env} \
+  --file services/secrets.yaml \
+  --file services/backup.yaml  \
+  --file services/proxy.yaml \
+  --file services/auth.yaml \
+  --file services/web.yaml \
+  --file services/git.yaml \
+  ps

scripts/pull

@@ -0,0 +1,8 @@
+podman compose --env-file ${ENV_FILE:-.env} \
+  --file services/secrets.yaml \
+  --file services/backup.yaml  \
+  --file services/proxy.yaml \
+  --file services/auth.yaml \
+  --file services/web.yaml \
+  --file services/git.yaml \
+  pull

scripts/run

@@ -0,0 +1,8 @@
+podman compose --env-file ${ENV_FILE:-.env} \
+  --file services/secrets.yaml \
+  --file services/backup.yaml  \
+  --file services/proxy.yaml \
+  --file services/auth.yaml \
+  --file services/web.yaml \
+  --file services/git.yaml \
+  run "$@"

scripts/up

@@ -1,7 +1,8 @@
-podman compose --env-file env.production \
+podman compose --env-file ${ENV_FILE:-.env} \
   --file services/secrets.yaml \
   --file services/backup.yaml  \
   --file services/proxy.yaml \
   --file services/auth.yaml \
   --file services/web.yaml \
+  --file services/git.yaml \
   up --build

services/auth.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 secrets:
   MASTER_KEY:
     file: ../secrets/auth/zitadel/MASTER_KEY
@@ -15,7 +13,7 @@ services:
 
   zitadel:
     restart: 'unless-stopped'
-    image: 'ghcr.io/zitadel/zitadel:latest'
+    image: 'ghcr.io/zitadel/zitadel:v2.48.3'
     environment:
       ZITADEL_DATABASE_COCKROACH_HOST: crdb
       ZITADEL_EXTERNALSECURE: true
@@ -25,11 +23,15 @@ services:
       ZITADEL_FIRSTINSTANCE_ORG_NAME: basement
       ZITADEL_FIRSTINSTANCE_ORG_HUMAN_USERNAME: ${ADMIN_USER}
       ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD: ${ADMIN_PASS}
-      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROM: basement
-      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROMNAME: ${DOMAIN}
-      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_HOST: "${SMTP_HOST}:${SMTP_PORT}"
+
+      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_HOST: "${SMTP_ADDR}:${SMTP_PORT}"
       ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_USER: ${SMTP_USER}
       ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_PASSWORD: ${SMTP_PASS}
+      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_SSL: true
+
+      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROM: basement@mail.${DOMAIN}
+      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROMNAME: basement
+      ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_REPLYTOADDRESS: basement@mail.${DOMAIN}
     secrets:
       - MASTER_KEY
     command: "start-from-init --masterkeyFile /run/secrets/MASTER_KEY --tlsMode external"

services/backup.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 secrets:
   B2_APPLICATION_KEY:
     file: ../secrets/backup/duplicity/B2_APPLICATION_KEY
@@ -12,7 +10,7 @@ secrets:
 
 services:
   backup:
-    image: tecnativa/docker-duplicity:latest
+    image: ghcr.io/tecnativa/docker-duplicity:3.3.1
     restart: unless-stopped
     depends_on: 
       generate-secrets:

services/git.yaml

@@ -0,0 +1,48 @@
+secrets:
+  DB_PASSWD:
+    file: ../secrets/git/gitea/DB_PASSWD
+
+services:
+  caddy:
+    volumes:
+      - ./git/Proxyfile:/etc/caddy.d/git
+  backup:
+    volumes:
+      - ../data/git:/mnt/backup/src/git
+
+  gitea:
+    image: gitea/gitea:1.21.3-rootless
+    secrets: [ DB_PASSWD ]
+    environment:
+      GITEA__database__DB_TYPE: postgres
+      GITEA__database__HOST: "db:5432"
+      GITEA__database__NAME: gitea
+      GITEA__database__USER: gitea
+      GITEA__database__PASSWD__FILE: /run/secrets/DB_PASSWD
+      GITEA__mailer__ENABLED: true
+      GITEA__mailer__FROM: gitea@mail.${DOMAIN}
+      GITEA__mailer__PROTOCOL: smtp+starttls
+      GITEA__mailer__SMTP_ADDR: ${SMTP_ADDR}
+      GITEA__mailer__SMTP_PORT: ${SMTP_PORT}
+      GITEA__mailer__USER: ${SMTP_USER}
+      GITEA__mailer__PASSWD: ${SMTP_PASS}
+    restart: unless-stopped
+    volumes:
+      - ../data/git/gitea/data:/data
+    ports:
+      - 3000:3000
+  db:
+    image: postgres:16.1-alpine
+    secrets: [ DB_PASSWD ]
+    environment:
+      POSTGRES_USER: gitea
+      POSTGRES_PASSWORD_FILE: /run/secrets/DB_PASSWD
+      POSTGRES_DB: gitea
+    restart: unless-stopped
+    volumes:
+      - db_data:/var/lib/postgresql/data
+    expose:
+      - 5432
+
+volumes:
+  db_data:

services/git/Proxyfile

@@ -0,0 +1,3 @@
+git.{$DOMAIN} {
+    reverse_proxy gitea:3000
+}

services/mail.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 secrets:
   SMTP_PASSWORD:
     file: ../secrets/mail/SMTP_PASSWORD
@@ -18,13 +16,12 @@ services:
       - ./mail/Proxyfile:/etc/caddy.d/mail:ro
 
   maddy:
-    image: foxcpp/maddy:latest
+    image: foxcpp/maddy:0.7
     secrets: [SMTP_PASSWORD]
     restart: unless-stopped
     depends_on:
       generate-secrets:
         condition: 'service_completed_successfully'
-
     environment:
       - MADDY_HOSTNAME=mx.mail.${DOMAIN}
       - MADDY_DOMAIN=mail.${DOMAIN}
@@ -40,7 +37,7 @@ services:
       - 993:993
 
   roundcube:
-    image: roundcube/roundcubemail:1.6.x-apache
+    image: roundcube/roundcubemail:1.6.5-fpm-alpine
     environment:
       ROUNDCUBEMAIL_DEFAULT_HOST: ssl://mx.mail.${DOMAIN}
       ROUNDCUBEMAIL_DEFAULT_PORT: 993

services/proxy.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 services:
   caddy:
     image: caddy

services/readme.md

@@ -16,9 +16,9 @@ we have a backup script that uses duplicity, this should be moved into a contain
 
 caddy is the web server, and handles https certificates, and proxying to all the services.
 
-#### [Authelia](https://www.authelia.com/overview/prologue/introduction/) **WIP**
+#### [Zitadel](ihttps://zitadel.com/docs/self-hosting/deploy/overview)
 
-authelia lets you have a single username and password to sign on to all your services.
+zitadel lets you have a single username and password to sign on to all your services.
 
 ### Optional Services
 

services/secrets.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 services:
   generate-secrets:
     image: alpine/openssl

services/web.yaml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 services:
   web:
     build: