micro
/
woodbine.nyc
1
0
Fork 0
Code Issues 4 Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: micro:main
Branches Tags
micro:main
micro:authelia
micro:caddy-docker-proxy
..
compare: micro:authelia
Branches Tags
micro:main
micro:authelia
micro:caddy-docker-proxy

No commits in common. 'main' and 'authelia' have entirely different histories.
main ... authelia

18 changed files with 50 additions and 211 deletions
Show Stats

14
env.template
Unescape View File

@ -1,15 +1,5 @@
# copy this to .env and it will be sourced by the appropriate services
# domain your services will be running on
DOMAIN=localhost
# admin user for auth
ADMIN_USER=
ADMIN_PASS=
# used for sending notifications and reset passwords
# only supports smtp+starttls
SMTP_ADDR=
SMTP_USER=admin
SMTP_HOST=localhost
SMTP_PORT=587
SMTP_USER=
SMTP_PASS=

121
readme.md
Unescape View File

@ -4,133 +4,58 @@ Experiment in digital autonomy
Latest code is hosted on https://git.woodbine.nyc/micro/woodbine.nyc
If you are new to running your own websites, welcome!
In general, everything is orchestrated by the compose files.
Note that a "service" is a fuzzy name for software that is expected to be always running.
A simple web server (`python3 -m http.server`) could be a service, as could something like Gmail.
Sometimes, you will see a -setup service in the compose file.
This usually runs a script that checks or generates secrets, and does initial configuration if needed.
## Goals
Understandable
- a person should be able to adapt this to their community while learning the least amount of new concepts and technology
- the person who set it up should not be needed to maintain the services
Resiliant
- services should work even when other parts of the web are not accessible
Lean
- we prefer lightweight software, which usually require less long-term maintenance
## Decisions
There are many other kinds of digital autonomy, but most people are used to the web.
We hope to share our decision making here, so you can follow our thought process.
### Decisions made for you
These needs are required for anyone who wants to deploy **web-based** services.
#### Auth
We need a way for people to either register an account or sign in with an external account to use the services.
After trying authelia, zitadel, authentik, and keycloak, got the furthest with zitadel.
#### Web
To host a webpage, you need some software that listens for http requests. We chose Caddy.
If you would like to edit the webpage, either change the files in `./data/web/site/` directly, or you can connect via WebDAV and edit the file remotely via https://web.localhost.
#### Backup
If you will be helping a community, its important to have backups and restore. We have two helper services, `backup-files` and `backup-database`.
These use duplicity to backup to a backblaze instance, so you will need to setup that beforehand.
We hope this is understandable by a single individual, after learning a bit about docker compose and caddy.
#### Secrets
We have two helper services for making sure secrets exist (`check-secrets`), or generating unique secrets for other services that need them (`generate-secrets`).
---
## getting started
### setup
## setup
Make a backblaze B2 account for backups. Add the secrets to ./secrets/backup/.
Fill out env.template and make sure to pass it in the next command
### running
Helper scripts can be found in [the scripts directory](./scripts)
## running
To start
We have two scripts in the `scripts/` directory - up and down
./scripts/up
To stop, you can press ctrl+c, or in another terminal run
To stop all the containers, you can ctrl+c, or
./scripts/down
To generate secrets for all services ahead-of-time
To generate secrets for all services
./scripts/generate-secrets
./scripts/secrets
### port forwarding
## port forwarding
The caddy service expects to be able to bind to ports 80 and 443
One simple way is to allow unprivileged users access to these low ports
If you are on linux, you can run
$ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
$ echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/sysctl.conf
The first command will set privileges until reboot. The second will make those privileges permanent.
If you are on macOS, using podman, you will want to run those commands in the linux virtual machine
$ podman machine ssh
core@localhost:~$ echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/systctl.conf
core@localhost:~$ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
---
echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee -a /etc/sysctl.conf
sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
## design
## alpha
All the services are defined by docker compose files.
We provide `backup-files`, `backup-database`, `check-secrets`, and `generate-secrets` helper services.
We have configured Caddy to import all files found in /etc/caddy.d/, so if you want to add a new service, you will need to make a small `Proxyfile` to tell caddy what subdomain to forward to what port.
See [the services readme](./services/readme.md) for a guide on adding a new service.
---
## roadmap
### alpha
- [ ] decide on single postgres instance or multiple
- [~] single sign-on (authelia)
- [ ] per-user webdav folders via authelia
- [ ] any OIDC service setup
- [~] file backup (duplicity)
- [ ] postgres backup (duplicity)
- [ ] single sign-on for webdav (one user per folder)
- [ ] single sign-on for one more service
- [x] identity provider (zitadel)
- [x] file backup (duplicity)
- [ ] decide on single postgres instance or multiple
- [x] reverse proxy (caddy)
- [x] personal home pages (caddy-webdav)
- [x] migrate from yaml to env for authelia config
- [x] setup notifications via smtp
### beta
## beta
- [ ] file restore
- [ ] postgres restore
@ -140,10 +65,10 @@ See [the services readme](./services/readme.md) for a guide on adding a new serv
- [ ] mailing list (listmonk)
- [ ] code forge (gitea or forgejo)
### 0.1
## 0.1
- [ ] only expose 443, 587, 993
- [ ] running on beta.woodbine.nyc
- [ ] running on betabasement-woodbine.nyc
- [ ] audit on secrets management
- [ ] audit on mail server
- [ ] audit on general architecture

3
scripts/down
Unescape View File

@ -1,8 +1,7 @@
podman compose --env-file ${ENV_FILE:-.env} \
podman compose --env-file env.production \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
down --volumes

8
scripts/exec
Unescape View File

@ -1,8 +0,0 @@
podman compose --env-file ${ENV_FILE:-.env} \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
exec "$@"

8
scripts/ps
Unescape View File

@ -1,8 +0,0 @@
podman compose --env-file ${ENV_FILE:-.env} \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
ps

8
scripts/pull
Unescape View File

@ -1,8 +0,0 @@
podman compose --env-file ${ENV_FILE:-.env} \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
pull

8
scripts/run
Unescape View File

@ -1,8 +0,0 @@
podman compose --env-file ${ENV_FILE:-.env} \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
run "$@"

3
scripts/up
Unescape View File

@ -1,8 +1,7 @@
podman compose --env-file ${ENV_FILE:-.env} \
podman compose --env-file env.production \
--file services/secrets.yaml \
--file services/backup.yaml \
--file services/proxy.yaml \
--file services/auth.yaml \
--file services/web.yaml \
--file services/git.yaml \
up --build

14
services/auth.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
secrets:
MASTER_KEY:
file: ../secrets/auth/zitadel/MASTER_KEY
@ -13,7 +15,7 @@ services:
zitadel:
restart: 'unless-stopped'
image: 'ghcr.io/zitadel/zitadel:v2.48.3'
image: 'ghcr.io/zitadel/zitadel:latest'
environment:
ZITADEL_DATABASE_COCKROACH_HOST: crdb
ZITADEL_EXTERNALSECURE: true
@ -23,15 +25,11 @@ services:
ZITADEL_FIRSTINSTANCE_ORG_NAME: basement
ZITADEL_FIRSTINSTANCE_ORG_HUMAN_USERNAME: ${ADMIN_USER}
ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD: ${ADMIN_PASS}
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_HOST: "${SMTP_ADDR}:${SMTP_PORT}"
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROM: basement
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROMNAME: ${DOMAIN}
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_HOST: "${SMTP_HOST}:${SMTP_PORT}"
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_USER: ${SMTP_USER}
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_PASSWORD: ${SMTP_PASS}
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_SSL: true
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROM: basement@mail.${DOMAIN}
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_FROMNAME: basement
ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_REPLYTOADDRESS: basement@mail.${DOMAIN}
secrets:
- MASTER_KEY
command: "start-from-init --masterkeyFile /run/secrets/MASTER_KEY --tlsMode external"

4
services/backup.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
secrets:
B2_APPLICATION_KEY:
file: ../secrets/backup/duplicity/B2_APPLICATION_KEY
@ -10,7 +12,7 @@ secrets:
services:
backup:
image: ghcr.io/tecnativa/docker-duplicity:3.3.1
image: tecnativa/docker-duplicity:latest
restart: unless-stopped
depends_on:
generate-secrets:

48
services/git.yaml
Unescape View File

@ -1,48 +0,0 @@
secrets:
DB_PASSWD:
file: ../secrets/git/gitea/DB_PASSWD
services:
caddy:
volumes:
- ./git/Proxyfile:/etc/caddy.d/git
backup:
volumes:
- ../data/git:/mnt/backup/src/git
gitea:
image: gitea/gitea:1.21.3-rootless
secrets: [ DB_PASSWD ]
environment:
GITEA__database__DB_TYPE: postgres
GITEA__database__HOST: "db:5432"
GITEA__database__NAME: gitea
GITEA__database__USER: gitea
GITEA__database__PASSWD__FILE: /run/secrets/DB_PASSWD
GITEA__mailer__ENABLED: true
GITEA__mailer__FROM: gitea@mail.${DOMAIN}
GITEA__mailer__PROTOCOL: smtp+starttls
GITEA__mailer__SMTP_ADDR: ${SMTP_ADDR}
GITEA__mailer__SMTP_PORT: ${SMTP_PORT}
GITEA__mailer__USER: ${SMTP_USER}
GITEA__mailer__PASSWD: ${SMTP_PASS}
restart: unless-stopped
volumes:
- ../data/git/gitea/data:/data
ports:
- 3000:3000
db:
image: postgres:16.1-alpine
secrets: [ DB_PASSWD ]
environment:
POSTGRES_USER: gitea
POSTGRES_PASSWORD_FILE: /run/secrets/DB_PASSWD
POSTGRES_DB: gitea
restart: unless-stopped
volumes:
- db_data:/var/lib/postgresql/data
expose:
- 5432
volumes:
db_data:

3
services/git/Proxyfile
Unescape View File

@ -1,3 +0,0 @@
git.{$DOMAIN} {
reverse_proxy gitea:3000
}

7
services/mail.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
secrets:
SMTP_PASSWORD:
file: ../secrets/mail/SMTP_PASSWORD
@ -16,12 +18,13 @@ services:
- ./mail/Proxyfile:/etc/caddy.d/mail:ro
maddy:
image: foxcpp/maddy:0.7
image: foxcpp/maddy:latest
secrets: [SMTP_PASSWORD]
restart: unless-stopped
depends_on:
generate-secrets:
condition: 'service_completed_successfully'
environment:
- MADDY_HOSTNAME=mx.mail.${DOMAIN}
- MADDY_DOMAIN=mail.${DOMAIN}
@ -37,7 +40,7 @@ services:
- 993:993
roundcube:
image: roundcube/roundcubemail:1.6.5-fpm-alpine
image: roundcube/roundcubemail:1.6.x-apache
environment:
ROUNDCUBEMAIL_DEFAULT_HOST: ssl://mx.mail.${DOMAIN}
ROUNDCUBEMAIL_DEFAULT_PORT: 993

2
services/proxy.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
services:
caddy:
image: caddy

4
services/readme.md
Unescape View File

@ -16,9 +16,9 @@ we have a backup script that uses duplicity, this should be moved into a contain
caddy is the web server, and handles https certificates, and proxying to all the services.
#### [Zitadel](ihttps://zitadel.com/docs/self-hosting/deploy/overview)
#### [Authelia](https://www.authelia.com/overview/prologue/introduction/) **WIP**
zitadel lets you have a single username and password to sign on to all your services.
authelia lets you have a single username and password to sign on to all your services.
### Optional Services

2
services/secrets.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
services:
generate-secrets:
image: alpine/openssl

2
services/web.yaml
Unescape View File

@ -1,3 +1,5 @@
version: "3.8"
services:
web:
build:

2
services/web/Caddyfile
Unescape View File

@ -5,7 +5,7 @@ root * /site
@notget not method GET
route @notget {
webdav
webdav
}
file_server browse

Write Preview
Loading…
Cancel
Save