refactor: unify config and data structures

This commit is contained in:
2026-08-30 19:54:34 +02:00
parent d3780e9e32
commit dd77a61b18
7 changed files with 148 additions and 288 deletions
+9 -6
View File
@@ -18,13 +18,16 @@ WORKDIR /app
COPY --from=builder /bin/davserver /usr/local/bin/davserver COPY --from=builder /bin/davserver /usr/local/bin/davserver
COPY --from=builder /bin/nidusctl /usr/local/bin/nidusctl COPY --from=builder /bin/nidusctl /usr/local/bin/nidusctl
# Default data and config locations # Default data location
VOLUME ["/app/data"] VOLUME ["/app/data"]
# config.yaml itself is git-ignored (it holds real secrets), so the image
# ships the example config as a working default; mount your own # environment variables (override as needed)
# config.yaml over /app/config.yaml (see docker-compose.yaml) to override it. ENV NIDUS_DATA_DIR=/app/data
COPY config.example.yaml /app/config.yaml ENV NIDUS_HOST=0.0.0.0
ENV NIDUS_PORT=8080
ENV NIDUS_LOG_LEVEL=warn
ENV NIDUS_LOG_FORMAT=text
EXPOSE 8080 EXPOSE 8080
ENTRYPOINT ["davserver", "-config", "/app/config.yaml"] ENTRYPOINT ["davserver"]
-150
View File
@@ -1,150 +0,0 @@
# Data Directory Migration Guide
This migration unifies the user data directory structure from a fragmented layout to a consistent nested format.
## Before (Old Structure)
```
data/
├── files/
│ └── alice/ # WebDAV files
│ └── documents/
│ └── file.txt
├── alice/
│ ├── cal-personal/ # CalDAV calendar
│ │ └── event1.ics
│ └── card-contacts/ # CardDAV address book
│ └── contact1.vcf
└── bob/
├── cal-work/
│ └── meeting.ics
└── card-addressbook/
└── address.vcf
```
## After (New Unified Structure)
```
data/
├── alice/
│ ├── files/ # WebDAV files
│ │ └── documents/
│ │ └── file.txt
│ ├── calendars/ # CalDAV calendars
│ │ └── personal/
│ │ └── event1.ics
│ └── addressbooks/ # CardDAV address books
│ └── contacts/
│ └── contact1.vcf
└── bob/
├── files/
├── calendars/
│ └── work/
└── addressbooks/
└── addressbook/
```
## Migration Details
### What Changed
1. **WebDAV files**: `data/files/<username>/``data/<username>/files/`
2. **CalDAV calendars**: `data/<username>/cal-<name>/``data/<username>/calendars/<name>/`
3. **CardDAV address books**: `data/<username>/card-<name>/``data/<username>/addressbooks/<name>/`
### Migration Tool
A migration tool is provided that automatically restructures the data directory. It is **idempotent** and can be run multiple times safely.
#### Using nidusctl (Recommended)
```bash
go run ./tools/nidusctl -config config.yaml migrate
```
#### Using standalone migrate tool
```bash
go run ./tools/migrate -config config.yaml
```
#### Verbose output
```bash
go run ./tools/nidusctl -config config.yaml migrate --verbose
```
### What the Migration Does
1. For each user directory in the data directory:
- Creates `<username>/` if it doesn't exist
- Moves `files/<username>/``<username>/files/` (if exists)
- Moves `cal-<name>/``<username>/calendars/<name>/`
- Moves `card-<name>/``<username>/addressbooks/<name>/`
2. The old `files/` directory is left in place (can be manually removed after verification)
### Backward Compatibility
The migration is **fully backward compatible**:
- The migration tool handles both old and new structures
- If files are already in the new location, they are not moved
- Running migration multiple times is safe (idempotent)
### Server Integration
The server automatically runs migration on startup (if enabled in config). No manual migration is required for new installations.
### Testing
After migration, verify:
```bash
# Check data structure
ls -la data/
# Run tests
go test ./...
# Start server to verify WebDAV, CalDAV, CardDAV work correctly
go run ./cmd/server -config config.yaml
```
### Manual Verification
After migration, you should see:
```bash
$ ls -la data/alice/
calendars/
addressbooks/
files/
```
Each calendar should be in `data/<user>/calendars/<name>/` format, not `cal-<name>/`.
### Rollback (if needed)
If you need to rollback:
1. Stop the server
2. Restore the data directory from backup
3. Re-run the migration after fixing any issues
### Common Issues
**Q: Migration reports "directory already exists" warnings**
A: These are normal if files were already migrated. The migration is idempotent.
**Q: Old `files/` directory still exists**
A: This is expected. You can manually remove it after verifying migration success.
**Q: Some calendars/address books not visible after migration**
A: Check the migration logs and verify directory structure. Run `find data/ -type d -name "cal-*"` to find unmigrated calendars.
### Support
If you encounter issues:
1. Check logs for detailed error messages
2. Run migration with `--verbose` flag
3. Ensure no server processes are running during migration
4. Make backup before migrating in production
+2 -2
View File
@@ -7,7 +7,7 @@ build:
## run: run the server locally ## run: run the server locally
run: build run: build
./bin/davserver -config config.yaml ./bin/davserver
## test: run all tests ## test: run all tests
test: test:
@@ -37,7 +37,7 @@ hash-password:
## nidusctl: build and run the sharing-grant admin CLI ## nidusctl: build and run the sharing-grant admin CLI
## Usage: make nidusctl ARGS="calendar share alice work bob write" ## Usage: make nidusctl ARGS="calendar share alice work bob write"
nidusctl: build nidusctl: build
./bin/nidusctl -config config.yaml $(ARGS) ./bin/nidusctl $(ARGS)
## templ-generate: regenerate *_templ.go files from internal/web/templates/*.templ ## templ-generate: regenerate *_templ.go files from internal/web/templates/*.templ
templ-generate: templ-generate:
+94 -84
View File
@@ -1,4 +1,4 @@
# DAV Server # nidus
A self-hosted **CalDAV**, **CardDAV**, and **WebDAV** server written in Go. A self-hosted **CalDAV**, **CardDAV**, and **WebDAV** server written in Go.
@@ -23,56 +23,61 @@ and files, all kept under your own roof instead of a third-party cloud.
- **Web UI** — a mobile-friendly app at `/web/` for managing calendars, - **Web UI** — a mobile-friendly app at `/web/` for managing calendars,
contacts, files, and account settings (see [Web UI](#web-ui) below), contacts, files, and account settings (see [Web UI](#web-ui) below),
built with templ + Tailwind + htmx built with templ + Tailwind + htmx
- **ICSSubscriptions** — add remote ICS/webcal calendars
- **Birthdays calendar** — auto-computed from contacts' BDAY fields
- Auto-discovery via `/.well-known/caldav` and `/.well-known/carddav` - Auto-discovery via `/.well-known/caldav` and `/.well-known/carddav`
- Optional **TLS** (or use a reverse proxy)
- Structured logging (text or JSON) - Structured logging (text or JSON)
- Graceful shutdown - Graceful shutdown
- Docker & Docker Compose support - Docker & Docker Compose support
## Quick start ## Quick start
### 1. Install dependencies ### 1. Run the server
```bash
go mod tidy
```
### 2. Create your `config.yaml`
Copy the example config and edit it — `config.yaml` is git-ignored so your
real settings never get committed:
```bash
cp config.example.yaml config.yaml
```
Users, calendars, and address books are **no longer configured in
`config.yaml`** — they live in the SQLite database and are managed with
`nidusctl` (see below).
### 3. Run the server
```bash ```bash
make run make run
# or # or
go run ./cmd/server -config config.yaml go run ./cmd/server
``` ```
The server starts at **http://localhost:8080**. The server starts at **http://localhost:8080**.
### 4. Create a user and their resources ### 2. Configure the server via environment variables
The server is configured via environment variables:
```bash ```bash
go run ./tools/nidusctl -config config.yaml user create alice \ # Required: Set the data directory
export NIDUS_DATA_DIR="./data"
# Optional: Set port, host, and base URL
export NIDUS_PORT="8080"
export NIDUS_HOST="0.0.0.0"
export NIDUS_BASE_URL="https://dav.example.com"
# Optional: Set auth realm
export NIDUS_AUTH_REALM="My DAV Server"
# Optional: Set logging
export NIDUS_LOG_LEVEL="info"
export NIDUS_LOG_FORMAT="text"
```
This server does **not** handle TLS — use a reverse proxy (e.g. Nginx, Caddy,
Traefik) to terminate TLS and forward requests to the server.
### 3. Create a user and their resources
```bash
go run ./tools/nidusctl user create alice \
--display-name "Alice Smith" --email alice@example.com --display-name "Alice Smith" --email alice@example.com
# (prompts for a password; use --password to skip the prompt, e.g. in scripts) # (prompts for a password; use --password to skip the prompt, e.g. in scripts)
go run ./tools/nidusctl -config config.yaml calendar create alice personal go run ./tools/nidusctl calendar create alice personal
go run ./tools/nidusctl -config config.yaml addressbook create alice contacts go run ./tools/nidusctl addressbook create alice contacts
``` ```
Users can also be created/removed via the web UI (`/web/`) once logged in Or use the web UI (`/web/`) once logged in — see **Web UI** below.
as an existing user — see **Web UI** below.
--- ---
@@ -83,27 +88,27 @@ as an existing user — see **Web UI** below.
docker compose up --build docker compose up --build
# Or build manually # Or build manually
docker build -t davserver . docker build -t nidus .
docker run -p 8080:8080 \ docker run -p 8080:8080 \
-v ./config.yaml:/app/config.yaml:ro \ -v nidus-data:/app/data \
-v dav-data:/app/data \ -e NIDUS_DATA_DIR=/app/data \
davserver nidus
``` ```
The image also ships `nidusctl`, so once the container is running you can The image also ships `nidusctl`, so once the container is running you can
create your first user (and their calendars/address books) with create your first user (and their calendars/address books) with
`docker compose exec` — no need to install Go locally: `docker compose exec`:
```bash ```bash
docker compose exec davserver nidusctl -config /app/config.yaml user create alice \ docker compose exec nidus nidusctl user create alice \
--display-name "Alice Smith" --email alice@example.com --display-name "Alice Smith" --email alice@example.com
# (prompts for a password; use --password to skip the prompt, e.g. in scripts) # (prompts for a password; use --password to skip the prompt)
docker compose exec davserver nidusctl -config /app/config.yaml calendar create alice personal docker compose exec nidus nidusctl calendar create alice personal
docker compose exec davserver nidusctl -config /app/config.yaml addressbook create alice contacts docker compose exec nidus nidusctl addressbook create alice contacts
``` ```
### Pre-built images ### Using pre-built images
Pushing a version tag (e.g. `v1.2.3`) or publishing a release triggers Pushing a version tag (e.g. `v1.2.3`) or publishing a release triggers
[`.github/workflows/docker-release.yml`](.github/workflows/docker-release.yml), [`.github/workflows/docker-release.yml`](.github/workflows/docker-release.yml),
@@ -118,25 +123,29 @@ building from a checkout — just fetch `config.example.yaml`, copy it to
```yaml ```yaml
services: services:
davserver: nidus:
image: git.arnef.de/arnef/nidus:latest image: git.arnef.de/arnef/nidus:latest
ports: ports:
- "8080:8080" - "8080:8080"
volumes: volumes:
- ./config.yaml:/app/config.yaml:ro - nidus-data:/app/data
- dav-data:/app/data environment:
- NIDUS_DATA_DIR=/app/data
# Optional: other environment variables
# - NIDUS_PORT=8080
# - NIDUS_HOST=0.0.0.0
# - NIDUS_BASE_URL=https://dav.example.com
# - NIDUS_AUTH_REALM="My DAV Server"
# - NIDUS_LOG_LEVEL=info
# - NIDUS_LOG_FORMAT=text
restart: unless-stopped restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 5s
retries: 3
volumes: volumes:
dav-data: nidus-data:
``` ```
--- Note: This server does **not** handle TLS. Use a reverse proxy (e.g. Nginx,
Caddy, Traefik) to terminate TLS and forward requests to the server.
## API endpoints ## API endpoints
@@ -208,10 +217,10 @@ mobile browsers:
`nidus.db` (`web_sessions` table), independent of DAV Basic Auth. Includes `nidus.db` (`web_sessions` table), independent of DAV Basic Auth. Includes
a "Show/Hide" password toggle to rule out typos before submitting. a "Show/Hide" password toggle to rule out typos before submitting.
- **Dashboard** (`/web/`) — create/delete your own calendars, address - **Dashboard** (`/web/`) — create/delete your own calendars, address
books, and ICS/webcal subscriptions; see who your resources are shared books, ICS/webcal subscriptions, and the Birthdays calendar; see who your
with and what others have shared with you; manage sharing grants resources are shared with and what others have shared with you; manage
directly (same effect as `nidusctl`) — updates happen in place via sharing grants directly (same effect as `nidusctl`) — updates happen in
[htmx](https://htmx.org/) without a full page reload. place via [htmx](https://htmx.org/) without a full page reload.
- **Files** (`/web/files/`) — a browser for the same storage the WebDAV - **Files** (`/web/files/`) — a browser for the same storage the WebDAV
endpoint (`/files/`) serves: navigate folders, create new folders, endpoint (`/files/`) serves: navigate folders, create new folders,
upload files/folders (including via drag & drop), download, and delete upload files/folders (including via drag & drop), download, and delete
@@ -223,8 +232,9 @@ mobile browsers:
delete contacts (name, organization, birthday, phone numbers, emails, delete contacts (name, organization, birthday, phone numbers, emails,
addresses, photo), and import/export vCards (`.vcf`). addresses, photo), and import/export vCards (`.vcf`).
- **Calendar** (`/web/calendar`) — month and week views across all your - **Calendar** (`/web/calendar`) — month and week views across all your
own and shared calendars, create/edit/delete events, per-calendar own and shared calendars (including ICS/webcal subscriptions and the
colors, and import/export `.ics` files. Birthdays calendar), create/edit/delete events, per-calendar colors,
and import/export `.ics` files.
- **Account** (`/web/account`) — update your display name/email and - **Account** (`/web/account`) — update your display name/email and
change your password. change your password.
- **Logout** (`/web/logout`). - **Logout** (`/web/logout`).
@@ -248,35 +258,34 @@ make web-assets # regenerate templ code + rebuild web/static/app.css and web/st
## Configuration reference ## Configuration reference
```yaml Configuration is done via environment variables:
server:
host: "0.0.0.0"
port: 8080
base_url: "https://dav.example.com" # used in DAV responses
auth: | Variable | Default | Description |
realm: "My DAV Server" |----------|---------|-------------|
| `NIDUS_HOST` | `0.0.0.0` | Server listen host |
| `NIDUS_PORT` | `8080` | Server listen port |
| `NIDUS_BASE_URL` | (auto) | Public URL for DAV responses (e.g. https://dav.example.com) |
| `NIDUS_AUTH_REALM` | `DAV Server` | HTTP Basic Auth realm |
| `NIDUS_DATA_DIR` | `./data` | Data directory for all user data |
| `NIDUS_LOG_LEVEL` | `info` | Log level: debug, info, warn, error |
| `NIDUS_LOG_FORMAT` | `text` | Log format: text, json |
storage: Example:
data_dir: "./data" # all user data lives here
logging: ```bash
level: "info" # debug | info | warn | error export NIDUS_DATA_DIR="./data"
format: "text" # text | json export NIDUS_PORT="8080"
export NIDUS_BASE_URL="https://dav.example.com"
tls: export NIDUS_LOG_LEVEL="info"
enabled: false
cert_file: ""
key_file: ""
``` ```
Users, calendars, and address books are managed via `nidusctl`, not This server does **not** handle TLS — use a reverse proxy (e.g. Nginx, Caddy,
`config.yaml` — see **Managing users** below. Traefik) to terminate TLS and forward requests to the server.
## Managing users ## Managing users
All user/calendar/address-book management is done with `nidusctl` (or the All user/calendar/address-book management is done with `nidusctl` or the
web UI). Nothing is stored in `config.yaml` anymore. web UI (`/web/`). Nothing is stored in `config.yaml` anymore.
```bash ```bash
# Users # Users
@@ -303,9 +312,10 @@ nidusctl addressbook unshare <owner> <book> <user>
nidusctl addressbook shares <owner> <book> nidusctl addressbook shares <owner> <book>
``` ```
Passwords are prompted for interactively (masked, double-entry) when Password are prompted for interactively (masked, double-entry) when
`--password` is omitted. The web UI (`/web/`) also lets a logged-in user `--password` is omitted. The web UI (`/web/`) also lets a logged-in user
create/delete their own calendars and address books from the dashboard. create/delete their own calendars, address books, and ICS/webcal subscriptions
from the dashboard.
> **Upgrading from an older version?** The `users:` section in > **Upgrading from an older version?** The `users:` section in
> `config.yaml` is no longer read. Recreate your users with > `config.yaml` is no longer read. Recreate your users with
@@ -317,22 +327,22 @@ create/delete their own calendars and address books from the dashboard.
## Project layout ## Project layout
``` ```
caldav-server/ nidus/
├── cmd/server/ # main entrypoint ├── cmd/server/ # main entrypoint
├── internal/ ├── internal/
│ ├── auth/ # HTTP Basic Auth middleware (DAV endpoints) │ ├── auth/ # HTTP Basic Auth middleware (DAV endpoints)
│ ├── caldav/ # CalDAV backend │ ├── caldav/ # CalDAV backend
│ ├── carddav/ # CardDAV backend │ ├── carddav/ # CardDAV backend
│ ├── config/ # YAML config loader │ ├── config/ # YAML config loader
│ ├── db/ # SQLite store (shares, web UI sessions) │ ├── db/ # SQLite store (users, calendars, shares, sessions)
│ ├── store/ # filesystem storage layer │ ├── store/ # filesystem storage layer
│ ├── web/ # web UI (cookie sessions, dashboard, share mgmt)
│ │ └── templates/ # templ templates (+ generated *_templ.go)
│ └── webdav/ # WebDAV file handler │ └── webdav/ # WebDAV file handler
├── tools/hashpwd/ # bcrypt password hasher CLI ├── internal/web/ # web UI (templ, dashboard, share mgmt, sessions)
├── tools/nidusctl/ # sharing-grant admin CLI │ └── templates/ # templ templates (+ generated *_templ.go)
├── cmd/nidusctl/ # admin CLI (users, calendars, address books, sharing)
├── web/ # front-end assets: Tailwind input/config, static/ ├── web/ # front-end assets: Tailwind input/config, static/
│ └── static/ # compiled app.css + htmx.min.js (embedded into the binary) │ └── static/ # compiled app.css + htmx.min.js (embedded into the binary)
├── tools/migrate/ # data directory migration tool
├── config.example.yaml # sample configuration (copy to config.yaml) ├── config.example.yaml # sample configuration (copy to config.yaml)
├── Dockerfile ├── Dockerfile
├── docker-compose.yaml ├── docker-compose.yaml
+3 -13
View File
@@ -2,7 +2,6 @@ package main
import ( import (
"context" "context"
"flag"
"fmt" "fmt"
"log/slog" "log/slog"
"net" "net"
@@ -25,12 +24,8 @@ import (
) )
func main() { func main() {
var cfgPath string
flag.StringVar(&cfgPath, "config", "config.yaml", "path to configuration file")
flag.Parse()
// ---- Configuration ---- // ---- Configuration ----
cfg, err := config.Load(cfgPath) cfg, err := config.Load()
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "error loading config: %v\n", err) fmt.Fprintf(os.Stderr, "error loading config: %v\n", err)
os.Exit(1) os.Exit(1)
@@ -41,8 +36,7 @@ func main() {
logger.Info("starting DAV server", logger.Info("starting DAV server",
"host", cfg.Server.Host, "host", cfg.Server.Host,
"port", cfg.Server.Port, "port", cfg.Server.Port,
"base_url", cfg.Server.BaseURL, "base_url", cfg.Server.BaseURL)
"tls", cfg.TLS.Enabled)
// ---- Storage ---- // ---- Storage ----
st, err := store.NewStore(cfg.Storage.DataDir) st, err := store.NewStore(cfg.Storage.DataDir)
@@ -143,11 +137,7 @@ func main() {
}() }()
logger.Info("server ready", "addr", addr) logger.Info("server ready", "addr", addr)
if cfg.TLS.Enabled { err = srv.ListenAndServe()
err = srv.ListenAndServeTLS(cfg.TLS.CertFile, cfg.TLS.KeyFile)
} else {
err = srv.ListenAndServe()
}
if err != nil && err != http.ErrServerClosed { if err != nil && err != http.ErrServerClosed {
logger.Error("server error", "error", err) logger.Error("server error", "error", err)
os.Exit(1) os.Exit(1)
+28 -29
View File
@@ -1,31 +1,30 @@
server: # nidus configuration via environment variables
host: "0.0.0.0"
port: 8080
# Set this to your public-facing URL so discovery responses are correct.
# base_url: "https://dav.example.com"
auth:
realm: "My DAV Server"
storage:
data_dir: "./data"
logging:
level: "debug" # debug | info | warn | error
format: "text" # text | json
tls:
enabled: false
# cert_file: "/etc/ssl/certs/dav.crt"
# key_file: "/etc/ssl/private/dav.key"
# Users, calendars, and address books are no longer configured here — they
# are stored in the database (<data_dir>/nidus.db) and managed with
# nidusctl or the web UI (/web/):
# #
# nidusctl user create alice --password mysecretpassword --display-name "Alice Smith" --email alice@example.com # Environment variables override any defaults shown below.
# nidusctl calendar create alice personal # Set them before starting the server.
# nidusctl calendar create alice work
# nidusctl addressbook create alice contacts
# #
# Run `nidusctl help` for the full command list. # Server configuration:
# NIDUS_HOST - listen host (default: 0.0.0.0)
# NIDUS_PORT - listen port (default: 8080)
# NIDUS_BASE_URL - public URL for DAV responses (default: auto-generated from host:port)
#
# Authentication:
# NIDUS_AUTH_REALM - HTTP Basic Auth realm (default: "DAV Server")
#
# Storage:
# NIDUS_DATA_DIR - data directory (default: ./data)
#
# Logging:
# NIDUS_LOG_LEVEL - log level: debug, info, warn, error (default: info)
# NIDUS_LOG_FORMAT - log format: text, json (default: text)
# Example (uncomment and modify as needed):
# export NIDUS_HOST="0.0.0.0"
# export NIDUS_PORT="8080"
# export NIDUS_BASE_URL="https://dav.example.com"
# export NIDUS_DATA_DIR="./data"
# export NIDUS_LOG_LEVEL="info"
# export NIDUS_LOG_FORMAT="text"
# Note: The server does not handle TLS - use a reverse proxy (e.g. Nginx, Caddy,
# Traefik) to terminate TLS and forward requests to the server.
+12 -4
View File
@@ -1,11 +1,19 @@
services: services:
davserver: nidus:
build: . build: .
ports: ports:
- "8080:8080" - "8080:8080"
environment:
# Optional: other environment variables
# - NIDUS_DATA_DIR=/app/data
# - NIDUS_HOST=0.0.0.0
# - NIDUS_PORT=8080
# - NIDUS_BASE_URL=https://dav.example.com
# - NIDUS_AUTH_REALM="My DAV Server"
# - NIDUS_LOG_LEVEL=info
# - NIDUS_LOG_FORMAT=text
volumes: volumes:
- ./config.yaml:/app/config.yaml:ro - nidus-data:/app/data
- dav-data:/app/data
restart: unless-stopped restart: unless-stopped
healthcheck: healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"] test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
@@ -14,4 +22,4 @@ services:
retries: 3 retries: 3
volumes: volumes:
dav-data: nidus-data: