arnef/nidus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Releases 3 Activity

refactor: unify config and data structures

Browse Source
This commit is contained in:
arnef
2026-08-30 19:54:34 +02:00
parent d3780e9e32
commit dd77a61b18
7 changed files with 148 additions and 288 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dockerfile
+9 -6
View File
@@ -18,13 +18,16 @@ WORKDIR /app
COPY --from=builder /bin/davserver /usr/local/bin/davserver
COPY --from=builder /bin/nidusctl /usr/local/bin/nidusctl
# Default data and config locations
# Default data location
VOLUME ["/app/data"]
# config.yaml itself is git-ignored (it holds real secrets), so the image
# ships the example config as a working default; mount your own
# config.yaml over /app/config.yaml (see docker-compose.yaml) to override it.
COPY config.example.yaml /app/config.yaml
# environment variables (override as needed)
ENV NIDUS_DATA_DIR=/app/data
ENV NIDUS_HOST=0.0.0.0
ENV NIDUS_PORT=8080
ENV NIDUS_LOG_LEVEL=warn
ENV NIDUS_LOG_FORMAT=text
EXPOSE 8080
ENTRYPOINT ["davserver", "-config", "/app/config.yaml"]
ENTRYPOINT ["davserver"]
MIGRATION.md
-150
View File
@@ -1,150 +0,0 @@
# Data Directory Migration Guide
This migration unifies the user data directory structure from a fragmented layout to a consistent nested format.
## Before (Old Structure)
```
data/
├── files/
│ └── alice/ # WebDAV files
│ └── documents/
│ └── file.txt
├── alice/
│ ├── cal-personal/ # CalDAV calendar
│ │ └── event1.ics
│ └── card-contacts/ # CardDAV address book
│ └── contact1.vcf
└── bob/
├── cal-work/
│ └── meeting.ics
└── card-addressbook/
└── address.vcf
```
## After (New Unified Structure)
```
data/
├── alice/
│ ├── files/ # WebDAV files
│ │ └── documents/
│ │ └── file.txt
│ ├── calendars/ # CalDAV calendars
│ │ └── personal/
│ │ └── event1.ics
│ └── addressbooks/ # CardDAV address books
│ └── contacts/
│ └── contact1.vcf
└── bob/
├── files/
├── calendars/
│ └── work/
└── addressbooks/
└── addressbook/
```
## Migration Details
### What Changed
1. **WebDAV files**: `data/files/<username>/` → `data/<username>/files/`
2. **CalDAV calendars**: `data/<username>/cal-<name>/` → `data/<username>/calendars/<name>/`
3. **CardDAV address books**: `data/<username>/card-<name>/` → `data/<username>/addressbooks/<name>/`
### Migration Tool
A migration tool is provided that automatically restructures the data directory. It is **idempotent** and can be run multiple times safely.
#### Using nidusctl (Recommended)
```bash
go run ./tools/nidusctl -config config.yaml migrate
```
#### Using standalone migrate tool
```bash
go run ./tools/migrate -config config.yaml
```
#### Verbose output
```bash
go run ./tools/nidusctl -config config.yaml migrate --verbose
```
### What the Migration Does
1. For each user directory in the data directory:
- Creates `<username>/` if it doesn't exist
- Moves `files/<username>/` → `<username>/files/` (if exists)
- Moves `cal-<name>/` → `<username>/calendars/<name>/`
- Moves `card-<name>/` → `<username>/addressbooks/<name>/`
2. The old `files/` directory is left in place (can be manually removed after verification)
### Backward Compatibility
The migration is **fully backward compatible**:
- The migration tool handles both old and new structures
- If files are already in the new location, they are not moved
- Running migration multiple times is safe (idempotent)
### Server Integration
The server automatically runs migration on startup (if enabled in config). No manual migration is required for new installations.
### Testing
After migration, verify:
```bash
# Check data structure
ls -la data/
# Run tests
go test ./...
# Start server to verify WebDAV, CalDAV, CardDAV work correctly
go run ./cmd/server -config config.yaml
```
### Manual Verification
After migration, you should see:
```bash
$ ls -la data/alice/
calendars/
addressbooks/
files/
```
Each calendar should be in `data/<user>/calendars/<name>/` format, not `cal-<name>/`.
### Rollback (if needed)
If you need to rollback:
1. Stop the server
2. Restore the data directory from backup
3. Re-run the migration after fixing any issues
### Common Issues
**Q: Migration reports "directory already exists" warnings**
A: These are normal if files were already migrated. The migration is idempotent.
**Q: Old `files/` directory still exists**
A: This is expected. You can manually remove it after verifying migration success.
**Q: Some calendars/address books not visible after migration**
A: Check the migration logs and verify directory structure. Run `find data/ -type d -name "cal-*"` to find unmigrated calendars.
### Support
If you encounter issues:
1. Check logs for detailed error messages
2. Run migration with `--verbose` flag
3. Ensure no server processes are running during migration
4. Make backup before migrating in production
Makefile
+2 -2
View File
@@ -7,7 +7,7 @@ build:
## run: run the server locally
run: build
./bin/davserver -config config.yaml
./bin/davserver
## test: run all tests
test:
@@ -37,7 +37,7 @@ hash-password:
## nidusctl: build and run the sharing-grant admin CLI
## Usage: make nidusctl ARGS="calendar share alice work bob write"
nidusctl: build
./bin/nidusctl -config config.yaml $(ARGS)
./bin/nidusctl $(ARGS)
## templ-generate: regenerate *_templ.go files from internal/web/templates/*.templ
templ-generate:
README.md
+94 -84
View File
@@ -1,4 +1,4 @@
# DAV Server
# nidus
A self-hosted **CalDAV**, **CardDAV**, and **WebDAV** server written in Go.
@@ -23,56 +23,61 @@ and files, all kept under your own roof instead of a third-party cloud.
- **Web UI** — a mobile-friendly app at `/web/` for managing calendars,
contacts, files, and account settings (see [Web UI](#web-ui) below),
built with templ + Tailwind + htmx
- **ICSSubscriptions** — add remote ICS/webcal calendars
- **Birthdays calendar** — auto-computed from contacts' BDAY fields
- Auto-discovery via `/.well-known/caldav` and `/.well-known/carddav`
- Optional **TLS** (or use a reverse proxy)
- Structured logging (text or JSON)
- Graceful shutdown
- Docker & Docker Compose support
## Quick start
### 1. Install dependencies
```bash
go mod tidy
```
### 2. Create your `config.yaml`
Copy the example config and edit it — `config.yaml` is git-ignored so your
real settings never get committed:
```bash
cp config.example.yaml config.yaml
```
Users, calendars, and address books are **no longer configured in
`config.yaml`** — they live in the SQLite database and are managed with
`nidusctl` (see below).
### 3. Run the server
### 1. Run the server
```bash
make run
# or
go run ./cmd/server -config config.yaml
go run ./cmd/server
```
The server starts at **http://localhost:8080**.
### 4. Create a user and their resources
### 2. Configure the server via environment variables
The server is configured via environment variables:
```bash
go run ./tools/nidusctl -config config.yaml user create alice \
# Required: Set the data directory
export NIDUS_DATA_DIR="./data"
# Optional: Set port, host, and base URL
export NIDUS_PORT="8080"
export NIDUS_HOST="0.0.0.0"
export NIDUS_BASE_URL="https://dav.example.com"
# Optional: Set auth realm
export NIDUS_AUTH_REALM="My DAV Server"
# Optional: Set logging
export NIDUS_LOG_LEVEL="info"
export NIDUS_LOG_FORMAT="text"
```
This server does **not** handle TLS — use a reverse proxy (e.g. Nginx, Caddy,
Traefik) to terminate TLS and forward requests to the server.
### 3. Create a user and their resources
```bash
go run ./tools/nidusctl user create alice \
--display-name "Alice Smith" --email alice@example.com
# (prompts for a password; use --password to skip the prompt, e.g. in scripts)
go run ./tools/nidusctl -config config.yaml calendar create alice personal
go run ./tools/nidusctl -config config.yaml addressbook create alice contacts
go run ./tools/nidusctl calendar create alice personal
go run ./tools/nidusctl addressbook create alice contacts
```
Users can also be created/removed via the web UI (`/web/`) once logged in
as an existing user — see **Web UI** below.
Or use the web UI (`/web/`) once logged in — see **Web UI** below.
---
@@ -83,27 +88,27 @@ as an existing user — see **Web UI** below.
docker compose up --build
# Or build manually
docker build -t davserver .
docker build -t nidus .
docker run -p 8080:8080 \
-v ./config.yaml:/app/config.yaml:ro \
-v dav-data:/app/data \
davserver
-v nidus-data:/app/data \
-e NIDUS_DATA_DIR=/app/data \
nidus
```
The image also ships `nidusctl`, so once the container is running you can
create your first user (and their calendars/address books) with
`docker compose exec` — no need to install Go locally:
`docker compose exec`:
```bash
docker compose exec davserver nidusctl -config /app/config.yaml user create alice \
docker compose exec nidus nidusctl user create alice \
--display-name "Alice Smith" --email alice@example.com
# (prompts for a password; use --password to skip the prompt, e.g. in scripts)
# (prompts for a password; use --password to skip the prompt)
docker compose exec davserver nidusctl -config /app/config.yaml calendar create alice personal
docker compose exec davserver nidusctl -config /app/config.yaml addressbook create alice contacts
docker compose exec nidus nidusctl calendar create alice personal
docker compose exec nidus nidusctl addressbook create alice contacts
```
### Pre-built images
### Using pre-built images
Pushing a version tag (e.g. `v1.2.3`) or publishing a release triggers
[`.github/workflows/docker-release.yml`](.github/workflows/docker-release.yml),
@@ -118,25 +123,29 @@ building from a checkout — just fetch `config.example.yaml`, copy it to
```yaml
services:
davserver:
nidus:
image: git.arnef.de/arnef/nidus:latest
ports:
- "8080:8080"
volumes:
- ./config.yaml:/app/config.yaml:ro
- dav-data:/app/data
- nidus-data:/app/data
environment:
- NIDUS_DATA_DIR=/app/data
# Optional: other environment variables
# - NIDUS_PORT=8080
# - NIDUS_HOST=0.0.0.0
# - NIDUS_BASE_URL=https://dav.example.com
# - NIDUS_AUTH_REALM="My DAV Server"
# - NIDUS_LOG_LEVEL=info
# - NIDUS_LOG_FORMAT=text
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 5s
retries: 3
volumes:
dav-data:
nidus-data:
```
---
Note: This server does **not** handle TLS. Use a reverse proxy (e.g. Nginx,
Caddy, Traefik) to terminate TLS and forward requests to the server.
## API endpoints
@@ -208,10 +217,10 @@ mobile browsers:
`nidus.db` (`web_sessions` table), independent of DAV Basic Auth. Includes
a "Show/Hide" password toggle to rule out typos before submitting.
- **Dashboard** (`/web/`) — create/delete your own calendars, address
books, and ICS/webcal subscriptions; see who your resources are shared
with and what others have shared with you; manage sharing grants
directly (same effect as `nidusctl`) — updates happen in place via
[htmx](https://htmx.org/) without a full page reload.
books, ICS/webcal subscriptions, and the Birthdays calendar; see who your
resources are shared with and what others have shared with you; manage
sharing grants directly (same effect as `nidusctl`) — updates happen in
place via [htmx](https://htmx.org/) without a full page reload.
- **Files** (`/web/files/`) — a browser for the same storage the WebDAV
endpoint (`/files/`) serves: navigate folders, create new folders,
upload files/folders (including via drag & drop), download, and delete
@@ -223,8 +232,9 @@ mobile browsers:
delete contacts (name, organization, birthday, phone numbers, emails,
addresses, photo), and import/export vCards (`.vcf`).
- **Calendar** (`/web/calendar`) — month and week views across all your
own and shared calendars, create/edit/delete events, per-calendar
colors, and import/export `.ics` files.
own and shared calendars (including ICS/webcal subscriptions and the
Birthdays calendar), create/edit/delete events, per-calendar colors,
and import/export `.ics` files.
- **Account** (`/web/account`) — update your display name/email and
change your password.
- **Logout** (`/web/logout`).
@@ -248,35 +258,34 @@ make web-assets # regenerate templ code + rebuild web/static/app.css and web/st
## Configuration reference
```yaml
server:
host: "0.0.0.0"
port: 8080
base_url: "https://dav.example.com" # used in DAV responses
Configuration is done via environment variables:
auth:
realm: "My DAV Server"
| Variable | Default | Description |
|----------|---------|-------------|
| `NIDUS_HOST` | `0.0.0.0` | Server listen host |
| `NIDUS_PORT` | `8080` | Server listen port |
| `NIDUS_BASE_URL` | (auto) | Public URL for DAV responses (e.g. https://dav.example.com) |
| `NIDUS_AUTH_REALM` | `DAV Server` | HTTP Basic Auth realm |
| `NIDUS_DATA_DIR` | `./data` | Data directory for all user data |
| `NIDUS_LOG_LEVEL` | `info` | Log level: debug, info, warn, error |
| `NIDUS_LOG_FORMAT` | `text` | Log format: text, json |
storage:
data_dir: "./data" # all user data lives here
Example:
logging:
level: "info" # debug | info | warn | error
format: "text" # text | json
tls:
enabled: false
cert_file: ""
key_file: ""
```bash
export NIDUS_DATA_DIR="./data"
export NIDUS_PORT="8080"
export NIDUS_BASE_URL="https://dav.example.com"
export NIDUS_LOG_LEVEL="info"
```
Users, calendars, and address books are managed via `nidusctl`, not
`config.yaml` — see **Managing users** below.
This server does **not** handle TLS — use a reverse proxy (e.g. Nginx, Caddy,
Traefik) to terminate TLS and forward requests to the server.
## Managing users
All user/calendar/address-book management is done with `nidusctl` (or the
web UI). Nothing is stored in `config.yaml` anymore.
All user/calendar/address-book management is done with `nidusctl` or the
web UI (`/web/`). Nothing is stored in `config.yaml` anymore.
```bash
# Users
@@ -303,9 +312,10 @@ nidusctl addressbook unshare <owner> <book> <user>
nidusctl addressbook shares <owner> <book>
```
Passwords are prompted for interactively (masked, double-entry) when
Password are prompted for interactively (masked, double-entry) when
`--password` is omitted. The web UI (`/web/`) also lets a logged-in user
create/delete their own calendars and address books from the dashboard.
create/delete their own calendars, address books, and ICS/webcal subscriptions
from the dashboard.
> **Upgrading from an older version?** The `users:` section in
> `config.yaml` is no longer read. Recreate your users with
@@ -317,22 +327,22 @@ create/delete their own calendars and address books from the dashboard.
## Project layout
```
caldav-server/
nidus/
├── cmd/server/ # main entrypoint
├── internal/
│ ├── auth/ # HTTP Basic Auth middleware (DAV endpoints)
│ ├── caldav/ # CalDAV backend
│ ├── carddav/ # CardDAV backend
│ ├── config/ # YAML config loader
│ ├── db/ # SQLite store (shares, web UI sessions)
│ ├── db/ # SQLite store (users, calendars, shares, sessions)
│ ├── store/ # filesystem storage layer
│ ├── web/ # web UI (cookie sessions, dashboard, share mgmt)
│ │ └── templates/ # templ templates (+ generated *_templ.go)
│ └── webdav/ # WebDAV file handler
├── tools/hashpwd/ # bcrypt password hasher CLI
├── tools/nidusctl/ # sharing-grant admin CLI
├── internal/web/ # web UI (templ, dashboard, share mgmt, sessions)
│ └── templates/ # templ templates (+ generated *_templ.go)
├── cmd/nidusctl/ # admin CLI (users, calendars, address books, sharing)
├── web/ # front-end assets: Tailwind input/config, static/
│ └── static/ # compiled app.css + htmx.min.js (embedded into the binary)
├── tools/migrate/ # data directory migration tool
├── config.example.yaml # sample configuration (copy to config.yaml)
├── Dockerfile
├── docker-compose.yaml
cmd/server/main.go
+3 -13
View File
@@ -2,7 +2,6 @@ package main
import (
"context"
"flag"
"fmt"
"log/slog"
"net"
@@ -25,12 +24,8 @@ import (
)
func main() {
var cfgPath string
flag.StringVar(&cfgPath, "config", "config.yaml", "path to configuration file")
flag.Parse()
// ---- Configuration ----
cfg, err := config.Load(cfgPath)
cfg, err := config.Load()
if err != nil {
fmt.Fprintf(os.Stderr, "error loading config: %v\n", err)
os.Exit(1)
@@ -41,8 +36,7 @@ func main() {
logger.Info("starting DAV server",
"host", cfg.Server.Host,
"port", cfg.Server.Port,
"base_url", cfg.Server.BaseURL,
"tls", cfg.TLS.Enabled)
"base_url", cfg.Server.BaseURL)
// ---- Storage ----
st, err := store.NewStore(cfg.Storage.DataDir)
@@ -143,11 +137,7 @@ func main() {
}()
logger.Info("server ready", "addr", addr)
if cfg.TLS.Enabled {
err = srv.ListenAndServeTLS(cfg.TLS.CertFile, cfg.TLS.KeyFile)
} else {
err = srv.ListenAndServe()
}
err = srv.ListenAndServe()
if err != nil && err != http.ErrServerClosed {
logger.Error("server error", "error", err)
os.Exit(1)
config.example.yaml
+28 -29
View File
@@ -1,31 +1,30 @@
server:
host: "0.0.0.0"
port: 8080
# Set this to your public-facing URL so discovery responses are correct.
# base_url: "https://dav.example.com"
auth:
realm: "My DAV Server"
storage:
data_dir: "./data"
logging:
level: "debug" # debug | info | warn | error
format: "text" # text | json
tls:
enabled: false
# cert_file: "/etc/ssl/certs/dav.crt"
# key_file: "/etc/ssl/private/dav.key"
# Users, calendars, and address books are no longer configured here — they
# are stored in the database (<data_dir>/nidus.db) and managed with
# nidusctl or the web UI (/web/):
# nidus configuration via environment variables
#
# nidusctl user create alice --password mysecretpassword --display-name "Alice Smith" --email alice@example.com
# nidusctl calendar create alice personal
# nidusctl calendar create alice work
# nidusctl addressbook create alice contacts
# Environment variables override any defaults shown below.
# Set them before starting the server.
#
# Run `nidusctl help` for the full command list.
# Server configuration:
# NIDUS_HOST - listen host (default: 0.0.0.0)
# NIDUS_PORT - listen port (default: 8080)
# NIDUS_BASE_URL - public URL for DAV responses (default: auto-generated from host:port)
#
# Authentication:
# NIDUS_AUTH_REALM - HTTP Basic Auth realm (default: "DAV Server")
#
# Storage:
# NIDUS_DATA_DIR - data directory (default: ./data)
#
# Logging:
# NIDUS_LOG_LEVEL - log level: debug, info, warn, error (default: info)
# NIDUS_LOG_FORMAT - log format: text, json (default: text)
# Example (uncomment and modify as needed):
# export NIDUS_HOST="0.0.0.0"
# export NIDUS_PORT="8080"
# export NIDUS_BASE_URL="https://dav.example.com"
# export NIDUS_DATA_DIR="./data"
# export NIDUS_LOG_LEVEL="info"
# export NIDUS_LOG_FORMAT="text"
# Note: The server does not handle TLS - use a reverse proxy (e.g. Nginx, Caddy,
# Traefik) to terminate TLS and forward requests to the server.
docker-compose.yaml
+12 -4
View File
@@ -1,11 +1,19 @@
services:
davserver:
nidus:
build: .
ports:
- "8080:8080"
environment:
# Optional: other environment variables
# - NIDUS_DATA_DIR=/app/data
# - NIDUS_HOST=0.0.0.0
# - NIDUS_PORT=8080
# - NIDUS_BASE_URL=https://dav.example.com
# - NIDUS_AUTH_REALM="My DAV Server"
# - NIDUS_LOG_LEVEL=info
# - NIDUS_LOG_FORMAT=text
volumes:
- ./config.yaml:/app/config.yaml:ro
- dav-data:/app/data
- nidus-data:/app/data
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
@@ -14,4 +22,4 @@ services:
retries: 3
volumes:
dav-data:
nidus-data: