- Add web-deps, nidusctl, and migrate commands to build section - Document ICS/webcal subscriptions and Birthdays calendar - Note Migrate() is idempotent - Add additional database tables - Include Docker/CI and database migration details
240 lines
16 KiB
Markdown
240 lines
16 KiB
Markdown
# Copilot Instructions for nidus
|
|
|
|
A self-hosted CalDAV, CardDAV, and WebDAV server written in Go, backed by a
|
|
filesystem store, with calendar/address book sharing grants tracked in a
|
|
small SQLite database. HTTP Basic Auth (bcrypt) with per-user isolated
|
|
collections. A small server-rendered web UI (templ + Tailwind + htmx) at
|
|
`/web/` lets users log in and manage their shares.
|
|
|
|
## Build, test, lint
|
|
|
|
```bash
|
|
make build # go build -o bin/davserver ./cmd/server + ./bin/nidusctl
|
|
make run # build + ./bin/davserver -config config.yaml
|
|
make test # go test ./... -v -race
|
|
make lint # golangci-lint run ./... (no config file; uses defaults)
|
|
make tidy # go mod tidy
|
|
make web-deps # install Tailwind CLI + TypeScript (needed once, or after web/package.json changes)
|
|
make templ-generate # regenerate *_templ.go after editing internal/web/templates/*.templ
|
|
make web-css # templ-generate + rebuild web/static/app.css
|
|
make web-ts # compile web/ts/*.ts to web/static/*.js
|
|
make web-assets # web-css + web-ts (everything under web/static/)
|
|
make nidusctl # build + run nidusctl with ARGS (e.g. `make nidusctl ARGS="user list"`)
|
|
go run ./tools/migrate -config config.yaml # restructure data directory
|
|
```
|
|
|
|
Test files: `internal/store/store_test.go`, `internal/webdav/handler_test.go`,
|
|
`internal/db/shares_test.go`, `internal/caldav/backend_test.go`,
|
|
`internal/carddav/backend_test.go`, `internal/web/server_test.go`,
|
|
`tools/nidusctl/main_test.go`.
|
|
|
|
## Architecture
|
|
|
|
- `cmd/server/main.go` — entrypoint. Loads config, builds the `slog.Logger`,
|
|
constructs the `store.Store` and `db.DB`, lists all users/calendars/
|
|
address-books from the DB (`dbase.ListUsers`/`ListCalendars`/
|
|
`ListAddressBooks`) to pre-create their collections on disk, warns if
|
|
zero users exist (`nidusctl user create ...`), wires up `auth.Middleware`,
|
|
and builds the `http.ServeMux` (`buildMux`). Routes: `/cal/`, `/card/`,
|
|
`/files/`, `/.well-known/{caldav,carddav}`, `/healthz` (unauthenticated),
|
|
and `/` (welcome page on GET/HEAD only; any other method — e.g. a WebDAV
|
|
client pointed at the wrong URL — gets `405` instead of a misleading
|
|
`200`).
|
|
- `internal/config` — YAML config loading (`config.Load`), defaults
|
|
(`applyDefaults`), and validation (`validate`). Holds only server/auth/
|
|
storage/logging/TLS settings — **no user, calendar, or address-book
|
|
data**; all of that lives in `internal/db` now (see below).
|
|
- `internal/auth` — HTTP Basic Auth middleware (`auth.Middleware.Wrap`).
|
|
Takes a `*db.DB` and validates credentials via `dbase.GetUser` +
|
|
`dbase.VerifyPassword` (bcrypt), then stores a
|
|
`*Principal{Username, DisplayName, Email}` in the request context.
|
|
Downstream code retrieves it with `auth.FromContext(ctx)` — every backend
|
|
method needs this and returns `webdav.NewHTTPError(http.StatusUnauthorized, ...)`
|
|
if it's nil. `auth.NewContext(ctx, p)` is the test-only inverse, used to
|
|
build authenticated contexts without a real Basic Auth handshake.
|
|
- `internal/store` — the single source of truth for all persisted data.
|
|
A thin filesystem KV abstraction: `<data_dir>/<user>/[calendars|addressbooks|files]/<name>`.
|
|
All collection/object names pass through `sanitize()` (via
|
|
`filepath.Base` + strip `..`) to prevent path traversal — preserve this
|
|
when adding new store methods. Writes use temp-file + rename for atomicity
|
|
(`PutObject`). Locking is sharded per-user (`lockFor(user)`, a
|
|
`map[string]*sync.RWMutex` guarded by its own mutex) rather than one
|
|
global lock, so different users' requests don't serialize against each
|
|
other. `Migrate()` restructures data from the old format (`cal-<name>`,
|
|
`card-<name>`) to the new unified layout and is idempotent.
|
|
- `internal/caldav` and `internal/carddav` — implement the
|
|
`caldav.Backend`/`carddav.Backend` interfaces from `github.com/emersion/go-webdav`
|
|
on top of `store.Store`. Calendars are stored as collections prefixed
|
|
`cal-<name>` and address books as `card-<name>` (see `ListCalendars`,
|
|
`parseCalPath`). **The URL scheme has no username segment**, but DOES
|
|
have a fixed literal `home` segment: `/cal/` (principal), `/cal/home/`
|
|
(calendar-home-set), `/cal/home/<calname>/` (calendar),
|
|
`/cal/home/<calname>/<objid>` (object) — and equivalently
|
|
`/card/`, `/card/home/`, `/card/home/<bookname>/`,
|
|
`/card/home/<bookname>/<objid>` for carddav. These are identical for
|
|
every user; the acting user always comes from `auth.FromContext(ctx)`,
|
|
never from the path. **The `home` segment is load-bearing, not
|
|
cosmetic**: go-webdav's `caldav`/`carddav` server (in the
|
|
`github.com/emersion/go-webdav` dependency, not our code) classifies
|
|
each request purely by counting URL path segments relative to the
|
|
handler's `Prefix` (which we leave `""`) — 1 segment = user principal, 2
|
|
= home-set, 3 = calendar/address book, 4 = object. If the segment counts
|
|
don't line up (e.g. removing `home` would make `/cal/` and
|
|
`/cal/<calname>/` collapse to 1 and 2 segments, misclassifying the
|
|
calendar collection itself as the home-set), PROPFIND requests silently
|
|
return an empty `<multistatus>` (200/207, zero `<response>` elements) —
|
|
no error, just nothing found, which breaks client auto-discovery (e.g.
|
|
DAVx5 reporting "no resources found"). Keep this in mind when touching
|
|
`parseCalPath`/`parseObjPath`/`parseBookPath`,
|
|
`calHomePath`/`cardHomePath`, or `CurrentUserPrincipal` — always
|
|
preserve the exact segment depth at each level. Query methods
|
|
(`QueryCalendarObjects`) currently list all objects and filter in-memory
|
|
via `caldav.Filter` — fine for small collections, not optimized for
|
|
scale.
|
|
**Sharing**: both backends take an `*db.DB` (required — used for both
|
|
the base `ListCalendars`/`ListAddressBooks`/`Create*`/`Delete*`
|
|
operations and sharing). A calendar/address book shared with a user is exposed under
|
|
the synthetic local name `<owner>~<name>` (see `sharedNameSep`,
|
|
`sharedCalendarName`/`sharedBookName`) in that user's own home-set —
|
|
`resolveCalendar`/`resolveBook` split the local name back into
|
|
owner+real name and check the grant's permission (`db.PermRead`/
|
|
`db.PermWrite`) via `dbase.CalendarShareFor`/`AddressBookShareFor`
|
|
before allowing reads (any share) or writes (write share only). The
|
|
shared data is never copied — it's read/written directly under the
|
|
owner's own `store.Store` namespace, just addressed via the synthetic
|
|
name from the grantee's requests.
|
|
**ICSSubscriptions**: read-only calendars backed by remote ICS/webcal
|
|
URLs (see `db.CreateICSSubscription`, `internal/db/ics.go`) are exposed
|
|
alongside real calendars under `/cal/home/<name>/` (shared namespace;
|
|
see `internal/caldav/ics.go`). `Birthdays` is a computed calendar
|
|
synthesized from contacts' BDAY fields (see `internal/caldav/birthdays.go`,
|
|
`internal/birthdays/birthdays.go`).
|
|
- `internal/db` — a small `database/sql` wrapper around
|
|
`modernc.org/sqlite` (pure Go, no CGO) at `<data_dir>/nidus.db`. Only
|
|
one open connection is used (`SetMaxOpenConns(1)`) since SQLite allows a
|
|
single writer; this is intentionally simple and not meant to scale to
|
|
heavy concurrent write load. Schema lives in `migrate()`; there's no
|
|
migration framework, just idempotent `CREATE TABLE IF NOT EXISTS`.
|
|
Foreign keys are enabled per-connection (`?_pragma=foreign_keys(1)` in
|
|
the DSN). **This is now the single source of truth for users,
|
|
calendars, and address books** (`internal/db/users.go`):
|
|
`CreateUser`/`SetPassword`/`DeleteUser`/`GetUser`/`VerifyPassword`/
|
|
`ListUsers`, and `CreateCalendar`/`DeleteCalendar`/`ListCalendars`,
|
|
`CreateAddressBook`/`DeleteAddressBook`/`ListAddressBooks`. `users` is
|
|
the parent table; `calendars`/`addressbooks` cascade-delete via FK on
|
|
`DeleteUser`; `calendar_shares`/`addressbook_shares`/`web_sessions`
|
|
reference usernames as plain strings (no FK) so `DeleteUser` explicitly
|
|
cleans those up in a transaction. `modernc.org/sqlite` has no typed
|
|
unique-constraint error, so `isUniqueConstraintErr()` string-matches the
|
|
driver's error message.
|
|
**Additional tables**: `birthday_calendars` (per-user display color for
|
|
the computed Birthdays calendar), `ics_subscriptions` (remote ICS/webcal
|
|
calendar subscriptions), `web_sessions` (server-side session tokens).
|
|
- `internal/webdav` — plain-file WebDAV via `golang.org/x/net/webdav`,
|
|
mounted at the single fixed URL `/files/` for all users (no username in
|
|
the path either). `NewHandler` caches one `*xwebdav.Handler` per
|
|
authenticated username (keyed off `auth.FromContext`), each rooted at
|
|
`<data_dir>/files/<username>/` on disk with its own persistent
|
|
`LockSystem` — the handler (and its lock table) must be created once and
|
|
reused, not per-request, or LOCK/UNLOCK state resets on every call.
|
|
- `tools/hashpwd` — standalone CLI (`go run ./tools/hashpwd <password>`) to
|
|
generate bcrypt hashes for ad-hoc testing (no longer needed for normal
|
|
user setup — see `nidusctl user create` below).
|
|
- `tools/migrate` — restructures data directory from old format (`cal-<name>`,
|
|
`card-<name>`) to unified layout (`calendars/`, `addressbooks/`,
|
|
`files/`). Run automatically on server startup; invoke manually with
|
|
`go run ./tools/migrate -config config.yaml`.
|
|
- `tools/nidusctl` — standalone admin CLI (`go run ./tools/nidusctl
|
|
-config config.yaml <user|calendar|addressbook> ...`) — the only way to
|
|
create/delete users, calendars, and address books, plus manage sharing
|
|
grants (`calendar|addressbook share|unshare|shares`). It's a thin
|
|
argv-parsing wrapper around `db.DB`'s methods (`tools/nidusctl/main.go`
|
|
routes subcommands, `tools/nidusctl/users.go` implements `user
|
|
create/delete/list/passwd` with interactive masked password prompting
|
|
via `golang.org/x/term`, falling back to a plain stdin read when not a
|
|
TTY) — no server interaction, no daemon, no RPC; it just opens the same
|
|
SQLite file the running server uses. Changes take effect immediately
|
|
without restarting the server since nothing is cached.
|
|
- `internal/web` — the web UI, mounted at `/web/` in `cmd/server/main.go`
|
|
(`mux.Handle("/web/", http.StripPrefix("/web", web.NewServer(cfg, st,
|
|
dbase, logger).Handler(webstatic.FS())))`, so `Server.Handler`'s own
|
|
routes are all unprefixed — `/login`, `/`, `/shares/...` — and only the
|
|
outer mux adds the `/web` prefix), entirely separate from `internal/auth`'s
|
|
Basic Auth: logins go through
|
|
`/web/login` (username/password checked against the DB the same way
|
|
Basic Auth does, via `dbase.VerifyPassword`) and issue an opaque random session token
|
|
stored in the `web_sessions` SQLite table (`db.CreateSession`/
|
|
`SessionUser`/`DeleteSession`, see `internal/db/sessions.go`), set as an
|
|
`HttpOnly` cookie (`sessionCookieName` in `internal/web/session.go`).
|
|
`requireLogin` is the auth-guard middleware for authenticated routes,
|
|
storing the username in the request context (`userFromContext`).
|
|
`internal/web/dashboard.go` renders the logged-in user's own
|
|
calendars/address books plus who they're shared with
|
|
(`SharesOfCalendar`/`SharesOfAddressBook`) and what's shared with them
|
|
(`CalendarsSharedWith`/`AddressBooksSharedWith`); `resourceCards(username)`
|
|
is the shared helper (also used by `internal/web/resources.go`, see
|
|
below) that builds the list of `templates.ResourceCard`s from the DB.
|
|
`internal/web/resources.go` handles POST (create) and DELETE (delete) at
|
|
`/web/resources/{calendar,addressbook}`, validating names against
|
|
`resourceNameRe` (`^[a-zA-Z0-9_-]{1,64}$`) and re-rendering the whole
|
|
`#resources` list (`templates.ResourceList`) since the *set* of cards
|
|
changes (unlike a share update, which only touches one card).
|
|
`internal/web/shares.go`
|
|
handles POST (create/update share) and DELETE (revoke) at
|
|
`/web/shares/{calendar,addressbook}`, re-rendering just the affected
|
|
resource card for htmx's `hx-swap="outerHTML"`; it always checks
|
|
`ownsResource` first so a user can only share resources actually
|
|
configured for their own account (never someone else's, even via a
|
|
forged form post). **htmx v2 quirk**: `hx-delete` requests send
|
|
`hx-vals`/form params as URL **query string** parameters, not a request
|
|
body (unlike POST/PUT/PATCH) — `handleShare` special-cases
|
|
`r.Method == http.MethodDelete` to read from `r.URL.Query()` instead of
|
|
calling `r.ParseForm()`. Templates live in `internal/web/templates/*.templ`
|
|
(compiled to `*_templ.go` via `templ generate`/`make templ-generate` —
|
|
regenerate after editing any `.templ` file, the generated files are
|
|
committed). Styling is Tailwind v4, scanned directly over the generated
|
|
`_templ.go` files (`web/input.css`'s `@source` directives) and compiled
|
|
to `web/static/app.css` via `make web-css` (needs Node/npm — see
|
|
`web/package.json`); htmx itself is vendored as a static file
|
|
(`web/static/htmx.min.js`, not npm-installed) to avoid a CDN dependency.
|
|
Client-side-only logic (currently just the login page's password-visibility
|
|
toggle) is written in TypeScript under `web/ts/*.ts`, compiled to plain
|
|
JS via `tsc` (`web/tsconfig.json`, `make web-ts`) into `web/static/*.js`
|
|
as ES modules (`<script type="module">`). All static assets (CSS, JS,
|
|
vendored htmx) are embedded into the Go binary at build time via
|
|
`web/staticassets.go` (`//go:embed static`), so the compiled server has
|
|
no runtime dependency on Node.js or the `web/` directory being present —
|
|
Node/npm are only needed when actually changing templates/styles/TS
|
|
(`make web-assets` rebuilds everything under `web/static/`).
|
|
**Web UI features**: dashboard (calendars/address books/ICS subscriptions
|
|
management), login (cookie-based sessions), share management (create/update/
|
|
revoke share grants via htmx), files browser (inline preview, upload, download),
|
|
contacts manager (vCard import/export, edit fields), calendar view (month/week,
|
|
per-calendar colors), account settings (name/email/password). The
|
|
synthetic `Birthdays` calendar (computed from contacts' BDAY fields) and
|
|
ICS/webcal subscriptions are managed via the web UI just like real
|
|
calendars.
|
|
|
|
## Conventions
|
|
|
|
- **No username in any DAV URL** (`/cal/`, `/card/`, `/files/` are the same
|
|
for every account) — the acting user is always resolved from the Basic
|
|
Auth identity (`auth.FromContext`), never parsed out of the request path.
|
|
Don't reintroduce a `<user>` path segment when adding routes/paths.
|
|
- Path parsing in the CalDAV/CardDAV backends assumes fixed URL segment
|
|
positions (e.g. `cal/home/<calname>/<objid>`) split on `/` — see
|
|
`parseCalPath`/`parseObjPath`/`parseBookPath`. The `home` segment must
|
|
stay exactly one fixed literal segment (see note above on go-webdav's
|
|
segment-count-based resource classification) — don't remove it or add/
|
|
remove segments elsewhere without re-checking all four resource-type
|
|
depths still line up. New path-based operations should follow the same
|
|
segment-index approach for consistency.
|
|
- Store errors are sentinel values (`store.ErrNotFound`, `store.ErrConflict`)
|
|
checked with `errors.Is`/direct comparison; backends translate them into
|
|
`webdav.NewHTTPError` with the appropriate HTTP status.
|
|
- Logging uses `log/slog` structured fields (e.g. `logger.Warn("...", "user", u, "error", err)`), passed down explicitly to every constructor (`NewBackend`, `NewHandler`, `NewMiddleware`) rather than a global logger.
|
|
- Config module path is `github.com/yourusername/caldav-server` (go.mod name
|
|
predates the `nidus` repo rename) — import paths still use this, not `nidus`.
|
|
- **Docker/CI**: Docker image pushed to `git.arnef.de/arnef/nidus` on tags (`v*`) or releases (`.github/workflows/docker-release.yml`), multi-arch (`linux/amd64`, `linux/arm64`). Use `docker-compose` for local dev with healthcheck on `/healthz`.
|
|
- **Database schema** lives in `internal/db/db.go` `migrate()` — all tables are created with `CREATE TABLE IF NOT EXISTS` and schema changes are applied via `ALTER TABLE` in `migrateAddColumns()`. No external migration framework.
|