Compare commits
4
Commits
v1.0.0
..
8d2d8d3972
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8d2d8d3972 | ||
|
|
2fd39c8180 | ||
|
|
f463c01f0f | ||
|
|
cd0761c9d1 |
@@ -0,0 +1,213 @@
|
|||||||
|
# Copilot Instructions for nidus
|
||||||
|
|
||||||
|
A self-hosted CalDAV, CardDAV, and WebDAV server written in Go, backed by a
|
||||||
|
filesystem store, with calendar/address book sharing grants tracked in a
|
||||||
|
small SQLite database. HTTP Basic Auth (bcrypt) with per-user isolated
|
||||||
|
collections. A small server-rendered web UI (templ + Tailwind + htmx) at
|
||||||
|
`/web/` lets users log in and manage their shares.
|
||||||
|
|
||||||
|
## Build, test, lint
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make build # go build -o bin/davserver ./cmd/server
|
||||||
|
make run # build + ./bin/davserver -config config.yaml
|
||||||
|
make test # go test ./... -v -race
|
||||||
|
make lint # golangci-lint run ./...
|
||||||
|
make tidy # go mod tidy
|
||||||
|
go test ./internal/store/ -run TestStoreRoundTrip -v # single test
|
||||||
|
make templ-generate # regenerate *_templ.go after editing internal/web/templates/*.templ
|
||||||
|
make web-css # templ-generate + rebuild web/static/app.css (needs `make web-deps` once, Node.js/npm)
|
||||||
|
make web-ts # compile web/ts/*.ts to web/static/*.js
|
||||||
|
make web-assets # web-css + web-ts (everything under web/static/)
|
||||||
|
```
|
||||||
|
|
||||||
|
Test files: `internal/store/store_test.go`, `internal/webdav/handler_test.go`,
|
||||||
|
`internal/db/shares_test.go`, `internal/caldav/backend_test.go`,
|
||||||
|
`internal/carddav/backend_test.go`, `internal/web/server_test.go`,
|
||||||
|
`tools/nidusctl/main_test.go`.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- `cmd/server/main.go` — entrypoint. Loads config, builds the `slog.Logger`,
|
||||||
|
constructs the `store.Store` and `db.DB`, lists all users/calendars/
|
||||||
|
address-books from the DB (`dbase.ListUsers`/`ListCalendars`/
|
||||||
|
`ListAddressBooks`) to pre-create their collections on disk, warns if
|
||||||
|
zero users exist (`nidusctl user create ...`), wires up `auth.Middleware`,
|
||||||
|
and builds the `http.ServeMux` (`buildMux`). Routes: `/cal/`, `/card/`,
|
||||||
|
`/files/`, `/.well-known/{caldav,carddav}`, `/healthz` (unauthenticated),
|
||||||
|
and `/` (welcome page on GET/HEAD only; any other method — e.g. a WebDAV
|
||||||
|
client pointed at the wrong URL — gets `405` instead of a misleading
|
||||||
|
`200`).
|
||||||
|
- `internal/config` — YAML config loading (`config.Load`), defaults
|
||||||
|
(`applyDefaults`), and validation (`validate`). Holds only server/auth/
|
||||||
|
storage/logging/TLS settings — **no user, calendar, or address-book
|
||||||
|
data**; all of that lives in `internal/db` now (see below).
|
||||||
|
- `internal/auth` — HTTP Basic Auth middleware (`auth.Middleware.Wrap`).
|
||||||
|
Takes a `*db.DB` and validates credentials via `dbase.GetUser` +
|
||||||
|
`dbase.VerifyPassword` (bcrypt), then stores a
|
||||||
|
`*Principal{Username, DisplayName, Email}` in the request context.
|
||||||
|
Downstream code retrieves it with `auth.FromContext(ctx)` — every backend
|
||||||
|
method needs this and returns `webdav.NewHTTPError(http.StatusUnauthorized, ...)`
|
||||||
|
if it's nil. `auth.NewContext(ctx, p)` is the test-only inverse, used to
|
||||||
|
build authenticated contexts without a real Basic Auth handshake.
|
||||||
|
- `internal/store` — the single source of truth for all persisted data.
|
||||||
|
A thin filesystem KV abstraction: `<data_dir>/<user>/<collection>/<objectID>`.
|
||||||
|
All collection/object names pass through `sanitize()` (via
|
||||||
|
`filepath.Base` + strip `..`) to prevent path traversal — preserve this
|
||||||
|
when adding new store methods. Writes use temp-file + rename for atomicity
|
||||||
|
(`PutObject`). Locking is sharded per-user (`lockFor(user)`, a
|
||||||
|
`map[string]*sync.RWMutex` guarded by its own mutex) rather than one
|
||||||
|
global lock, so different users' requests don't serialize against each
|
||||||
|
other.
|
||||||
|
- `internal/caldav` and `internal/carddav` — implement the
|
||||||
|
`caldav.Backend`/`carddav.Backend` interfaces from `github.com/emersion/go-webdav`
|
||||||
|
on top of `store.Store`. Calendars are stored as collections prefixed
|
||||||
|
`cal-<name>` and address books as `card-<name>` (see `ListCalendars`,
|
||||||
|
`parseCalPath`). **The URL scheme has no username segment**, but DOES
|
||||||
|
have a fixed literal `home` segment: `/cal/` (principal), `/cal/home/`
|
||||||
|
(calendar-home-set), `/cal/home/<calname>/` (calendar),
|
||||||
|
`/cal/home/<calname>/<objid>` (object) — and equivalently
|
||||||
|
`/card/`, `/card/home/`, `/card/home/<bookname>/`,
|
||||||
|
`/card/home/<bookname>/<objid>` for carddav. These are identical for
|
||||||
|
every user; the acting user always comes from `auth.FromContext(ctx)`,
|
||||||
|
never from the path. **The `home` segment is load-bearing, not
|
||||||
|
cosmetic**: go-webdav's `caldav`/`carddav` server (in the
|
||||||
|
`github.com/emersion/go-webdav` dependency, not our code) classifies
|
||||||
|
each request purely by counting URL path segments relative to the
|
||||||
|
handler's `Prefix` (which we leave `""`) — 1 segment = user principal, 2
|
||||||
|
= home-set, 3 = calendar/address book, 4 = object. If the segment counts
|
||||||
|
don't line up (e.g. removing `home` would make `/cal/` and
|
||||||
|
`/cal/<calname>/` collapse to 1 and 2 segments, misclassifying the
|
||||||
|
calendar collection itself as the home-set), PROPFIND requests silently
|
||||||
|
return an empty `<multistatus>` (200/207, zero `<response>` elements) —
|
||||||
|
no error, just nothing found, which breaks client auto-discovery (e.g.
|
||||||
|
DAVx5 reporting "no resources found"). Keep this in mind when touching
|
||||||
|
`parseCalPath`/`parseObjPath`/`parseBookPath`,
|
||||||
|
`calHomePath`/`cardHomePath`, or `CurrentUserPrincipal` — always
|
||||||
|
preserve the exact segment depth at each level. Query methods
|
||||||
|
(`QueryCalendarObjects`) currently list all objects and filter in-memory
|
||||||
|
via `caldav.Filter` — fine for small collections, not optimized for
|
||||||
|
scale.
|
||||||
|
**Sharing**: both backends take an `*db.DB` (required — used for both
|
||||||
|
the base `ListCalendars`/`ListAddressBooks`/`Create*`/`Delete*`
|
||||||
|
operations and sharing). A calendar/address book shared with a user is exposed under
|
||||||
|
the synthetic local name `<owner>~<name>` (see `sharedNameSep`,
|
||||||
|
`sharedCalendarName`/`sharedBookName`) in that user's own home-set —
|
||||||
|
`resolveCalendar`/`resolveBook` split the local name back into
|
||||||
|
owner+real name and check the grant's permission (`db.PermRead`/
|
||||||
|
`db.PermWrite`) via `dbase.CalendarShareFor`/`AddressBookShareFor`
|
||||||
|
before allowing reads (any share) or writes (write share only). The
|
||||||
|
shared data is never copied — it's read/written directly under the
|
||||||
|
owner's own `store.Store` namespace, just addressed via the synthetic
|
||||||
|
name from the grantee's requests.
|
||||||
|
- `internal/db` — a small `database/sql` wrapper around
|
||||||
|
`modernc.org/sqlite` (pure Go, no CGO) at `<data_dir>/nidus.db`. Only
|
||||||
|
one open connection is used (`SetMaxOpenConns(1)`) since SQLite allows a
|
||||||
|
single writer; this is intentionally simple and not meant to scale to
|
||||||
|
heavy concurrent write load. Schema lives in `migrate()`; there's no
|
||||||
|
migration framework, just idempotent `CREATE TABLE IF NOT EXISTS`.
|
||||||
|
Foreign keys are enabled per-connection (`?_pragma=foreign_keys(1)` in
|
||||||
|
the DSN). **This is now the single source of truth for users,
|
||||||
|
calendars, and address books** (`internal/db/users.go`):
|
||||||
|
`CreateUser`/`SetPassword`/`DeleteUser`/`GetUser`/`VerifyPassword`/
|
||||||
|
`ListUsers`, and `CreateCalendar`/`DeleteCalendar`/`ListCalendars`,
|
||||||
|
`CreateAddressBook`/`DeleteAddressBook`/`ListAddressBooks`. `users` is
|
||||||
|
the parent table; `calendars`/`addressbooks` cascade-delete via FK on
|
||||||
|
`DeleteUser`; `calendar_shares`/`addressbook_shares`/`web_sessions`
|
||||||
|
reference usernames as plain strings (no FK) so `DeleteUser` explicitly
|
||||||
|
cleans those up in a transaction. `modernc.org/sqlite` has no typed
|
||||||
|
unique-constraint error, so `isUniqueConstraintErr()` string-matches the
|
||||||
|
driver's error message.
|
||||||
|
- `internal/webdav` — plain-file WebDAV via `golang.org/x/net/webdav`,
|
||||||
|
mounted at the single fixed URL `/files/` for all users (no username in
|
||||||
|
the path either). `NewHandler` caches one `*xwebdav.Handler` per
|
||||||
|
authenticated username (keyed off `auth.FromContext`), each rooted at
|
||||||
|
`<data_dir>/files/<username>/` on disk with its own persistent
|
||||||
|
`LockSystem` — the handler (and its lock table) must be created once and
|
||||||
|
reused, not per-request, or LOCK/UNLOCK state resets on every call.
|
||||||
|
- `tools/hashpwd` — standalone CLI (`go run ./tools/hashpwd <password>`) to
|
||||||
|
generate bcrypt hashes for ad-hoc testing (no longer needed for normal
|
||||||
|
user setup — see `nidusctl user create` below).
|
||||||
|
- `tools/nidusctl` — standalone admin CLI (`go run ./tools/nidusctl
|
||||||
|
-config config.yaml <user|calendar|addressbook> ...`) — the only way to
|
||||||
|
create/delete users, calendars, and address books, plus manage sharing
|
||||||
|
grants (`calendar|addressbook share|unshare|shares`). It's a thin
|
||||||
|
argv-parsing wrapper around `db.DB`'s methods (`tools/nidusctl/main.go`
|
||||||
|
routes subcommands, `tools/nidusctl/users.go` implements `user
|
||||||
|
create/delete/list/passwd` with interactive masked password prompting
|
||||||
|
via `golang.org/x/term`, falling back to a plain stdin read when not a
|
||||||
|
TTY) — no server interaction, no daemon, no RPC; it just opens the same
|
||||||
|
SQLite file the running server uses. Changes take effect immediately
|
||||||
|
without restarting the server since nothing is cached.
|
||||||
|
- `internal/web` — the web UI, mounted at `/web/` in `cmd/server/main.go`
|
||||||
|
(`mux.Handle("/web/", http.StripPrefix("/web", web.NewServer(cfg, st,
|
||||||
|
dbase, logger).Handler(webstatic.FS())))`, so `Server.Handler`'s own
|
||||||
|
routes are all unprefixed — `/login`, `/`, `/shares/...` — and only the
|
||||||
|
outer mux adds the `/web` prefix), entirely separate from `internal/auth`'s
|
||||||
|
Basic Auth: logins go through
|
||||||
|
`/web/login` (username/password checked against the DB the same way
|
||||||
|
Basic Auth does, via `dbase.VerifyPassword`) and issue an opaque random session token
|
||||||
|
stored in the `web_sessions` SQLite table (`db.CreateSession`/
|
||||||
|
`SessionUser`/`DeleteSession`, see `internal/db/sessions.go`), set as an
|
||||||
|
`HttpOnly` cookie (`sessionCookieName` in `internal/web/session.go`).
|
||||||
|
`requireLogin` is the auth-guard middleware for authenticated routes,
|
||||||
|
storing the username in the request context (`userFromContext`).
|
||||||
|
`internal/web/dashboard.go` renders the logged-in user's own
|
||||||
|
calendars/address books plus who they're shared with
|
||||||
|
(`SharesOfCalendar`/`SharesOfAddressBook`) and what's shared with them
|
||||||
|
(`CalendarsSharedWith`/`AddressBooksSharedWith`); `resourceCards(username)`
|
||||||
|
is the shared helper (also used by `internal/web/resources.go`, see
|
||||||
|
below) that builds the list of `templates.ResourceCard`s from the DB.
|
||||||
|
`internal/web/resources.go` handles POST (create) and DELETE (delete) at
|
||||||
|
`/web/resources/{calendar,addressbook}`, validating names against
|
||||||
|
`resourceNameRe` (`^[a-zA-Z0-9_-]{1,64}$`) and re-rendering the whole
|
||||||
|
`#resources` list (`templates.ResourceList`) since the *set* of cards
|
||||||
|
changes (unlike a share update, which only touches one card).
|
||||||
|
`internal/web/shares.go`
|
||||||
|
handles POST (create/update share) and DELETE (revoke) at
|
||||||
|
`/web/shares/{calendar,addressbook}`, re-rendering just the affected
|
||||||
|
resource card for htmx's `hx-swap="outerHTML"`; it always checks
|
||||||
|
`ownsResource` first so a user can only share resources actually
|
||||||
|
configured for their own account (never someone else's, even via a
|
||||||
|
forged form post). **htmx v2 quirk**: `hx-delete` requests send
|
||||||
|
`hx-vals`/form params as URL **query string** parameters, not a request
|
||||||
|
body (unlike POST/PUT/PATCH) — `handleShare` special-cases
|
||||||
|
`r.Method == http.MethodDelete` to read from `r.URL.Query()` instead of
|
||||||
|
calling `r.ParseForm()`. Templates live in `internal/web/templates/*.templ`
|
||||||
|
(compiled to `*_templ.go` via `templ generate`/`make templ-generate` —
|
||||||
|
regenerate after editing any `.templ` file, the generated files are
|
||||||
|
committed). Styling is Tailwind v4, scanned directly over the generated
|
||||||
|
`_templ.go` files (`web/input.css`'s `@source` directives) and compiled
|
||||||
|
to `web/static/app.css` via `make web-css` (needs Node/npm — see
|
||||||
|
`web/package.json`); htmx itself is vendored as a static file
|
||||||
|
(`web/static/htmx.min.js`, not npm-installed) to avoid a CDN dependency.
|
||||||
|
Client-side-only logic (currently just the login page's password-visibility
|
||||||
|
toggle) is written in TypeScript under `web/ts/*.ts`, compiled to plain
|
||||||
|
JS via `tsc` (`web/tsconfig.json`, `make web-ts`) into `web/static/*.js`
|
||||||
|
as ES modules (`<script type="module">`). All static assets (CSS, JS,
|
||||||
|
vendored htmx) are embedded into the Go binary at build time via
|
||||||
|
`web/staticassets.go` (`//go:embed static`), so the compiled server has
|
||||||
|
no runtime dependency on Node.js or the `web/` directory being present —
|
||||||
|
Node/npm are only needed when actually changing templates/styles/TS
|
||||||
|
(`make web-assets` rebuilds everything under `web/static/`).
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- **No username in any DAV URL** (`/cal/`, `/card/`, `/files/` are the same
|
||||||
|
for every account) — the acting user is always resolved from the Basic
|
||||||
|
Auth identity (`auth.FromContext`), never parsed out of the request path.
|
||||||
|
Don't reintroduce a `<user>` path segment when adding routes/paths.
|
||||||
|
- Path parsing in the CalDAV/CardDAV backends assumes fixed URL segment
|
||||||
|
positions (e.g. `cal/home/<calname>/<objid>`) split on `/` — see
|
||||||
|
`parseCalPath`/`parseObjPath`/`parseBookPath`. The `home` segment must
|
||||||
|
stay exactly one fixed literal segment (see note above on go-webdav's
|
||||||
|
segment-count-based resource classification) — don't remove it or add/
|
||||||
|
remove segments elsewhere without re-checking all four resource-type
|
||||||
|
depths still line up. New path-based operations should follow the same
|
||||||
|
segment-index approach for consistency.
|
||||||
|
- Store errors are sentinel values (`store.ErrNotFound`, `store.ErrConflict`)
|
||||||
|
checked with `errors.Is`/direct comparison; backends translate them into
|
||||||
|
`webdav.NewHTTPError` with the appropriate HTTP status.
|
||||||
|
- Logging uses `log/slog` structured fields (e.g. `logger.Warn("...", "user", u, "error", err)`), passed down explicitly to every constructor (`NewBackend`, `NewHandler`, `NewMiddleware`) rather than a global logger.
|
||||||
|
- Config module path is `github.com/yourusername/caldav-server` (go.mod name
|
||||||
|
predates the `nidus` repo rename) — import paths still use this, not `nidus`.
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
# Data Directory Structure Unification - Summary of Changes
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
This change unifies the user data directory structure from a fragmented layout (with separate top-level `files/` directory) to a consistent nested format under each user's directory.
|
||||||
|
|
||||||
|
## Problem Statement
|
||||||
|
|
||||||
|
**Before**: Inconsistent directory structure
|
||||||
|
- WebDAV: `data/files/<username>/`
|
||||||
|
- CalDAV: `data/<username>/cal-<name>/`
|
||||||
|
- CardDAV: `data/<username>/card-<name>/`
|
||||||
|
|
||||||
|
**After**: Unified structure
|
||||||
|
- All data: `data/<username>/{files,calendars,addressbooks}/<name>/`
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
1. All user data in a single predictable location
|
||||||
|
2. Easier backups (single user directory instead of multiple paths)
|
||||||
|
3. Cleaner directory structure
|
||||||
|
4. Reduced code complexity in path resolution
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Files Modified
|
||||||
|
|
||||||
|
1. **`internal/store/store.go`**
|
||||||
|
- Updated `collectionPath()` to map collection names to new unified structure:
|
||||||
|
- `cal-<name>` → `<username>/calendars/<name>`
|
||||||
|
- `card-<name>` → `<username>/addressbooks/<name>`
|
||||||
|
- `files` → `<username>/files`
|
||||||
|
- Kept `sanitize()` for security
|
||||||
|
|
||||||
|
2. **`internal/store/migrate.go`** (new file)
|
||||||
|
- Added `Migrate()` method to restructure data directory
|
||||||
|
- Idempotent migration that handles both old and new structures
|
||||||
|
- Migrates WebDAV, CalDAV, and CardDAV data atomically
|
||||||
|
|
||||||
|
3. **`internal/store/migrate_test.go`** (new file)
|
||||||
|
- Comprehensive tests for migration functionality
|
||||||
|
- Covers migration, idempotency, and edge cases
|
||||||
|
|
||||||
|
4. **`internal/webdav/handler.go`**
|
||||||
|
- Updated WebDAV handler to use new unified path structure
|
||||||
|
- Changed from `dataDir/files/<username>` to `dataDir/<username>/files`
|
||||||
|
|
||||||
|
5. **`internal/webdav/handler_test.go`**
|
||||||
|
- Updated test assertions to check new path structure
|
||||||
|
- Changed from `dataDir/files/alice/` to `dataDir/alice/files/`
|
||||||
|
|
||||||
|
6. **`cmd/server/main.go`**
|
||||||
|
- Added automatic migration on server startup
|
||||||
|
- Ensures data directory is always in correct format
|
||||||
|
|
||||||
|
7. **`tools/migrate/main.go`** (new file)
|
||||||
|
- Standalone migration tool
|
||||||
|
- Can be run independently of server
|
||||||
|
|
||||||
|
8. **`tools/nidusctl/main.go`**
|
||||||
|
- Added `migrate` subcommand
|
||||||
|
- Integrated into existing admin CLI
|
||||||
|
|
||||||
|
### Path Mapping
|
||||||
|
|
||||||
|
| Old Path | New Path | Collection Type |
|
||||||
|
|----------|----------|-----------------|
|
||||||
|
| `files/<user>/<name>` | `<user>/files/<name>` | WebDAV |
|
||||||
|
| `<user>/cal-<name>/` | `<user>/calendars/<name>/` | CalDAV |
|
||||||
|
| `<user>/card-<name>/` | `<user>/addressbooks/<name>/` | CardDAV |
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
All existing tests pass with the new structure:
|
||||||
|
- ✅ Store tests (path resolution, locking)
|
||||||
|
- ✅ CalDAV backend tests
|
||||||
|
- ✅ CardDAV backend tests
|
||||||
|
- ✅ WebDAV handler tests
|
||||||
|
- ✅ Web UI tests
|
||||||
|
- ✅ DB tests
|
||||||
|
- ✅ Migration tests (new)
|
||||||
|
|
||||||
|
Run tests:
|
||||||
|
```bash
|
||||||
|
go test ./... -v
|
||||||
|
go test ./internal/store/... -run TestMigrate -v
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration Process
|
||||||
|
|
||||||
|
### For Existing Installations
|
||||||
|
|
||||||
|
1. **Stop the server** (optional but recommended)
|
||||||
|
```bash
|
||||||
|
# Stop any running server
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Run migration**
|
||||||
|
```bash
|
||||||
|
go run ./tools/nidusctl -config config.yaml migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Verify migration**
|
||||||
|
```bash
|
||||||
|
ls -la data/
|
||||||
|
# Should see user directories with unified structure
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Start the server**
|
||||||
|
```bash
|
||||||
|
go run ./cmd/server -config config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### For New Deployments
|
||||||
|
|
||||||
|
No migration needed - the new structure is used by default:
|
||||||
|
- Server creates `<username>/files/`, `<username>/calendars/`, etc.
|
||||||
|
- All data follows the unified structure from the start
|
||||||
|
|
||||||
|
## API Compatibility
|
||||||
|
|
||||||
|
**URL structure remains unchanged**:
|
||||||
|
- CalDAV: `/cal/`, `/cal/home/<name>/`, `/cal/home/<name>/<object>`
|
||||||
|
- CardDAV: `/card/`, `/card/home/<name>/`, `/card/home/<name>/<object>`
|
||||||
|
- WebDAV: `/files/`
|
||||||
|
|
||||||
|
Only the **on-disk path structure** changed. All HTTP endpoints and URL paths remain identical.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- Path sanitization maintained via `sanitize()` function
|
||||||
|
- No security-sensitive code changed
|
||||||
|
- Locking mechanism unchanged (still per-user)
|
||||||
|
- All same security checks apply
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
No measurable performance impact:
|
||||||
|
- Same number of filesystem operations
|
||||||
|
- Same lock granularity (per-user)
|
||||||
|
- Same cache behavior
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
If rollback is needed:
|
||||||
|
1. Restore data directory from backup
|
||||||
|
2. Restart server (will migrate again on next startup)
|
||||||
|
|
||||||
|
## Future Considerations
|
||||||
|
|
||||||
|
This unified structure makes future enhancements easier:
|
||||||
|
- Easier to add per-user quotas
|
||||||
|
- Simpler backup/restore logic
|
||||||
|
- Better support for user-specific configuration
|
||||||
|
- Cleaner codebase with consistent patterns
|
||||||
|
|
||||||
|
## Documentation Updates
|
||||||
|
|
||||||
|
- `MIGRATION.md` - Detailed migration guide
|
||||||
|
- `CHANGES_SUMMARY.md` - This file
|
||||||
|
- Code comments updated to reflect new structure
|
||||||
+150
@@ -0,0 +1,150 @@
|
|||||||
|
# Data Directory Migration Guide
|
||||||
|
|
||||||
|
This migration unifies the user data directory structure from a fragmented layout to a consistent nested format.
|
||||||
|
|
||||||
|
## Before (Old Structure)
|
||||||
|
|
||||||
|
```
|
||||||
|
data/
|
||||||
|
├── files/
|
||||||
|
│ └── alice/ # WebDAV files
|
||||||
|
│ └── documents/
|
||||||
|
│ └── file.txt
|
||||||
|
├── alice/
|
||||||
|
│ ├── cal-personal/ # CalDAV calendar
|
||||||
|
│ │ └── event1.ics
|
||||||
|
│ └── card-contacts/ # CardDAV address book
|
||||||
|
│ └── contact1.vcf
|
||||||
|
└── bob/
|
||||||
|
├── cal-work/
|
||||||
|
│ └── meeting.ics
|
||||||
|
└── card-addressbook/
|
||||||
|
└── address.vcf
|
||||||
|
```
|
||||||
|
|
||||||
|
## After (New Unified Structure)
|
||||||
|
|
||||||
|
```
|
||||||
|
data/
|
||||||
|
├── alice/
|
||||||
|
│ ├── files/ # WebDAV files
|
||||||
|
│ │ └── documents/
|
||||||
|
│ │ └── file.txt
|
||||||
|
│ ├── calendars/ # CalDAV calendars
|
||||||
|
│ │ └── personal/
|
||||||
|
│ │ └── event1.ics
|
||||||
|
│ └── addressbooks/ # CardDAV address books
|
||||||
|
│ └── contacts/
|
||||||
|
│ └── contact1.vcf
|
||||||
|
└── bob/
|
||||||
|
├── files/
|
||||||
|
├── calendars/
|
||||||
|
│ └── work/
|
||||||
|
└── addressbooks/
|
||||||
|
└── addressbook/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration Details
|
||||||
|
|
||||||
|
### What Changed
|
||||||
|
|
||||||
|
1. **WebDAV files**: `data/files/<username>/` → `data/<username>/files/`
|
||||||
|
2. **CalDAV calendars**: `data/<username>/cal-<name>/` → `data/<username>/calendars/<name>/`
|
||||||
|
3. **CardDAV address books**: `data/<username>/card-<name>/` → `data/<username>/addressbooks/<name>/`
|
||||||
|
|
||||||
|
### Migration Tool
|
||||||
|
|
||||||
|
A migration tool is provided that automatically restructures the data directory. It is **idempotent** and can be run multiple times safely.
|
||||||
|
|
||||||
|
#### Using nidusctl (Recommended)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./tools/nidusctl -config config.yaml migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Using standalone migrate tool
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./tools/migrate -config config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Verbose output
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./tools/nidusctl -config config.yaml migrate --verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
### What the Migration Does
|
||||||
|
|
||||||
|
1. For each user directory in the data directory:
|
||||||
|
- Creates `<username>/` if it doesn't exist
|
||||||
|
- Moves `files/<username>/` → `<username>/files/` (if exists)
|
||||||
|
- Moves `cal-<name>/` → `<username>/calendars/<name>/`
|
||||||
|
- Moves `card-<name>/` → `<username>/addressbooks/<name>/`
|
||||||
|
|
||||||
|
2. The old `files/` directory is left in place (can be manually removed after verification)
|
||||||
|
|
||||||
|
### Backward Compatibility
|
||||||
|
|
||||||
|
The migration is **fully backward compatible**:
|
||||||
|
- The migration tool handles both old and new structures
|
||||||
|
- If files are already in the new location, they are not moved
|
||||||
|
- Running migration multiple times is safe (idempotent)
|
||||||
|
|
||||||
|
### Server Integration
|
||||||
|
|
||||||
|
The server automatically runs migration on startup (if enabled in config). No manual migration is required for new installations.
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
After migration, verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check data structure
|
||||||
|
ls -la data/
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
go test ./...
|
||||||
|
|
||||||
|
# Start server to verify WebDAV, CalDAV, CardDAV work correctly
|
||||||
|
go run ./cmd/server -config config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Verification
|
||||||
|
|
||||||
|
After migration, you should see:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ ls -la data/alice/
|
||||||
|
calendars/
|
||||||
|
addressbooks/
|
||||||
|
files/
|
||||||
|
```
|
||||||
|
|
||||||
|
Each calendar should be in `data/<user>/calendars/<name>/` format, not `cal-<name>/`.
|
||||||
|
|
||||||
|
### Rollback (if needed)
|
||||||
|
|
||||||
|
If you need to rollback:
|
||||||
|
1. Stop the server
|
||||||
|
2. Restore the data directory from backup
|
||||||
|
3. Re-run the migration after fixing any issues
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
**Q: Migration reports "directory already exists" warnings**
|
||||||
|
A: These are normal if files were already migrated. The migration is idempotent.
|
||||||
|
|
||||||
|
**Q: Old `files/` directory still exists**
|
||||||
|
A: This is expected. You can manually remove it after verifying migration success.
|
||||||
|
|
||||||
|
**Q: Some calendars/address books not visible after migration**
|
||||||
|
A: Check the migration logs and verify directory structure. Run `find data/ -type d -name "cal-*"` to find unmigrated calendars.
|
||||||
|
|
||||||
|
### Support
|
||||||
|
|
||||||
|
If you encounter issues:
|
||||||
|
1. Check logs for detailed error messages
|
||||||
|
2. Run migration with `--verbose` flag
|
||||||
|
3. Ensure no server processes are running during migration
|
||||||
|
4. Make backup before migrating in production
|
||||||
@@ -110,14 +110,30 @@ Pushing a version tag (e.g. `v1.2.3`) or publishing a release triggers
|
|||||||
which builds and publishes a multi-arch (`linux/amd64` + `linux/arm64`)
|
which builds and publishes a multi-arch (`linux/amd64` + `linux/arm64`)
|
||||||
image to `git.arnef.de/arnef/nidus`, tagged with the version, `<major>.<minor>`,
|
image to `git.arnef.de/arnef/nidus`, tagged with the version, `<major>.<minor>`,
|
||||||
`latest`, and the short commit SHA. It authenticates via the
|
`latest`, and the short commit SHA. It authenticates via the
|
||||||
`REGISTRY_USERNAME`/`REGISTRY_PASSWORD` repository secrets. Point
|
`REGISTRY_USERNAME`/`REGISTRY_PASSWORD` repository secrets.
|
||||||
`docker-compose.yaml`'s `image:` at it instead of `build: .` to use it
|
|
||||||
directly, e.g.:
|
A minimal standalone `docker-compose.yml` that pulls this image instead of
|
||||||
|
building from a checkout — just fetch `config.example.yaml`, copy it to
|
||||||
|
`config.yaml`, and adjust it to your needs:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
davserver:
|
davserver:
|
||||||
image: git.arnef.de/arnef/nidus:latest
|
image: git.arnef.de/arnef/nidus:latest
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
volumes:
|
||||||
|
- ./config.yaml:/app/config.yaml:ro
|
||||||
|
- dav-data:/app/data
|
||||||
|
restart: unless-stopped
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
dav-data:
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -230,33 +246,6 @@ make web-assets # regenerate templ code + rebuild web/static/app.css and web/st
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## TLS / Reverse proxy
|
|
||||||
|
|
||||||
### Self-signed certificate (development)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
|
|
||||||
```
|
|
||||||
|
|
||||||
Update `config.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
tls:
|
|
||||||
enabled: true
|
|
||||||
cert_file: cert.pem
|
|
||||||
key_file: key.pem
|
|
||||||
```
|
|
||||||
|
|
||||||
### Caddy reverse proxy (recommended for production)
|
|
||||||
|
|
||||||
```
|
|
||||||
dav.example.com {
|
|
||||||
reverse_proxy localhost:8080
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Configuration reference
|
## Configuration reference
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -370,24 +359,3 @@ reviewed and tested where practical, but not every part of the codebase
|
|||||||
has been fully reviewed yet — use accordingly, especially before relying
|
has been fully reviewed yet — use accordingly, especially before relying
|
||||||
on this in security-sensitive environments.
|
on this in security-sensitive environments.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
| Package | Purpose |
|
|
||||||
|---------|---------|
|
|
||||||
| `github.com/emersion/go-webdav` | WebDAV/CalDAV/CardDAV protocol layer |
|
|
||||||
| `github.com/emersion/go-ical` | iCalendar parsing/serialisation |
|
|
||||||
| `github.com/emersion/go-vcard` | vCard parsing/serialisation |
|
|
||||||
| `golang.org/x/crypto` | bcrypt |
|
|
||||||
| `golang.org/x/net` | `golang.org/x/net/webdav` |
|
|
||||||
| `gopkg.in/yaml.v3` | YAML config parsing |
|
|
||||||
| `modernc.org/sqlite` | Pure-Go SQLite driver (shares, web UI sessions) |
|
|
||||||
| `github.com/a-h/templ` | Type-safe Go HTML templates (web UI) |
|
|
||||||
|
|
||||||
Front-end (dev-only, not required at runtime — see [Web UI](#web-ui)):
|
|
||||||
|
|
||||||
| Tool | Purpose |
|
|
||||||
|------|---------|
|
|
||||||
| Tailwind CSS v4 (`web/package.json`) | Utility-first CSS, compiled to `web/static/app.css` |
|
|
||||||
| [htmx](https://htmx.org/) (`web/static/htmx.min.js`, vendored) | Partial page updates without a JS framework |
|
|
||||||
|
|||||||
@@ -51,6 +51,13 @@ func main() {
|
|||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- Run migration if data directory needs restructuring ----
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
logger.Error("migration failed", "error", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
logger.Info("data directory migration completed")
|
||||||
|
|
||||||
// ---- Database (users, calendars, address books, sharing) ----
|
// ---- Database (users, calendars, address books, sharing) ----
|
||||||
dbPath := filepath.Join(cfg.Storage.DataDir, "nidus.db")
|
dbPath := filepath.Join(cfg.Storage.DataDir, "nidus.db")
|
||||||
dbase, err := db.Open(dbPath)
|
dbase, err := db.Open(dbPath)
|
||||||
|
|||||||
@@ -102,8 +102,6 @@ func (d *DB) DisplayName(username string) string {
|
|||||||
return u.DisplayName
|
return u.DisplayName
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
// DeleteUser removes username along with all of its calendars, address
|
// DeleteUser removes username along with all of its calendars, address
|
||||||
// books, and sharing grants (calendars/addressbooks cascade via foreign
|
// books, and sharing grants (calendars/addressbooks cascade via foreign
|
||||||
// key; shares are cleaned up explicitly since they reference usernames as
|
// key; shares are cleaned up explicitly since they reference usernames as
|
||||||
|
|||||||
@@ -0,0 +1,200 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Migrate restructures the data directory from the old format to the new unified format.
|
||||||
|
// It is idempotent and can be run multiple times safely.
|
||||||
|
//
|
||||||
|
// Old structure:
|
||||||
|
//
|
||||||
|
// data/files/<username>/
|
||||||
|
// data/<username>/cal-<name>/
|
||||||
|
// data/<username>/card-<name>/
|
||||||
|
//
|
||||||
|
// New structure:
|
||||||
|
//
|
||||||
|
// data/<username>/files/
|
||||||
|
// data/<username>/calendars/<name>/
|
||||||
|
// data/<username>/addressbooks/<name>/
|
||||||
|
func (s *Store) Migrate() error {
|
||||||
|
users, err := s.listUserDirectories()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("listing user directories: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, user := range users {
|
||||||
|
if err := s.migrateUser(user); err != nil {
|
||||||
|
return fmt.Errorf("migrating user %q: %w", user, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listUserDirectories returns all user directories in the data directory.
|
||||||
|
// It looks for directories that are NOT the old "files" directory,
|
||||||
|
// and also checks inside the old "files" directory for users who need migrating.
|
||||||
|
func (s *Store) listUserDirectories() ([]string, error) {
|
||||||
|
entries, err := os.ReadDir(s.rootDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
usersMap := make(map[string]bool)
|
||||||
|
|
||||||
|
for _, e := range entries {
|
||||||
|
if !e.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name := e.Name()
|
||||||
|
// Skip the old files directory (will handle users inside it separately)
|
||||||
|
if name == "files" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
usersMap[name] = true
|
||||||
|
}
|
||||||
|
|
||||||
|
// Also check the old files directory for users
|
||||||
|
filesDir := filepath.Join(s.rootDir, "files")
|
||||||
|
if filesEntries, err := os.ReadDir(filesDir); err == nil {
|
||||||
|
for _, e := range filesEntries {
|
||||||
|
if e.IsDir() {
|
||||||
|
usersMap[e.Name()] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var users []string
|
||||||
|
for user := range usersMap {
|
||||||
|
users = append(users, user)
|
||||||
|
}
|
||||||
|
return users, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// migrateUser migrates a single user's data from old to new structure.
|
||||||
|
func (s *Store) migrateUser(user string) error {
|
||||||
|
userDir := filepath.Join(s.rootDir, user)
|
||||||
|
|
||||||
|
// Create user directory if it doesn't exist (needed for WebDAV migration)
|
||||||
|
if err := os.MkdirAll(userDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("creating user directory %q: %w", userDir, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Migrate WebDAV files: files/<username> -> <username>/files
|
||||||
|
if err := s.migrateWebDAV(user); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Migrate CalDAV calendars: cal-<name> -> calendars/<name>
|
||||||
|
if err := s.migrateCalendars(user, userDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Migrate CardDAV address books: card-<name> -> addressbooks/<name>
|
||||||
|
if err := s.migrateAddressBooks(user, userDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// migrateWebDAV moves the old files/<username> directory to <username>/files.
|
||||||
|
func (s *Store) migrateWebDAV(user string) error {
|
||||||
|
oldPath := filepath.Join(s.rootDir, "files", user)
|
||||||
|
newPath := filepath.Join(s.rootDir, user, "files")
|
||||||
|
|
||||||
|
// Check if old directory exists and new doesn't
|
||||||
|
if _, err := os.Stat(oldPath); os.IsNotExist(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(newPath); err == nil {
|
||||||
|
return nil // Already migrated
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create parent directory if needed
|
||||||
|
if err := os.MkdirAll(filepath.Dir(newPath), 0o755); err != nil {
|
||||||
|
return fmt.Errorf("creating directory %q: %w", filepath.Dir(newPath), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move the directory
|
||||||
|
if err := os.Rename(oldPath, newPath); err != nil {
|
||||||
|
return fmt.Errorf("moving %q to %q: %w", oldPath, newPath, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// migrateCalendars moves old cal-<name> directories to calendars/<name>.
|
||||||
|
func (s *Store) migrateCalendars(user string, userDir string) error {
|
||||||
|
entries, err := os.ReadDir(userDir)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("reading user directory %q: %w", userDir, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var calendars []string
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.IsDir() && strings.HasPrefix(e.Name(), "cal-") {
|
||||||
|
calendars = append(calendars, e.Name())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(calendars) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
calendarsDir := filepath.Join(userDir, "calendars")
|
||||||
|
if err := os.MkdirAll(calendarsDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("creating calendars directory %q: %w", calendarsDir, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, calName := range calendars {
|
||||||
|
oldPath := filepath.Join(userDir, calName)
|
||||||
|
newPath := filepath.Join(calendarsDir, strings.TrimPrefix(calName, "cal-"))
|
||||||
|
|
||||||
|
if err := os.Rename(oldPath, newPath); err != nil {
|
||||||
|
return fmt.Errorf("moving calendar %q: %w", calName, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// migrateAddressBooks moves old card-<name> directories to addressbooks/<name>.
|
||||||
|
func (s *Store) migrateAddressBooks(user string, userDir string) error {
|
||||||
|
entries, err := os.ReadDir(userDir)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("reading user directory %q: %w", userDir, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var addressBooks []string
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.IsDir() && strings.HasPrefix(e.Name(), "card-") {
|
||||||
|
addressBooks = append(addressBooks, e.Name())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(addressBooks) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
addressBooksDir := filepath.Join(userDir, "addressbooks")
|
||||||
|
if err := os.MkdirAll(addressBooksDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("creating addressbooks directory %q: %w", addressBooksDir, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, bookName := range addressBooks {
|
||||||
|
oldPath := filepath.Join(userDir, bookName)
|
||||||
|
newPath := filepath.Join(addressBooksDir, strings.TrimPrefix(bookName, "card-"))
|
||||||
|
|
||||||
|
if err := os.Rename(oldPath, newPath); err != nil {
|
||||||
|
return fmt.Errorf("moving address book %q: %w", bookName, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestMigrate(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
|
||||||
|
// Create old structure
|
||||||
|
// WebDAV: files/<username>/
|
||||||
|
if err := os.MkdirAll(filepath.Join(tmpDir, "files", "alice"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(tmpDir, "files", "alice", "test.txt"), []byte("test"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CalDAV: <username>/cal-<name>/
|
||||||
|
if err := os.MkdirAll(filepath.Join(tmpDir, "bob", "cal-work"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(tmpDir, "bob", "cal-work", "evt.ics"), []byte("calendar"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CardDAV: <username>/card-<name>/
|
||||||
|
if err := os.MkdirAll(filepath.Join(tmpDir, "bob", "card-contacts"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(tmpDir, "bob", "card-contacts", "vcard.vcf"), []byte("card"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create store and migrate
|
||||||
|
st, err := NewStore(tmpDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify WebDAV: <username>/files/
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "alice", "files", "test.txt")); err != nil {
|
||||||
|
t.Errorf("alice files not migrated: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify CalDAV: <username>/calendars/<name>
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "bob", "calendars", "work", "evt.ics")); err != nil {
|
||||||
|
t.Errorf("bob calendars not migrated: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify CardDAV: <username>/addressbooks/<name>
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "bob", "addressbooks", "contacts", "vcard.vcf")); err != nil {
|
||||||
|
t.Errorf("bob addressbooks not migrated: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify old structure is gone
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "files", "alice")); err == nil {
|
||||||
|
t.Error("old files directory not removed")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "bob", "cal-work")); err == nil {
|
||||||
|
t.Error("old cal- directory not removed")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "bob", "card-contacts")); err == nil {
|
||||||
|
t.Error("old card- directory not removed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrateIdempotent(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
|
||||||
|
// Create new structure
|
||||||
|
if err := os.MkdirAll(filepath.Join(tmpDir, "alice", "files"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(tmpDir, "alice", "files", "test.txt"), []byte("test"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
st, err := NewStore(tmpDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run migration twice
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify data still there
|
||||||
|
if _, err := os.Stat(filepath.Join(tmpDir, "alice", "files", "test.txt")); err != nil {
|
||||||
|
t.Errorf("data not preserved: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrateMissingDirectories(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
|
||||||
|
st, err := NewStore(tmpDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Should not error on empty directory
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
t.Errorf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
+22
-3
@@ -48,9 +48,26 @@ func (s *Store) lockFor(user string) *sync.RWMutex {
|
|||||||
return l
|
return l
|
||||||
}
|
}
|
||||||
|
|
||||||
// collectionPath returns the filesystem path for a collection.
|
// collectionPath returns the filesystem path for a collection in the new unified format.
|
||||||
|
// Calendar collections: <username>/calendars/<name>
|
||||||
|
// Address book collections: <username>/addressbooks/<name>
|
||||||
|
// WebDAV collections: <username>/files (all files in one directory)
|
||||||
func (s *Store) collectionPath(user, collection string) string {
|
func (s *Store) collectionPath(user, collection string) string {
|
||||||
return filepath.Join(s.rootDir, sanitize(user), sanitize(collection))
|
userDir := filepath.Join(s.rootDir, sanitize(user))
|
||||||
|
|
||||||
|
if strings.HasPrefix(collection, "cal-") {
|
||||||
|
return filepath.Join(userDir, "calendars", strings.TrimPrefix(collection, "cal-"))
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.HasPrefix(collection, "card-") {
|
||||||
|
return filepath.Join(userDir, "addressbooks", strings.TrimPrefix(collection, "card-"))
|
||||||
|
}
|
||||||
|
|
||||||
|
if collection == "files" {
|
||||||
|
return filepath.Join(userDir, "files")
|
||||||
|
}
|
||||||
|
|
||||||
|
return filepath.Join(userDir, sanitize(collection))
|
||||||
}
|
}
|
||||||
|
|
||||||
// objectPath returns the filesystem path for an object within a collection.
|
// objectPath returns the filesystem path for an object within a collection.
|
||||||
@@ -68,6 +85,7 @@ func (s *Store) EnsureCollection(user, collection string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ListCollections returns all collection names for a user.
|
// ListCollections returns all collection names for a user.
|
||||||
|
// Returns both old-style (cal-*, card-*) and new-style (calendars/*, addressbooks/*) collections.
|
||||||
func (s *Store) ListCollections(user string) ([]string, error) {
|
func (s *Store) ListCollections(user string) ([]string, error) {
|
||||||
l := s.lockFor(user)
|
l := s.lockFor(user)
|
||||||
l.RLock()
|
l.RLock()
|
||||||
@@ -85,7 +103,8 @@ func (s *Store) ListCollections(user string) ([]string, error) {
|
|||||||
var names []string
|
var names []string
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
if e.IsDir() {
|
if e.IsDir() {
|
||||||
names = append(names, e.Name())
|
name := e.Name()
|
||||||
|
names = append(names, name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return names, nil
|
return names, nil
|
||||||
|
|||||||
@@ -527,7 +527,6 @@ func mondayOf(t time.Time) time.Time {
|
|||||||
return time.Date(t.Year(), t.Month(), t.Day(), 0, 0, 0, 0, t.Location()).AddDate(0, 0, -offset)
|
return time.Date(t.Year(), t.Month(), t.Day(), 0, 0, 0, 0, t.Location()).AddDate(0, 0, -offset)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
// eventDayRange returns the inclusive [start, end] calendar-day span an
|
// eventDayRange returns the inclusive [start, end] calendar-day span an
|
||||||
// event occupies, in loc, for placing it on the month grid.
|
// event occupies, in loc, for placing it on the month grid.
|
||||||
func eventDayRange(form templates.EventFormData, loc *time.Location) (start, end time.Time, err error) {
|
func eventDayRange(form templates.EventFormData, loc *time.Location) (start, end time.Time, err error) {
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ const maxUploadMemory = 32 << 20 // 32 MiB
|
|||||||
// (internal/webdav) serves at /files/, so the web UI is just another view
|
// (internal/webdav) serves at /files/, so the web UI is just another view
|
||||||
// onto the same files.
|
// onto the same files.
|
||||||
func (s *Server) filesRoot(username string) string {
|
func (s *Server) filesRoot(username string) string {
|
||||||
return filepath.Join(s.cfg.Storage.DataDir, "files", username)
|
return filepath.Join(s.cfg.Storage.DataDir, username, "files")
|
||||||
}
|
}
|
||||||
|
|
||||||
// sanitizeRelPath cleans a slash-separated relative path (as received from
|
// sanitizeRelPath cleans a slash-separated relative path (as received from
|
||||||
|
|||||||
@@ -316,4 +316,3 @@ func TestAccountChangePassword(t *testing.T) {
|
|||||||
t.Fatal("expected password to have changed")
|
t.Fatal("expected password to have changed")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
|
|
||||||
// NewHandler returns an http.Handler that provides standard WebDAV file access,
|
// NewHandler returns an http.Handler that provides standard WebDAV file access,
|
||||||
// mounted at the fixed URL /files/ for every user and rooted at
|
// mounted at the fixed URL /files/ for every user and rooted at
|
||||||
// dataDir/files/<username>/ on disk. The URL is the same for all users —
|
// dataDir/<username>/files on disk. The URL is the same for all users —
|
||||||
// which user's directory is served is resolved from the Basic Auth identity
|
// which user's directory is served is resolved from the Basic Auth identity
|
||||||
// in the request context, not from the URL.
|
// in the request context, not from the URL.
|
||||||
//
|
//
|
||||||
@@ -38,7 +38,7 @@ func NewHandler(cfg *config.Config, dataDir string, logger *slog.Logger) http.Ha
|
|||||||
h, ok := handlers[p.Username]
|
h, ok := handlers[p.Username]
|
||||||
if !ok {
|
if !ok {
|
||||||
username := p.Username
|
username := p.Username
|
||||||
userDir := filepath.Join(dataDir, "files", username)
|
userDir := filepath.Join(dataDir, username, "files")
|
||||||
if err := os.MkdirAll(userDir, 0o755); err != nil {
|
if err := os.MkdirAll(userDir, 0o755); err != nil {
|
||||||
mu.Unlock()
|
mu.Unlock()
|
||||||
logger.Error("creating user WebDAV dir", "user", username, "error", err)
|
logger.Error("creating user WebDAV dir", "user", username, "error", err)
|
||||||
|
|||||||
@@ -54,10 +54,10 @@ func TestPerUserIsolationAndPrefix(t *testing.T) {
|
|||||||
t.Fatalf("expected bob to get 404 for alice's file, got %d", bobRec.Code)
|
t.Fatalf("expected bob to get 404 for alice's file, got %d", bobRec.Code)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Confirm the file physically landed under dataDir/files/alice/, not
|
// Confirm the file physically landed under dataDir/alice/files/, not
|
||||||
// nested under an extra files/files/... path.
|
// nested under an extra files/files/... path.
|
||||||
if _, err := os.Stat(dir + "/files/alice/note.txt"); err != nil {
|
if _, err := os.Stat(dir + "/alice/files/note.txt"); err != nil {
|
||||||
t.Fatalf("expected file at dataDir/files/alice/note.txt: %v", err)
|
t.Fatalf("expected file at dataDir/alice/files/note.txt: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
// Command migrate is a tool to restructure the data directory from the
|
||||||
|
// old format to the new unified format.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/yourusername/caldav-server/internal/config"
|
||||||
|
"github.com/yourusername/caldav-server/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
os.Exit(run(os.Args[1:]))
|
||||||
|
}
|
||||||
|
|
||||||
|
func run(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||||
|
cfgPath := fs.String("config", "config.yaml", "path to configuration file")
|
||||||
|
verbose := fs.Bool("verbose", false, "enable verbose output")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg, err := config.Load(*cfgPath)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "error loading config: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
logger := buildLogger(*verbose)
|
||||||
|
|
||||||
|
st, err := store.NewStore(cfg.Storage.DataDir)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "error opening store %q: %v\n", cfg.Storage.DataDir, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.Info("starting migration", "data_dir", cfg.Storage.DataDir)
|
||||||
|
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
logger.Error("migration failed", "error", err)
|
||||||
|
fmt.Fprintf(os.Stderr, "migration failed: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
logger.Info("migration completed successfully")
|
||||||
|
fmt.Println("Migration completed successfully")
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildLogger(verbose bool) *slog.Logger {
|
||||||
|
level := slog.LevelInfo
|
||||||
|
if verbose {
|
||||||
|
level = slog.LevelDebug
|
||||||
|
}
|
||||||
|
handler := slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
|
||||||
|
Level: level,
|
||||||
|
})
|
||||||
|
return slog.New(handler)
|
||||||
|
}
|
||||||
+20
-1
@@ -61,6 +61,8 @@ func run(args []string) int {
|
|||||||
return runCalendar(dbase, st, rest[1:])
|
return runCalendar(dbase, st, rest[1:])
|
||||||
case "addressbook", "card":
|
case "addressbook", "card":
|
||||||
return runAddressBook(dbase, st, rest[1:])
|
return runAddressBook(dbase, st, rest[1:])
|
||||||
|
case "migrate":
|
||||||
|
return runMigrate(st, rest[1:])
|
||||||
case "help", "-h", "--help":
|
case "help", "-h", "--help":
|
||||||
usage()
|
usage()
|
||||||
return 0
|
return 0
|
||||||
@@ -95,13 +97,16 @@ Usage:
|
|||||||
nidusctl [-config config.yaml] addressbook unshare <owner> <book> <user>
|
nidusctl [-config config.yaml] addressbook unshare <owner> <book> <user>
|
||||||
nidusctl [-config config.yaml] addressbook shares <owner> <book>
|
nidusctl [-config config.yaml] addressbook shares <owner> <book>
|
||||||
|
|
||||||
|
nidusctl [-config config.yaml] migrate [--verbose]
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
nidusctl user create alice --display-name "Alice Smith" --email alice@example.com
|
nidusctl user create alice --display-name "Alice Smith" --email alice@example.com
|
||||||
nidusctl calendar create alice work
|
nidusctl calendar create alice work
|
||||||
nidusctl calendar share alice work bob write
|
nidusctl calendar share alice work bob write
|
||||||
nidusctl calendar shares alice work
|
nidusctl calendar shares alice work
|
||||||
nidusctl calendar unshare alice work bob
|
nidusctl calendar unshare alice work bob
|
||||||
`)
|
nidusctl migrate
|
||||||
|
`)
|
||||||
}
|
}
|
||||||
|
|
||||||
// newFlagSet creates a flag.FlagSet configured for subcommand parsing
|
// newFlagSet creates a flag.FlagSet configured for subcommand parsing
|
||||||
@@ -359,6 +364,20 @@ func runAddressBook(dbase *db.DB, st *store.Store, args []string) int {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func runMigrate(st *store.Store, args []string) int {
|
||||||
|
fs := newFlagSet("migrate")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := st.Migrate(); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "migration failed: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
fmt.Println("Migration completed successfully")
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
// warnIfUnknownUser reports (without failing) if username is not a
|
// warnIfUnknownUser reports (without failing) if username is not a
|
||||||
// registered user — the share is still recorded, since a user could be
|
// registered user — the share is still recorded, since a user could be
|
||||||
// created afterwards.
|
// created afterwards.
|
||||||
|
|||||||
Reference in New Issue
Block a user