matrix-search

A fast local full-text search for your Matrix message history, running alongside Element instead of inside it. Syncs your rooms (including encrypted ones, with caveats below) into a local SQLite FTS5 index and serves a small web UI to search it.

Why

Element's built-in search (server-side /search, or Seshat locally) is slow, inconsistent, and often just doesn't find things. This runs its own full-text index and gives you a plain search box + snippet results, with one-click links back into Element.

Setup

1. Copy the env file:

   cp .env.example .env
   

2. Fill in MATRIX_HOMESERVER and MATRIX_USER_ID.

   MATRIX_USER_ID is your full Matrix ID, e.g. @yourname:example.com. Don't leave it commented out - it's required no matter how you log in.

   MATRIX_HOMESERVER must be the actual client-server API base URL, which is often not the same as your account's server name (the part after the : in your user ID). Find the real one with:

   curl -s https://<your-server-name>/.well-known/matrix/client
   

   and use the m.homeserver.base_url value from the response (e.g. your server name might be example.com but the API actually lives at https://matrix.example.com). If that returns nothing useful, check Element's Settings > Help & About > Advanced, which shows the resolved homeserver URL it's actually talking to.

3. Choose how the indexer logs in - this depends on how you log into Element for this account:

   A) You log in with a username + password. Set MATRIX_PASSWORD in .env to that password. That's it, skip to step 4. It's only needed for the first run - once the indexer logs in successfully it saves its own session to data/credentials.json, and you can delete the password from .env after that.

   B) You log in via SSO (clicking "Continue with Google/Microsoft/a company login page" - no separate Matrix password exists for your account). Password login will fail with M_FORBIDDEN in this case even with the right credentials, because there's no password to check. Instead you get the indexer a one-time login token through the same SSO flow Element itself uses:

   1. Confirm the homeserver actually offers this: curl -s https://<homeserver-from-step-2>/_matrix/client/v3/login should list "type":"m.login.token" alongside m.login.sso in its flows. (Nearly all SSO-enabled homeservers offer this as a compatibility path for non-browser clients like this one.)

   2. On your own machine, in a browser where you're already signed into SSO (e.g. the same browser Element runs in), visit:

      https://<homeserver-from-step-2>/_matrix/client/v3/login/sso/redirect?redirectUrl=http://localhost:17999/
      

      The port number doesn't matter and nothing needs to be listening on it. Complete the SSO prompt if asked.

   3. The browser will end up trying (and failing) to load something like http://localhost:17999/?loginToken=AbCdEf123... - that's expected. Copy the loginToken= value straight out of the address bar.

   4. This token is single-use and expires within a couple of minutes. Paste it into .env as MATRIX_LOGIN_TOKEN=... and move straight to step 4 below - don't let time pass between copying the token and starting the container, and don't reuse a token from an earlier attempt. If it expires before you start the container, just repeat steps 2-3 for a fresh one.

   5. Once login succeeds (check the logs in step 5), delete MATRIX_LOGIN_TOKEN from .env - it's already been consumed and the saved session in data/credentials.json is what gets used from then on.

4. Build and start:

   docker compose up -d --build
   

5. Watch the logs during the first run — it does a full backfill of every joined room's history, which can take a while on a large account:

   docker compose logs -f
   

   A successful login looks like Logged in and created new device <id>... followed by Performing initial sync.... If it instead exits with a RuntimeError, re-check steps 2-3 - almost every login failure traces back to a wrong MATRIX_HOMESERVER or the SSO token having expired.

6. Open http://<docker-host>:8080 (or whatever host port you mapped in docker-compose.yml) and search. The status line under the search box shows how many messages/rooms are indexed so far.

Encrypted rooms

Matrix end-to-end encryption means a message is only decryptable by devices that were actually given the room's encryption key. The indexer logs in as a brand-new device, so by default it has none of those keys and every encrypted message - past or future - shows up as undecryptable and gets skipped rather than indexed. Unencrypted rooms need none of this and index fully automatically; this section is only about rooms that show a lock icon in Element.

Historical messages (everything sent before you set this up): import your existing room keys once.

1. In Element: Settings > Security & Privacy > Export keys. Pick a passphrase and save the file.
2. Copy that file to the docker host, into the data/ directory, e.g. data/element-keys.txt.
3. In .env, set:

   MATRIX_KEYS_FILE=/data/element-keys.txt
   MATRIX_KEYS_PASSPHRASE=<the passphrase you chose in step 1>
   

4. Restart (no rebuild needed unless you also changed code):

   docker compose restart
   docker compose logs -f
   

   On every startup the indexer re-syncs and redoes the full backfill; since it skips messages it's already indexed, this just fills in whatever's newly decryptable with the imported keys. You can confirm how much was recovered by checking:

   docker compose logs | grep -i "backfill complete"
   

   which prints how many events still couldn't be decrypted.

New messages going forward: a key export only covers the past. For the indexer to keep decrypting new encrypted messages as they arrive, your other Matrix sessions need to trust its device enough to share new keys with it. Two ways to handle that:

• Recommended, one-time setup: in Element, go to Settings > Sessions, find the session named matrix-search-indexer, and verify it using your recovery key. Once verified, other devices will share new room keys with it automatically, no further steps needed.
• Manual fallback: periodically repeat the key-export steps above and docker compose restart. Anything sent between exports won't be searchable until the next export.

Data & security notes

• data/search.db contains the decrypted plaintext of every indexed message. Treat the data/ directory like you'd treat your Matrix account itself — it's a local archive, not encrypted at rest.
• data/credentials.json holds an access token for a real logged-in Matrix session. Don't commit it or copy it around.
• This only runs against the account you log in as; it doesn't touch or need admin access to your homeserver.

API

• GET /api/search?q=...&limit=50&room_id=... — search results with snippets and matrix.to / Element deep links.
• GET /api/status — indexed message/room counts, useful for a "is the backfill done yet" check.