troubleshooting
This page is for operators. Each section gives a symptom, a check and a fix. The examples come from reilly.asia in November 2026.
Images and avatars do not load
Element X shows "Sorry, an error occurred" for an image, or it shows no avatars. Check these causes in this order.
The client asks at the unstable media prefix
Element X 26.11 asks for media at
/_matrix/client/unstable/org.matrix.msc3916/media/. It does this when
/versions names org.matrix.msc3916.stable. Releases before v0.1.7 serve
only /_matrix/client/v1/media/, so each request gets a 404.
To check, request one image at each prefix with an access token:
for prefix in v1 unstable/org.matrix.msc3916; do
curl -s -o /dev/null -w "%{http_code} $prefix\n" \
-H "Authorization: Bearer $TOKEN" \
"https://matrix.example.org/_matrix/client/$prefix/media/thumbnail/matrix.org/MEDIA_ID?width=96&height=96"
done
A 200 for v1 and a 404 for the unstable prefix is this cause. Upgrade to
v0.1.7 or later. On an older release, rewrite the prefix in the proxy.
This is the Caddy form:
@msc3916 path_regexp msc3916 ^/_matrix/client/unstable/org\.matrix\.msc3916/media/(.*)$
rewrite @msc3916 /_matrix/client/v1/media/{re.msc3916.1}
A remote file is in the cache with zero bytes
Some servers send a Location header in place of the file. Beeper does
this for avatars. Releases before v0.1.5 kept the empty part as the file.
Such a file has a size of zero and the type application/octet-stream.
v0.1.5 gets these files again when a client asks for them.
A remote image has the type application/octet-stream
The object store of some servers sends no image type. The thumbnail route
then answers M_UNSUPPORTED: no thumbnails for application/octet-stream.
From v0.1.6, Spindle reads the first bytes of the file to find the type.
A remote server does not answer
The thumbnail answers 404 with did not yield after about 30 seconds. The
other server is down. Spindle cannot fix this.
A room shows no members
The room list is correct, but the member list stays empty. The member avatars do not show. This occurs after a join to a large room.
Such a join is partial (MSC3706). Later, the server gets the list of
all the members. Until then, /members and /joined_members do not answer. To
check, look for this line in the log:
cannot resync a partial-state room yet: ...
The text after the colon gives the cause. Releases before v0.1.6 stop on one member event with a bad signature. They also stop when the room's server refuses one of its auth events with a 403. v0.1.6 leaves out those member events and completes the resync. This line shows the count:
resync leaves out member events that do not verify ... dropped=3
Find what a client asks for
The metrics of Spindle count the requests for each route. A request that no
route matches has the route label unmatched. To see the path, log the
requests in the proxy. Do not log the headers or the access token. This is
the Caddy form:
log {
output file /data/access.log
format filter {
wrap json
fields {
request>headers delete
resp_headers delete
request>uri query {
delete access_token
}
}
}
}
Write the log to a file. If the Docker log file of the proxy is gone,
docker logs shows no new lines.
Docker mounts a single file by its inode. sed -i and most editors write
a new file, so the container continues to read the old one. Write the file
in place, or restart the container.
Make a test account when registration is closed
With require_token = true, a new account needs a registration token.
An admin can make one with an access token. Without an admin token, do these steps:
- Add
shared_secret = "<random>"to[registration]and restart. - Register the account through
/_synapse/admin/v1/register. The MAC is the Synapse form: HMAC-SHA1 of the nonce, user, password andnotadmin, separated by NUL bytes. - Remove the secret and restart. The route then answers 404.
The secret can make admin accounts. Do not leave it in the file.
A release does not tag
release.yml tags main only when every check on that commit is
complete. The Matrix compliance suite takes about an hour. A dispatch
before then logs one of these lines and builds nothing:
CI is still running on <sha>; not releasing it yet.
The Rust quality gate has not passed on <sha>; not releasing it.
Wait for all the checks on the commit, then dispatch again.