Your Mastodon instance uses cryptographic public keys to sign every message sent to other servers in the fediverse. When those keys expire or leak, you must rotate them to maintain secure federation. However, rotating a public key incorrectly can break communication with thousands of remote instances, causing posts, boosts, and follows to fail silently. This article explains why key rotation causes federation disruptions and provides a step-by-step method to rotate Mastodon’s public keys without losing connectivity.
Key Takeaways: Rotate Mastodon’s Federation Public Keys Without Causing Outages
- Sidekiq queues and remote key cache flush: Clear the cached public keys on remote servers by restarting Sidekiq after rotation to force re-fetch.
- Rails console key rotation command: Use
Account.find_local('instance').update!(public_key: OpenSSL::PKey::RSA.new(2048).public_key)to rotate a single account’s key without affecting others. - Federation worker retry behavior: Remote instances automatically retry failed deliveries up to 7 times over 48 hours, so a gradual key rotation window is safe.
Why Rotating a Mastodon Public Key Breaks Federation
Every Mastodon account has an RSA public-private key pair stored in the database. When your instance sends an activity to a remote server, it signs the HTTP request with the private key. The remote server looks up the public key from your account’s Actor object (retrieved via ActivityPub) and verifies the signature. If you rotate the key on your instance but the remote server still holds the old public key, all future signed requests will fail verification. The remote server rejects the activity, and your posts, boosts, and follows never appear on that server. This state persists until the remote server fetches your updated Actor object, which contains the new public key. The delay can last hours because remote servers cache Actor objects aggressively to reduce load.
Mastodon does not broadcast key changes automatically. The protocol has no standard message to say “my key has changed.” Instead, the remote server must re-fetch your Actor object. The re-fetch happens only when the remote server receives an activity from you and the cached signature verification fails. This creates a catch-22: the remote server cannot verify the signature, so it rejects the activity, and therefore never learns the new key. The fix is to force the remote server to re-fetch your Actor object before or immediately after the key rotation.
What Happens to Existing Signed Items After Rotation
Activities already delivered and accepted by remote servers remain valid. The remote server stores the activity content, not the signature. Key rotation only affects future deliveries. Old posts, boosts, and follows that were already accepted remain visible on remote instances. However, any pending deliveries in your Sidekiq queue that were signed with the old key will fail after rotation. Those jobs must be retried after the remote servers have the new key.
Steps to Rotate Mastodon Federation Keys Without Breakage
The following method rotates the public key for a single account while minimizing federation downtime. Perform these steps during a maintenance window with low traffic.
- Back up the current key pair
Log into your Mastodon server via SSH. Navigate to the Mastodon directory. Runcd /home/mastodon/liveand thenRAILS_ENV=production bin/tootctl accounts rotate [username]. This command rotates the key using Mastodon’s built-in tool. It also sends a forced re-fetch signal to remote servers that have recently interacted with the account. This is the safest method because it updates the Actor object immediately. - Verify the key update on the local Actor object
Open a Rails console:RAILS_ENV=production bin/rails c. Runa = Account.find_local('username'); puts a.public_key. Confirm the key string is different from the backup you saved earlier. This verifies the rotation succeeded. - Clear the local Sidekiq queue for federation deliveries
Remote servers that already failed to deliver an activity may have the old key cached. Restart Sidekiq to clear the in-memory cache:systemctl restart sidekiq. This forces Sidekiq to reload the account’s public key from the database before retrying any failed deliveries. - Force a re-fetch from remote servers
Send a dummy activity to a few major remote instances to trigger a re-fetch. Use the Mastodon web interface to post a public toot mentioning@instanceadmin@remoteinstance.social. The remote server will attempt to verify the signature, fail, fetch the updated Actor object, and cache the new key. After this, all subsequent deliveries will succeed. - Monitor federation logs for signature failures
Check the Mastodon logs:journalctl -u mastodon-web -f | grep "signature". Look for lines containing “signature verification failed.” If you see failures more than 10 minutes after step 4, the remote server may not have re-fetched. Repeat step 4 with a different remote instance. - Verify key rotation on a remote instance
Use a Mastodon client on a different server. Open the account profile page of the rotated account. The public key URL is athttps://yourinstance.social/users/username#main-key. A remote admin can fetch this URL directly to see the new key. If the key matches what you see in your Rails console, the rotation is visible to the fediverse.
Alternative Method: Manual Key Update via Rails Console
If the built-in tootctl command is unavailable or you need to rotate only the public key without changing the private key, use the Rails console directly. This method is riskier because it does not trigger the forced re-fetch signal. Use it only if you understand the consequences.
- Generate a new RSA key pair
In the Rails console, runnew_key = OpenSSL::PKey::RSA.new(2048). This creates a new private key. The public key is derived from it. - Update the account record
RunAccount.find_local('username').update!(public_key: new_key.public_key). This overwrites the public key in the database. The private key remains unchanged in the database, so you must also update it:Account.find_local('username').update!(private_key: new_key.to_pem). If you change only the public key, signature generation will fail because the private key no longer matches. - Restart Mastodon web and Sidekiq
Runsystemctl restart mastodon-web sidekiq. This forces the application to reload the account keys from the database. - Force re-fetch manually
Because the manual method does not send a re-fetch signal, you must rely on the first activity reaching a remote server. Post a public toot immediately. Monitor the logs for signature failures as described in the main method step 5.
If Mastodon Still Has Federation Issues After Key Rotation
Remote Servers Still Reject Activities After 24 Hours
Some remote instances cache Actor objects for up to 48 hours. If you rotated the key and remote servers still reject activities after 24 hours, the remote server may have a bug in its caching logic. Contact the remote instance admin and ask them to manually flush the cache for your account’s Actor object. Alternatively, post a public toot that includes a link to your account’s Actor object URL. Some servers will re-fetch when they see the URL in a toot.
Sidekiq Queue Backlog After Rotation
After rotation, Sidekiq may have thousands of failed delivery jobs in the retry queue. These jobs were signed with the old key. They will continue to fail until the remote server re-fetches your key. To speed up recovery, clear the retry queue and let the system re-create the jobs: RAILS_ENV=production bin/tootctl sidekiq clear. This removes all pending jobs, including retries. New activities will be created and delivered with the new key. This is safe because Mastodon regenerates missed activities from the database when needed.
Key Rotation Does Not Take Effect for Some Accounts
If you rotated the key for one account but other accounts on the same instance are unaffected, the issue is isolated to that account. Verify that the account’s public key in the database is unique. Mastodon stores each account’s key pair separately. If the database update failed silently, run the rotation command again and confirm the key changed using the Rails console method described in step 2 of the main method.
| Item | Built-in tootctl Rotation | Manual Rails Console Rotation |
|---|---|---|
| Description | Uses Mastodon’s official key rotation tool | Directly updates database fields via Rails console |
| Forced re-fetch signal sent | Yes | No |
| Risk of federation breakage | Low | High |
| Requires server access | SSH and tootctl | SSH and Rails console |
| Best use case | Routine key rotation | Emergency key change when tootctl is broken |
You can now rotate your Mastodon instance’s federation public keys with minimal disruption by using the built-in tootctl command and forcing a re-fetch from remote servers. For routine rotations, schedule them during low-traffic hours and monitor the federation logs for 30 minutes afterward. As an advanced tip, set up a monitoring alert for the signature verification failed log pattern so you can detect key-related issues before users report them.