Mastodon Federation Rate Limiting: How to Avoid Sidekiq Spikes

Mastodon instances exchange posts and activities through federation. When a popular account on a remote instance posts, your instance must process that activity for every local follower. This can cause a sudden flood of jobs in Sidekiq, the background job processor, leading to high memory usage, queue backlogs, and degraded performance. The root cause is the lack of per-account rate limits on inbound federation traffic. This article explains how federation rate limiting works and provides concrete steps to configure Sidekiq and Mastodon settings to prevent spikes without blocking legitimate activity.

Why Federation Traffic Causes Sidekiq Spikes

Mastodon uses Sidekiq to process all background tasks, including inbound federation activities. When a remote account with many followers on your instance posts a status, Mastodon must create a job for each local follower. If that remote account posts multiple times in quick succession, Sidekiq receives a burst of identical jobs.

The default Sidekiq configuration processes jobs in FIFO order from a single queue. Without rate limiting, a federation spike can occupy all Sidekiq threads, delaying other important tasks like email delivery or media processing. The instance may become unresponsive to local users.

Mastodon version 3.5 introduced a per-account inbound rate limit. This limit applies to the number of activities a single remote account can deliver per second. When the limit is exceeded, Mastodon queues the activity for later processing instead of dropping it. This prevents spikes while still delivering all content.

Steps to Configure Federation Rate Limiting in Mastodon

The configuration involves two areas: the Mastodon application settings and the Sidekiq worker configuration. Both must be tuned together for the best results.

Step 1: Set the Inbound Rate Limit in Mastodon Environment Variables

1. Open the Mastodon environment file
   Navigate to your Mastodon installation directory and open the .env.production file using a text editor such as nano or vim.
2. Add or modify the INBOUND_RATE_LIMIT variable
   Set the value to the number of activities per second you want to allow from a single remote account. A common starting value is 5. Add this line: INBOUND_RATE_LIMIT=5. Save the file.
3. Restart the Mastodon web process
   Run systemctl restart mastodon-web or the equivalent restart command for your deployment method. This applies the new rate limit.

Step 2: Tune Sidekiq Concurrency and Queues

1. Edit the Sidekiq configuration file
   Open config/sidekiq.yml in your editor.
2. Set the concurrency value
   Reduce the concurrency to a value that matches your server CPU cores. For a small instance with 2 CPU cores, set concurrency: 5. For a larger instance with 8 cores, set concurrency: 15. This prevents Sidekiq from consuming all memory during spikes.
3. Reorder queues for priority
   Place the push and pull queues after default and mailers. Example: queues: [default, mailers, push, pull, scheduler]. This ensures local user actions and emails are processed before federation traffic.
4. Restart the Sidekiq service
   Run systemctl restart mastodon-sidekiq to apply the changes.

Step 3: Adjust Federation Retry and Backoff Settings

1. Add retry delay variables to .env.production
   Add these lines: FEDERATION_RETRY_DELAY=30 and FEDERATION_MAX_BACKOFF=3600. The first sets the initial retry delay in seconds. The second sets the maximum backoff in seconds. These values reduce the number of repeated retries for failing federation jobs.
2. Restart the Mastodon web process
   Run systemctl restart mastodon-web again to load the new variables.

Common Configuration Mistakes and Their Fixes

Sidekiq Memory Usage Still Spikes After Setting Rate Limits

If Sidekiq memory remains high, the concurrency value may still be too high for your server. Reduce concurrency further, for example to 3 for a 2-core server. Also check that the Sidekiq process is not running with the -t flag that disables timeout. Remove -t if present in the systemd service file.

Inbound Rate Limit Prevents All Federation Activity

Setting the INBOUND_RATE_LIMIT too low, such as 1, can cause delays for all remote accounts. Increase the value to 10 or 20 if you notice that remote posts appear hours late. Monitor the Sidekiq queue length using Sidekiq::Queue.new.size in a Rails console to find a balanced value.

Federation Jobs Are Dropped Instead of Queued

If jobs disappear before processing, the Sidekiq process may be restarting too often. Increase the Sidekiq::Config::DEFAULT_JOB_OPTIONS timeout in the Sidekiq initializer. Alternatively, set Sidekiq.default_worker_options = { 'backtrace' => true, 'retry' => 5 } in config/initializers/sidekiq.rb to ensure retries.

Mastodon Federation Rate Limiting vs Sidekiq Queue Tuning

Item	Inbound Rate Limit	Sidekiq Queue Tuning
Purpose	Limits activities from a single remote account per second	Controls how many threads process jobs and in what order
Configuration location	.env.production	config/sidekiq.yml
Effect on spikes	Reduces the number of jobs entering the queue	Limits how many jobs run simultaneously
Risk	May delay delivery if set too low	May cause backlogs if concurrency is too low
Best used together	Yes	Yes

Mastodon federation rate limiting and Sidekiq queue tuning complement each other. The inbound rate limit reduces the volume of jobs, while Sidekiq tuning ensures the server can process remaining jobs without memory exhaustion. Use both for maximum stability.

You can now configure Mastodon to handle federation spikes without crashing Sidekiq. Start by setting the INBOUND_RATE_LIMIT to 5 and Sidekiq concurrency to match your CPU cores. Monitor the Sidekiq dashboard for queue length and memory usage after the changes. For advanced tuning, consider adding a separate Sidekiq process dedicated to the pull queue to isolate federation traffic from other jobs.