View source for Lua for Rate Limiting

# Lua for Rate Limiting in MediaWiki

This article provides a comprehensive introduction to using Lua scripting for implementing rate limiting within a MediaWiki installation.  Rate limiting is crucial for protecting your wiki from abuse, such as spamming, denial-of-service (DoS) attacks, and excessive API usage.  This guide is geared towards beginners with some basic understanding of MediaWiki and a willingness to learn Lua.

== What is Rate Limiting? ==

Rate limiting controls the frequency with which users (or IP addresses) can perform specific actions within a given timeframe.  It's a fundamental security measure employed across many web applications.  In the context of a wiki, this can involve limiting:

*   Edits per minute/hour
*   API calls per minute/hour
*   Login attempts per minute
*   Creation of new pages per minute

Without rate limiting, a malicious actor could overwhelm your wiki with requests, potentially crashing it or filling it with unwanted content.  Effective rate limiting enhances the stability and security of your wiki environment.  Understanding [[User rights management]] is also crucial in conjunction with rate limiting.

== Why Use Lua for Rate Limiting? ==

MediaWiki 1.40 and later versions offer robust Lua scripting capabilities through the [[Scribunto]] extension.  Using Lua for rate limiting provides several advantages:

*   **Flexibility:** Lua allows for complex and customized rate limiting rules tailored to your wiki's specific needs.  You can define different limits for different user groups, actions, or even specific pages.
*   **Performance:** Lua is a lightweight and fast scripting language, minimizing the performance impact of rate limiting checks.  Compared to implementing rate limiting solely in PHP, Lua generally offers better performance.
*   **Maintainability:**  Lua code is relatively easy to read and maintain, making it simpler to update and modify your rate limiting rules as needed.
*   **Centralized Logic:** Lua modules can encapsulate rate limiting logic, promoting code reuse and reducing redundancy.  This ties into the broader concept of [[Extension development]].
*   **Integration with Existing Systems:** Lua can interact with other MediaWiki components and external data sources, allowing you to incorporate additional factors into your rate limiting decisions.

== Core Concepts ==

Before diving into the code, let's outline the core concepts involved in rate limiting:

*   **Identifier:**  This uniquely identifies the entity being rate limited.  Common identifiers include:
    *   User ID (for registered users)
    *   IP Address (for anonymous users)
    *   API Token (for API requests)
*   **Action:** The specific action being rate limited (e.g., "edit", "api_call", "login").
*   **Window:** The timeframe over which the rate is measured (e.g., 60 seconds, 3600 seconds).
*   **Limit:** The maximum number of actions allowed within the window.
*   **Storage:** A mechanism to store the number of actions performed by each identifier within each window.  Common storage methods include:
    *   [[mw.storage]] (MediaWiki's built-in storage system)
    *   External databases (e.g., Redis, Memcached) – requires additional extension support.

== Implementing Rate Limiting with Lua and mw.storage ==

This example demonstrates a basic rate limiting implementation using Lua and `mw.storage`. This approach is suitable for smaller wikis or as a starting point for more complex implementations.  Larger wikis might benefit from using an external database for improved scalability.

First, create a Lua module (e.g., `Module:RateLimit`).

```lua
local p = {}

local storage = mw.storage.new("ratelimit") -- Use mw.storage for persistence

local function get_key(identifier, action, window)
  -- Construct a unique key for each identifier, action, and window
  return identifier .. ":" .. action .. ":" .. window
end

function p.is_rate_limited(identifier, action, window, limit)
  -- Check if the identifier is rate limited for the given action and window
  local key = get_key(identifier, action, window)
  local count = tonumber(storage:get(key) or "0") -- Get the current count, default to 0

  if count >= limit then
    return true -- Rate limited
  else
    storage:set(key, count + 1) -- Increment the count
    return false -- Not rate limited
  end
end

function p.reset_rate_limit(identifier, action, window)
  -- Reset the rate limit for the given identifier, action, and window
  local key = get_key(identifier, action, window)
  storage:delete(key)
end

return p
```

**Explanation:**

1.  **`local p = {}`**: Creates a table `p` which will hold the module's functions.
2.  **`local storage = mw.storage.new("ratelimit")`**:  Creates a new storage object using `mw.storage`. The string "ratelimit" is the storage key, used to identify this specific storage area.  This allows the data to be persisted across page views.
3.  **`local function get_key(identifier, action, window)`**:  A helper function to construct a unique key for each rate limiting entry. This key is used to store and retrieve the count in `mw.storage`.
4.  **`function p.is_rate_limited(identifier, action, window, limit)`**:  This is the core function. It checks if the identifier has exceeded the rate limit.
    *   It retrieves the current count from `mw.storage` using the generated key. If the key doesn't exist, it defaults to 0.
    *   If the count is greater than or equal to the limit, it returns `true` (rate limited).
    *   Otherwise, it increments the count in `mw.storage` and returns `false` (not rate limited). `tonumber` ensures the value retrieved from storage is treated as a number.
5.  **`function p.reset_rate_limit(identifier, action, window)`**: This function allows you to manually reset the rate limit for a specific identifier, action, and window. This might be useful for administrative purposes or to handle exceptional cases.
6.  **`return p`**: Returns the table containing the module's functions.

== Integrating Rate Limiting into MediaWiki Hooks ==

Now, let's integrate this Lua module into MediaWiki using hooks.  We'll use the `EditPage::beforeSave` hook to rate limit edits.  You'll need to add this code to your `LocalSettings.php` file.

```php
<?php
// ... other settings ...

$wgHooks['EditPage::beforeSave'][] = 'myRateLimitEdit';

function myRateLimitEdit($editor, $article, $content, $summary, $editmode, $watchlistrights) {
  $title = $article->getTitle();
  $user = $editor->getUser();
  $identifier = $user->getId() ? $user->getId() : $editor->getIPAddress(); // Use user ID if logged in, IP otherwise
  $action = "edit";
  $window = 60; // 60 seconds
  $limit = 5; // 5 edits per minute

  $module = mw.loadModule('Module:RateLimit');
  if ($module.is_rate_limited(identifier, action, window, limit)) {
    // Rate limited!  Prevent the edit.
    $editor->getOutput()->addError("You have exceeded the edit rate limit. Please try again later.");
    return false; // Prevent the save
  }

  return true; // Allow the save
}
```

**Explanation:**

1.  **`$wgHooks['EditPage::beforeSave'][] = 'myRateLimitEdit';`**:  Registers the `myRateLimitEdit` function as a callback for the `EditPage::beforeSave` hook. This hook is triggered before an edit is saved.
2.  **`$title = $article->getTitle();`**: Gets the title of the page being edited.
3.  **`$user = $editor->getUser();`**: Gets the user object.
4.  **`$identifier = $user->getId() ? $user->getId() : $editor->getIPAddress();`**: Determines the identifier to use for rate limiting. If the user is logged in, it uses their user ID. Otherwise, it uses their IP address.
5.  **`$action = "edit";`**: Sets the action to "edit".
6.  **`$window = 60;`**: Sets the window to 60 seconds (1 minute).
7.  **`$limit = 5;`**: Sets the limit to 5 edits per minute.
8.  **`$module = mw.loadModule('Module:RateLimit');`**: Loads the Lua module `Module:RateLimit`.
9.  **`if ($module.is_rate_limited(identifier, action, window, limit))`**: Calls the `is_rate_limited` function in the Lua module with the appropriate parameters.
10. **`$editor->getOutput()->addError("You have exceeded the edit rate limit. Please try again later.");`**: If the user is rate limited, an error message is added to the output.
11. **`return false;`**:  Returns `false` to prevent the edit from being saved.
12. **`return true;`**:  Returns `true` to allow the edit to be saved.

== Advanced Considerations ==

*   **External Databases:** For larger wikis, consider using an external database (e.g., Redis, Memcached) to store rate limiting data. This provides better scalability and performance than `mw.storage`. You'll need to use a PHP extension to connect to the database from Lua.
*   **Different Rate Limits:** Implement different rate limits for different user groups (e.g., administrators, confirmed users, anonymous users).  Use [[User groups]] and conditional logic in your Lua code.
*   **API Rate Limiting:**  Protect your wiki's API by implementing rate limiting for API calls.  Use the `ApiRestBase::beforePost` or `ApiRestBase::beforeGet` hooks.  Consider using API tokens as identifiers.
*   **Dynamic Rate Limits:** Adjust rate limits dynamically based on server load or other factors.  This requires more complex logic and potentially external monitoring tools.
*   **Whitelisting:**  Allow certain users or IP addresses to bypass rate limiting.  This is useful for administrators or trusted users.
*   **CAPTCHAs:** Combine rate limiting with CAPTCHAs to further mitigate abuse.  [[reCAPTCHA]] is a popular option.
*   **Logging:** Log rate limiting events for auditing and analysis.  Use MediaWiki's logging mechanisms or a custom logging system.
*    **Consider using a sliding window counter:**  Instead of a fixed window, a sliding window counter provides more accurate rate limiting.  Implementations are more complex but provide better granularity.  See resources like [https://redis.io/docs/reference/data-types/sliding-window/](https://redis.io/docs/reference/data-types/sliding-window/) for examples.
*   **Using Bloom Filters:** Bloom filters can be used to efficiently check if an IP address or user ID has been seen before, reducing the load on the storage system. [https://en.wikipedia.org/wiki/Bloom_filter](https://en.wikipedia.org/wiki/Bloom_filter)
*   **Implement a backoff strategy:** When a user is rate limited, suggest a backoff time to avoid repeated attempts. [https://en.wikipedia.org/wiki/Exponential_backoff](https://en.wikipedia.org/wiki/Exponential_backoff)

== Monitoring and Tuning ==

After implementing rate limiting, it's crucial to monitor its effectiveness and tune the limits as needed.  Monitor:

*   The number of rate limiting events.
*   Server load and performance.
*   User feedback.

Adjust the limits based on this data to find the optimal balance between security and usability.  Tools like [[Prometheus]] and [[Grafana]] can be used for monitoring.  Analyzing [[Network traffic]] patterns can also help identify potential abuse.  Consider using [[Statistical analysis]] to determine optimal threshold values.  Look for [[Correlation]] between rate limiting events and other security incidents.  Utilize [[Data visualization]] to understand trends.  Employ [[Anomaly detection]] algorithms to identify unusual activity.  Regularly review [[Security logs]] for suspicious patterns.  Monitor [[Response times]] to ensure rate limiting doesn't negatively impact user experience.  Track [[Error rates]] to identify potential issues with the implementation. Utilize [[A/B testing]] to compare different rate limiting configurations.  Implement [[Alerting systems]] to notify administrators of potential problems.  Analyze [[User behavior]] to identify patterns and adjust limits accordingly.  Consider using [[Machine learning]] to predict and prevent abuse.  Employ [[Trend analysis]] to identify emerging threats.  Utilize [[Forensic analysis]] to investigate security incidents.  Monitor [[API usage]] to identify potential abuse.  Track [[Resource consumption]] to ensure rate limiting doesn't negatively impact server performance.  Analyze [[Log data]] for suspicious patterns.  Employ [[Threat intelligence feeds]] to identify known malicious actors.  Utilize [[Security information and event management (SIEM)]] systems for centralized logging and analysis.  Monitor [[System metrics]] to identify potential bottlenecks.  Track [[User activity]] to identify potential abuse.  Analyze [[Network flows]] to identify suspicious traffic.  Implement [[Intrusion detection systems (IDS)]] to detect and prevent attacks.  Utilize [[Vulnerability scanners]] to identify security weaknesses.  Employ [[Penetration testing]] to simulate attacks and identify vulnerabilities.  Monitor [[Database activity]] to identify potential abuse.  Track [[File uploads]] to prevent malicious content.  Analyze [[Page views]] to identify unusual traffic patterns.  Utilize [[Web analytics]] to understand user behavior.  Employ [[Content delivery networks (CDNs)]] to improve performance and reduce load.  Monitor [[Cache hit rates]] to ensure efficient caching.  Track [[DNS queries]] to identify potential threats.  Analyze [[SSL/TLS certificates]] to ensure secure connections.  Utilize [[Firewalls]] to block malicious traffic.  Employ [[Load balancers]] to distribute traffic and improve performance.  Monitor [[Server uptime]] to ensure high availability.  Track [[Disk space usage]] to prevent outages.  Analyze [[CPU usage]] to identify potential bottlenecks.  Utilize [[Memory usage monitoring]] to identify potential leaks.  Employ [[Process monitoring]] to identify rogue processes.  Monitor [[Network bandwidth usage]] to identify potential congestion. Track [[Latency]] to identify performance issues.  Analyze [[Packet loss]] to identify network problems. Utilize [[Network monitoring tools]] to identify potential threats.  Employ [[Security audits]] to identify vulnerabilities and weaknesses.  Monitor [[Compliance with security standards]] to ensure adherence to best practices.  Track [[Incident response times]] to ensure efficient handling of security incidents.  Analyze [[Root cause analysis]] to identify the underlying causes of security incidents. Utilize [[Post-incident reviews]] to learn from security incidents and improve security posture.

== Conclusion ==

Lua provides a powerful and flexible way to implement rate limiting in MediaWiki. By leveraging the `Scribunto` extension and `mw.storage`, you can protect your wiki from abuse and ensure its stability and security. Remember to carefully consider your specific needs and tune the limits accordingly.

[[Security]]
[[Extension:Scribunto]]
[[mw.storage]]
[[User rights management]]
[[Extension development]]
[[reCAPTCHA]]
[[Prometheus]]
[[Grafana]]
[[Network traffic]]
[[Statistical analysis]]
[[Data visualization]]
[[Anomaly detection]]
[[Security logs]]
[[A/B testing]]
[[Alerting systems]]

[[Category:Security]]

== Start Trading Now ==
[https://affiliate.iqbroker.com/redir/?aff=1085&instrument=options_WIKI Sign up at IQ Option] (Minimum deposit $10)
[http://redir.forex.pm/pocketo Open an account at Pocket Option] (Minimum deposit $5)
=== Join Our Community ===
Subscribe to our Telegram channel [https://t.me/strategybin @strategybin] to receive:
✓ Daily trading signals
✓ Exclusive strategy analysis
✓ Market trend alerts
✓ Educational materials for beginners