Retracker

  1. Home
  2. Wiki

A Retracker (also sometimes referred to as a pseudotracker) is a technology for optimizing BitTorrent traffic exchange within local area networks (LANs).[1][2] A retracker enables direct BitTorrent protocol connections between subscribers of the same Internet Service Provider (ISP) (or multiple ISPs with local traffic exchange agreements), where each subscriber uses an internal (local) IP address. Technically, a retracker is a very simple BitTorrent tracker, typically anonymous, operating within the address space of an ISP's local network. Using a retracker allows peers to exchange traffic at higher speeds using the ISP's internal local network addressing, as access to the local network for many Russian ISPs is permitted at a higher speed than internet access. Furthermore, using a retracker reduces the load on backbone links because subscribers do not need to download the same data separately.

For a BitTorrent client to take advantage of a retracker, the retracker's announce URL must be added to the torrent's list of announce URLs. Some trackers do this automatically, adding the addresses of authorized retrackers based on an analysis of the user's IP address. In other cases, users have to add the retracker's announce URL to .torrent files themselves or by using special programs called torrent patchers.[3] Some ISPs (see below) attempt to install transparent proxies that intercept HTTP requests to popular trackers. Such a proxy modifies .torrent files during download and adds a local retracker to them. Additionally, some trackers add the address "retracker.local"[4] to all .torrent files, which is then resolved to the IP of an active retracker by the ISP's DNS servers. Other solutions to the problem of local BitTorrent traffic exchange also exist.

It is important not to confuse retrackers with regular local trackers for LANs. A retracker does not aim to facilitate the exchange of .torrent files, organize them, enable searching, etc. A retracker serves exclusively as a means to increase data exchange speed between peers.

It should be noted that retracker support is only possible in multitracker-capable BitTorrent clients, such as µTorrent. Torrent clients that do not support multitracker functionality, like Vuze, will only contact the first tracker that responds; thus, the retracker will either not be used or only it will be used.

History of Retrackers

[edit]

On August 24, 2007, the idea of a retracker was described and implemented by a subscriber under the pseudonym unxed on the Corbina Telecom network (Saint Petersburg).[5] On September 21, 2007, the administration of the tracker rutracker.org came forward with a similar initiative.[6][7] However, the rutracker.org administration was highly critical[8] of attempts by local network users to organize their own retrackers and proposed its own centralized installation of retrackers on ISP technical platforms. The first retracker from rutracker.org began operating in the peering space of the TiERA home network in Saint Petersburg on February 5, 2008.[9]

The rutracker.org approach did not satisfy users whose ISPs were unwilling to officially install retrackers. Therefore, as an alternative to "official provider" retrackers, a system of local retrackers (Re-Tracker.ru, created by an initiative group known by the nicks FreeM@N (Dmitry) and Shift (Stanislav)) was launched in September 2008, again on the Corbina Telecom network in Saint Petersburg by users of that network. From October 2008, a similar system began operating in the Moscow network of Corbina Telecom, covering almost the entire city's local network.

Subsequently, the administration of the rutracker.org tracker abandoned the practice of adding individual retracker addresses based on the client's IP range. Instead, the address retracker.local is added to all torrents (thus, willing ISPs can redirect this address to their retracker via their DNS servers).

The retracker.local address is not supported in all local networks and, furthermore, does not work if the user uses DNS servers other than their ISP's. Therefore, the problem of creating user-run retrackers in local networks remains relevant.

System of Local (User-Run) Retrackers

[edit]

The system of local retrackers works as follows: a .torrent file downloaded by a user is intercepted by the Torrent Patcher program, after which the modified file is passed to a torrent client, such as µTorrent. During patching, lists of retrackers are added. The program also supports a "safe" method of adding retrackers via a Magnet link (works only on µTorrent version 1.8.1 and higher). After this, the torrent client makes requests to the trackers and to the local retrackers. After a request to a local retracker, it receives a list of local peers, enabling data transfer via the BitTorrent protocol at the speed of the ISP's local network. The local retrackers themselves are usually set up by enthusiast users.

If the network where a local retracker is installed uses dual addressing (i.e., a subscriber receives two IP addresses—a public one and an internal network one), the retracker must operate on the network interface with the internal network address. Otherwise, each retracker user would need to configure a static route so that traffic to the retracker goes through the local interface—otherwise, the retracker's advantages would be nullified.

Advantages

[edit]

Advantages of creating retrackers on subscriber computers (as opposed to retrackers installed on the ISP's platform):

  • Independence from both tracker administrations and ISP administrations. Consequently:
    • Ability to use the benefits of retrackers in networks where official retrackers are absent.
    • Ability for retrackers to work with any internet trackers (not just those with which the ISP "agrees").

Disadvantages

[edit]

Disadvantages of this approach:

  • Users are forced to add retracker addresses to torrents manually or install specialized torrent patchers.
  • In the event of a simultaneous failure of all user-run retrackers (which is, of course, highly unlikely), the system will cease to function, and subscribers' BitTorrent clients will generate meaningless traffic.

Support for Retrackers via the retracker.local Address

[edit]

The original implementation of the retracker idea was published on March 10, 2009, on the nag.ru website.[10] The essence is to automatically add an address like http://retracker.local/ to all torrents, either on the tracker side or on the BitTorrent client side. Thus, each ISP can independently specify the address of its retracker server through its DNS server settings (and if the ISP does not have a retracker server, user-run retrackers can be used by editing the hosts file). Since September 18, 2009, the popular tracker rutracker.org has been adding http://retracker.local to torrents for Russian IP addresses.[11]

Advantages

[edit]
  • In most cases, users do not need to perform additional actions, apply special settings, or install additional programs.
  • Trackers do not need to maintain IP range databases or expend resources on processing torrents.
  • No problems when an ISP changes its external addresses (it is sufficient to change the DNS record for retracker.local).

Disadvantages

[edit]
  • Using the .local TLD conflicts with Zeroconf services — mDNS (e.g., Avahi), making it impossible[12] to use the retracker.local address by default in OS X, Linux, and similar operating systems, and contradicts the Draft RFC "Multicast DNS" [13].
  • If a subscriber uses third-party DNS servers (OpenDNS, Google Public DNS, etc.) instead of the ISP's DNS servers, they will need to manually specify the retracker's IP address in their hosts file. However, this does not guarantee the constant operability of the retracker due to possible changes in the retracker.local IP address by the ISP. Also, such manipulations with the hosts file on a mobile computer will make it impossible to use the retracker when connecting to other networks.

Other Solutions for Local BitTorrent Traffic Exchange

[edit]

BEP 22

[edit]

A proposal for a BitTorrent protocol extension for automatic discovery of local trackers was published on bittorrent.org.[14] The popular BitTorrent client µTorrent supports this protocol extension by enabling the isp.bep22 option in the "Advanced" section of the settings window. In the latest version as of 2011-10-14 (version 3.0.25756), this option is enabled by default. To check if this technology is supported in your network:

  • Determine the hostname assigned to you by your ISP (e.g., using this service). For example, let it be 12-34-56-78.clients.provider.org
  • Type in the command line: `nslookup -q=srv _bittorrent-tracker._tcp.clients.provider.org` or `nslookup -q=srv _bittorrent-tracker._tcp.provider.org` (replace the "tail" of the string with your hostname).

If you receive an error message, the technology is not supported.

Ozerki.net ISP Solution

[edit]

The Saint Petersburg ISP Ozerki.net (later acquired by the operator Summa Telecom as part of the operator Severen-Home, which operated under the brand Tsifra) used its own technology for adding a retracker address to .torrent files. According to a network administrator's message, traffic from torrent trackers was redirected using the firewall iptables to a proxy server based on Squid, which in turn processed all .torrent files downloaded by users with a patcher program.[15]

SmartHome ISP Solution

[edit]

On November 13, 2009, the Saint Petersburg ISP SmartHome announced the launch of its implementation for intercepting and adding a retracker address, publishing a full description of the technology on the nag.ru forum[16], and also announced support for the isp.bep22 extension in its local network.[17]

Local Peer Discovery

[edit]

The "Local Peer Discovery" feature, present in the BitTorrent client µTorrent since version 1.7 build 1065 (released April 5, 2007),[18] provides capabilities similar to using retrackers. Since this function uses multicast packets, which are blocked by most ISPs to protect against virus epidemics, its application is limited to small local networks consisting of a single segment.

IPv6

[edit]

The transition to IP version 6 addresses the problem of local address connectivity, as the number of possible IPv6 addresses is so vast that it, in principle, eliminates the need for local IP addresses. However, not all software (both client and server-side) currently supports IPv6. As a temporary solution, tunneling protocols such as Teredo can be used. Starting with Windows Vista, Teredo support is enabled in the OS by default. For earlier versions, the µTorrent client for Windows OS offers the option to install Teredo automatically via the settings menu.

Interesting Facts

[edit]
  • The popular BitTorrent client µTorrent has a built-in anonymous tracker that can be used as a retracker. It can be enabled in the "Settings" menu, under "Advanced," by setting "bt.enable_tracker" to true. The tracker address will be http://your_IP:uTorrent_port/announce

Notes

[edit]
  1. ^ http://nag.ru/articles/reviews/15540/odichavshiy-wi-fi.html Archived 2011-06-13 at the Wayback Machine chapter "Torrents. Taming" (in Russian)
  2. ^ CARMA Based MST Approximation for Multicast Provision in P2P Networks Archived 2012-01-11 at the Wayback Machine doi:10.1109/ICNS.2010.25 quote:"Recently there have been some advances in the locality awareness for BitTorrent networks. Popular nationwide trackers (torrents.ru, for instance) have introduced so-called “retrackers” - dedicated secondary servers. These servers are optionally connected to primary database but mainly supposed to only return peer list local to specific network scope."
  3. ^ Torrent Patcher Programs Archived 2013-05-02 at the Wayback Machine
  4. ^ The introduction of retracker.local is discussed in topic 2234744 on rutracker.org (in Russian)
  5. ^ "On the introduction of a retracker (Corbina, Saint Petersburg)". Archived from the original on 2022-07-02. Retrieved 2008-12-20. (in Russian)
  6. ^ The idea was to determine the client's ISP based on IP address analysis and add the corresponding retracker address to the .torrent file. Only retrackers officially installed by ISPs (i.e., based on an agreement with the tracker administration) were added.
  7. ^ Optimization of BitTorrent traffic exchange in local networks. Discussion on rutracker.org (topic No. 405935) (in Russian)
  8. ^ Criticism from rutracker.org regarding independent solutions by home network users for local BitTorrent traffic exchange can be found in topic 405935 on rutracker.org (in Russian).
  9. ^ "Retracker from TiERA". Archived from the original on 2009-12-21. Retrieved 2010-02-22. (in Russian)
  10. ^ "The idea of supporting retrackers via the retracker.local address on nag.ru". Archived from the original on 2011-06-13. Retrieved 2011-06-23. (in Russian)
  11. ^ Automatic addition of the retracker.local address to all torrents was announced in a congratulatory message from the tracker administrator on its anniversary. (in Russian)
  12. ^ "Impossibility of using retracker.local in OS X and Linux". Archived from the original on 2013-06-05. Retrieved 2013-06-13. (in Russian)
  13. ^ "Draft RFC "Multicast DNS"". Archived from the original on 2009-11-25. Retrieved 2009-11-16.
  14. ^ "BitTorrent Protocol Extension Proposal". Archived from the original on 2012-05-07. Retrieved 2009-03-19.
  15. ^ "• View topic — Retracker retracker.ozerki.net". Archived from the original on 2009-06-27. Retrieved 2009-03-21. (in Russian)
  16. ^ "Description of the retracker address interception and addition technology from SmartHome ISP". Archived from the original on 2016-03-04. Retrieved 2010-01-16. (in Russian)
  17. ^ "SmartHome ISP announcement on isp.bep22 extension support". Archived from the original on 2013-07-31. Retrieved 2009-11-25. (in Russian)
  18. ^ "µTorrent 1.7 Changelog". Archived from the original on 2009-01-22. Retrieved 2008-12-21.
[edit]

See Also

[edit]