RFC-Internet-DraftRetroShare Protocol (RSSIMFTP) August 2007
RetroShare Serverless Instant Messaging and File Transfer Protocol - short: RETROSHARE PROTOCOL (RS-SIM-FTP)
(so called: “JABBER-2” / “XMPP-2” or “SMPP”/“SMP(-FT)P”
Serverless Messaging & Presence & Filetransfer Protocol)
draft-ietf-RETROSHARE-PROTOCOL-RSSIMFTP-(JABBER2)-01.txt
Status of this Memo for the retroshare protocol (aka “Jabber 2”/SMPP):
This document specifies an internet standard for the purpose of using serverless messaging and encrypted file transfer and provides information for the internet community, which is requested for discussion and suggestions for improvements. Distribution of this memo is free, open source and unlimited.
By submitting this Internet-Draft or any updates, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English, other than to extract optional sections as-is for separate use. Citations for any scientific usage are allowed, if marked as a “work in progress” ietf-draft for the RETROSHARE PROTOCOL.
This document may be posted in an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other improved documents at any time. Discussion and suggestions for improvements are requested to upgrade the "work in progress."
The list of current Internet-Drafts can be accessed at
The list of Internet-Draft Shadow Directories can be accessed at
This Internet-Draft will expire six months after the publish date and was sent at the given date to and .
The discussion about this paper takes place in the retroshare forum here: or the retroshare mailinglist: and on the jabber-mailinglists: and the mailinglist.
Submissions for requests for comments should be sent to the retroshare forum:
Copyright Notice
See Copyright statement in the end.
Abstract
RetroShare is a serverless Instant Messenger with File-Transfer and -Sharing functions.
The RetroShare Serverless Instant Messaging and File Transfer Protocol (RSSIMFTP) - short: RETROSHARE PROTOCOL – is described here in a standardized form to write down the methods, functions and features the protocol and the RetroShare.sf.net-friendslist-application use by default or optional.
Messaging is here defined as sending private Instant Messages, Emails, Groupchat Messages, Channel Messages (like Mailinglists or Newsgroups) and OPTIONAL Video-streams or Voice-over-IP-Calls.
File Transfer is defined as sending Files to friends from the friendslist or to download files – either transferred from one friend or swarmed from several friends (and/or friends of friends).
The main new features of this Message and/or File Transfer is, that it is a network total decentral in the hand of the users and works without any servers or account registering at a central authority (server). Second, each transfer is encrypted by a symmetric RSA-Key exchange.
Search (for files or a database in general, e.g. URLs), Chat and (Email- or Instant) Message (to predefined friends in the friendslist) are the core competencies of a serverless friendslist.
While RETROSHARE PROTOCOL provides a generalized, extensible framework for exchanging messaging and filetransfer data, it is used mainly for the purpose of building instant messaging presence and filetransfer applications without servers that meet the requirements of the serverbased-defined RFC 959 (“FTP”), RFC 2779 (“IM”), and the more developed RFC 3921 (“XMPP”, “Jabber”). So, this RFC-document summarizes the given main functions in a serverless way - as an additional serverless and encrypted standard. Jabber2 offers a complementary view on Jabber and for serverless filetransfer the same.
Terminology and Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119.
Table of Contents
1. Introduction for the RetroShare Protocol
2. Serverless Distributed Hashtable (DHT) & Connecting
2.1. DHT-Bootstrap
2.2. TCP-Connections
2.3. UDP-Connections: UDP OpenSSL and Proxy Connection System.
2.4. Ports
2.5. Firewall
2.6. Router/NAT
2.7. UpnP
3. Encryption and Authentication
3.1. Encryption
3.2. OpenSSL: Interface to OpenSSL / (X)PGP Encryption & Authentication.
3.3. PQI / PEM Files: The base classes used throughout Retroshare/PQI.
3.4. Streaming functionality:
3.5. PQI-Exchange:
3.6. PQI-Text-Certificates-Exchange
3.7. Managing Subscriptions / Add a Friend and other Utility Functions:
4. Network and Session Establishment - OpenSSL Connection to a Peer:
4.1. Web of Trust & Neighbour Discovery Service:
4.2. Messenger Status & Exchange Presence - Classes to combine the low level interfaces:
4.3. Block Communication
5. Message
5.1. Personal Message Exchange
5.2. Groupchat
5.3. Email 2.0
5.4. Videochat
5.5. VOIP
5.6. Sending Messages in Channels
6. Filetransfer
6.1. Hash Verfication & Search
6.2. Protocol for Searches - Loopback Interfaces
6.3. Partial Data (Chunks)
6.4. Search Friends Database: XML-Libraries
6.5. Incoming Directory
7. Search Friends of Friends Database: Hopping (OPTIONAL)
7.1. Turtle F2F (OPTIONAL)
7.2. Imule hopping hybridity in the application (OPTIONAL)
7.3. OPTIONAL: Chunk Cache
8. OPTIONAL INTEGRATIONS
8.1. OPTIONAL: Amule/Imule-Implementation......
8.2. OPTIONAL: Thunderbird Implementation......
8.3. OPTIONAL: Implementation into Jabber Messenger (e.g. any c++ written jabber messenger or any with a QT Gui)
8.4. OPTIONAL: Multimessenger SIM......
8.5. OPTIONAL: OpenOffice Component (Messenger & Email)......
8.6. OPTIONAL: Improve the class streaming/Add VoIP and Video-Streaming capabilities
8.7. OPTIONAL: Add virtual P2P network adapter (VPN-like) for Gaming / Apps.
8.8. OPTIONAL: Search Extension Database: a p2p website index search-engine
9. User Interface & Program Interface Considerations
10. Security Considerations
11. IANA Considerations
12. Conclusions
13. Acknowledgments
APPENDIX A: First Appendix
A.1. Retroshare Website and Forum for discussing this paper
14. References
Author's Address......
Intellectual Property Statement......
Disclaimer of Validity......
1. Introduction for the RetroShare Protocol
Presence and Instant Messaging, Email, (Video) Chat and Voicecalls – short: online and offline communication – is one of the main usage of the Internet. File Transfer and File Sharing is especially in a peer-to-peer way the second main usage of the internet.
Till now the usage of communication over online/internet technology or file transfer is done with servers or unencrypted.
The here described RETROSHARE PROTOCOL introduces a serverless and encrypted standard protocol for online communication and file transfer.
This document defines a minimal set of requirements that a retroshare protocol implementation for serverless and encrypted file transfer and messaging in the sense of online communication must meet.
Serverless initiation of online communication or file transfers between two friends (or among friends) have been mentioned less in RFC documents. This document offers a documentation for a serverless alternative of online communication and file transfer / file sharing.
As servers not only can initiate a service or communication, they also can analyze or even log the communication and communication behaviour of the users. Second, users are depending on an account registration and a central service for communication and data packet exchange.
A Quotation of the RFC 2779: “Instant Messaging / Presence Protocol Requirements” explains the importance of encryption:
2.5.1. The protocol MUST provide means to ensure confidence that a
received message (NOTIFICATION or INSTANT MESSAGE) has not been
corrupted or tampered with.
2.5.2. The protocol MUST provide means to ensure confidence that a
received message (NOTIFICATION or INSTANT MESSAGE) has not been
recorded and played back by an adversary.
2.5.3. The protocol MUST provide means to ensure that a sent message
(NOTIFICATION or INSTANT MESSAGE) is only readable by ENTITIES that
the sender allows.
Many message and file transfer protocols do not exclude server adminstrators or Internet Service Providers (ISP) in reading the packets. Recently added encryption services to serverbased online communication or file sharing applications are forwarding the encrypted packets over a server, though then the server is a waste and bandwidth redundancy – why then not transferring directly from user to user…
In shortness: Users need, want and use, and second the modern technical applications offer this more and more: a serverless alternative to serverbased communication services.
The RETROSHARE PROTOCOL RFC documented both for the main internet transactions – online communication and file transfer –: an encrypted and serverless protocol specification.
Let´s start with the serverless identification of the IP-Adress of the online friend:
2. Serverless Distributed Hashtable (DHT) & Connecting
Retroshare Instant Messenger generates by the KAD-C library a client hash, which is stored in a Distributed Hash Table (DHT) [ For this purpose each installation has a kad.ini file stored, with IP-Adresses and client hashes taking part in the DHT.
Example: 75.24.217.31 / 10355 / 00aa7968454821d83cc47f470f5f113
Description: IP-Adress / Port / Client Hash
The DHT is the technical method, to find out the actual IP adress of online-friends.
2.1. DHT-Bootstrap
This is a so called bootstrap method to the DHT: The client hash then is stored with the actual IP-adress within the DHT. Other Retroshare Instant Messenger clients do the same.
As the DHT is searchable for hashes of the friends clients, the actual IP address of the friends in the own friendslist could be found.
A ping to them reveals the online or offline status of them.
With this DHT Method no central servers are needed to get the presence information or the IP address of a friend in the own friendslist.
The code-class for the DHT-Bootstrap is:
retroshare/rs-core/src/dht: dynamic hash table (kadc) interface.
Michael Rogers and Saleem Bhatti write in their Survey of Private Peer-to-Peer Networks “Retroshare is an instant messaging and file sharing network that uses a distributed hash table for address discovery. Users can communicate indirectly through mutual friends and request direct connections. Other applications allow friends of friends to communicate in a similar way, but they do not use distributed hash tables for address discovery, so addresses and encryption keys must be exchanged out-of-band or through mutually trusted friends”[ROGERS].
2.2. TCP-Connections
The Transmission Control Protocol (TCP) is one of the core protocols of the Internet protocols, simply referred to as TCP/IP. Using TCP, the retroshare applications on networked hosts can create connections to one each another, over which they can exchange streams of data using Stream Sockets. TCP retroshare connections have three phases: 1. connection establishment, 2. data transfer, 3. connection termination (see as well RFC 1323 / RFC 4614 / RFC 675).
2.3. UDP-Connections: UDP OpenSSL and Proxy Connection System.
User Datagram Protocol / Universal Datagram Protocol (UDP) is one of the core protocols: Using UDP, retroshare can send short messages sometimes known as datagrams (using Datagram Sockets) to one another.
A combination of Proxy Tunnel Service to initiate connections, and a OpenSSL UDP system (uses TCP-On-UDP library).
This is designed to enable firewalled peers to connect directly to each other and requires a third peer to proxy the connections.
The core-classes are:
pqissludp.cc/.h
pqistunner.cc/.h
pqiproxy.cc/.h
pqiudpproxy.cc/.h
pqitunnelproxy.cc/.h
pqitunnelproxyudp.cc/.h
retroshare/rs-core/src/tcponudp: networking.
These classes provide a full TCP over UDP implementation, with OpenSSL BIO for secure transport.
2.4. Ports
The retroshare protocol uses two ports for connections: 7812 for tcp and the next one up for udp: 7813. These are the ports by default, they can be adjusted to any port and the next one up.
2.5. Firewall
If a user is behind a firewall, and is able to make an exception for retroshare to the firewall, this is the easiest way to connect to the friends. Then the firewalled user can contact the non-firewalled node and connect. If both users are behind a firewall, at least one should make an exception in the firewall. If this is not possible, then a third friend of both, which is non-firewalled, can initiate the connection for them.
2.6. Router/NAT
If anyone is using a router/nat and wants to connect, then there must be two ports (e.g. 7812 and 7812) forwarded to the local machine. Users which have no knowledge in adjusting their router, can use in retroshare options “use UpnP” to adjust the router forwarding automatically.
2.7. UpnP
Universal Plug and Play (UPnP) is a set of computer network protocols used in retroshare client. The goals of UPnP are to allow devices to connect seamlessly and to simplify the implementation of networks in the home (data sharing, communications, and entertainment) and corporate environments. UPnP achieves this by defining and publishing UPnP device control protocols built upon open, Internet-based communication standards. The classes for that are found here:
retroshare/rs-core/src/upnp: automatic port routing (miniupnpc) interface.
Furthermore some additional tools help the processes and an old gui is provided as well, especially for linux machines with less than 300 MHZ :
retroshare/rs-core/src/util: various utilities.
retroshare/rs-core/src/fltkgui: old V0.2.1 FLTK GUI program.
3. Encryption and Authentication
Every communication or file transfer over the retroshare protocol is encrypted and allows to connect to trusted friends only over an approval of friends signed as “authenticated” or even “trusted”.
3.1. Encryption
Retroshare uses symmetric key encryption. This means, any user generates a private and public key, and the public key has to be exchanged with a friend. Both users need this to do and accept the key of the other one. This means, that a key is transferred in a different information channel - by usb-stick or by (password protected) email or any classic messenger. This means to be more safe and secure than sending an encryption key over the same channel like in asymmetric encrypting networks, e.g. with end-to-end encryption with all the problems of a node in the middle attack.
3.2. OpenSSL: Interface to OpenSSL / (X)PGP Encryption & Authentication.
For the encryption at the moment a modification of OpenSSl is used, called XPGP-certificates. But this is OPTIONAL as well as using PGP-Keys, which are soon implemented due to the roadmap.
This has the side effect to define “email 2.0”, that any conventional “@”-based email with a footer signature of a PGP Key can be used to message this PGP-Key of the friend over the retroshare protocol. Email 2.0 combines here offline and online communication among PGP-certificates over the retroshare protocol, aka Jabber2.(See below, as well the retroshare wiki and the code-classes for further details.)
The code basis is:
xpgpcert.cc/.h
Classes/Terms:
cert: Key Class for handling your peers.
sslroot: Manages OpenSSL side of things, Saves/Loads/Authenticates certificates. Hashs/Signs/Verfies Data. >
So not only the best way in encryption is given, as well additionally each certificate has to be signed/approved by both nodes, before they can communicate, this is a MUST and creates a spam-free and secure environment for the communication between both nodes.
3.3. PQI / PEM Files: The base classes used throughout Retroshare/PQI.
The generated certificate (public and private key) (with which encryption class ever: XPGP or PGP OpenSSL) has the result to maintain a PQI-File. The PQI file is a file, which contains the encryption key, which OPTIONAL can be displayed as a XPGP-PGP-Signature in plain text.
So it is possible to send the certificate text as plain text in an email or message or to append it as a footer message in any email.
The code basis for that is:
pqi.cc/h
pqi_base.cc/.h
pqi_data.h
Classes/Interfaces:
PQItem : Base of all the streamed classes. inherited by PQFileItem, MsgItem, ChatItem, etc.
Person: Generic Identity of a Peer with Network addresses etc. inherited by "cert" which is associated with a Certificate.
PQInterface: Basic Exchange Interface.
NetInterface: High Level Connection Interface.
BinInterface: Basic Binary Streaming Interface.
NetBinInterface: standard binary network interface.
P3Interface: High-Level Interface for Retroshare.
3.4. Streaming functionality:
Retroshare has as well a streaming functionality implemented, which allows to send files, emails or data packets in general in a stream over to the friend(s).
The code basis for that is:
pqipacket.cc/.h streaming functions.
pqistreamer.cc/.h Implements a PQ-Interface, transforming the PQ-Item classes into binary Data Packets, and passing them into a BinInterface.
pqitunnel.cc/.h Generic packet tunnelling. This allows pqitunneltst.cc/.h & other services to built on top of the base system.
This allows to add e.g. Video and Voice Streams over the retroshare protocol, which is essential to built a Video Chat or internetphone (VoIP) among the friends.
3.5. PQI-Exchange:
Each user has to exchange the PQI file (which is the PGP (XPGP) certificate/signature) with a friend. This can be done over any other communication channel like email, traditional messenger, usb-stick etc.
Additionally each friend is sending all know PQI-Files of the friends to all friends as well, but as they are not approved by them and there is no symmetric exchange or approval, the communication is denied.
Though, the names of the friend or the certificates are delivered to friends of the friend as well.
This guarantees that a communication can only be established among approved friend from both sides and on the other hand it supports the growth of the network. Two persons, which meet at a party of a common friend, can approve online after the party, they need not to exchange the certificates individually, as the common friend deliver the certificate of both to them. They only need to approve each other for communication. This describes the social networking type of the retroshare protocol and is OPTIONAL.
3.6. PQI-Text-Certificates-Exchange