From 8d0811a0f32b3005424763fe39788e14d4162ab6 Mon Sep 17 00:00:00 2001 From: iphydf Date: Fri, 9 Feb 2024 00:59:38 +0000 Subject: [PATCH] docs: Run prettier-markdown on markdown files. --- .restyled.yaml | 5 +- INSTALL.md | 201 +++++++++++++++++++++++++++----------------- README.md | 50 ++++++----- docs/Group-Chats.md | 107 ++++++++++++++++------- docs/apidsl.md | 51 ----------- docs/av_api.md | 129 +++++++++++++++++----------- docs/minpgc.md | 109 ++++++++++++------------ 7 files changed, 372 insertions(+), 280 deletions(-) delete mode 100644 docs/apidsl.md diff --git a/.restyled.yaml b/.restyled.yaml index b53f949019..123ef65efe 100644 --- a/.restyled.yaml +++ b/.restyled.yaml @@ -1,7 +1,9 @@ --- exclude: - # shfmt doesn't support this file + # shfmt doesn't support this file. - "other/analysis/run-clang-tidy" + # Generated file. + - "CHANGELOG.md" restylers: - astyle: @@ -17,6 +19,7 @@ restylers: include: - "**/*.cc" - "**/*.hh" + - prettier-markdown - prettier-yaml - reorder-python-imports - shellharden diff --git a/INSTALL.md b/INSTALL.md index 8bb901bd5b..0d515b3f6b 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,6 +1,8 @@ # Installation instructions -These instructions will guide you through the process of building and installing the toxcore library and its components, as well as getting already pre-built binaries. +These instructions will guide you through the process of building and installing +the toxcore library and its components, as well as getting already pre-built +binaries. ## Table of contents @@ -30,37 +32,44 @@ These instructions will guide you through the process of building and installing #### Main -This repository, although called `toxcore`, in fact contains several libraries besides `toxcore` which complement it, as well as several executables. However, note that although these are separate libraries, at the moment, when building the libraries, they are all merged into a single `toxcore` library. Here is the full list of the main components that can be built using the CMake, their dependencies and descriptions. - -| Name | Type | Dependencies | Platform | Description | -|------------------|------------|------------------------------------|----------------|----------------------------------------------------------------------------| -| `toxcore` | Library | libsodium, libm, libpthread, librt | Cross-platform | The main Tox library that provides the messenger functionality. | -| `toxav` | Library | libtoxcore, libopus, libvpx | Cross-platform | Provides audio/video functionality. | -| `toxencryptsave` | Library | libtoxcore, libsodium | Cross-platform | Provides encryption of Tox profiles (savedata), as well as arbitrary data. | -| `DHT_bootstrap` | Executable | libtoxcore | Cross-platform | A simple DHT bootstrap node. | -| `tox-bootstrapd` | Executable | libtoxcore, libconfig | Unix-like | Highly configurable DHT bootstrap node daemon (systemd, SysVinit, Docker). | +This repository, although called `toxcore`, in fact contains several libraries +besides `toxcore` which complement it, as well as several executables. However, +note that although these are separate libraries, at the moment, when building +the libraries, they are all merged into a single `toxcore` library. Here is the +full list of the main components that can be built using the CMake, their +dependencies and descriptions. + +| Name | Type | Dependencies | Platform | Description | +| ---------------- | ---------- | ---------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------- | +| `toxcore` | Library | libsodium, libm, libpthread, librt | Cross-platform | The main Tox library that provides the messenger functionality. | +| `toxav` | Library | libtoxcore, libopus, libvpx | Cross-platform | Provides audio/video functionality. | +| `toxencryptsave` | Library | libtoxcore, libsodium | Cross-platform | Provides encryption of Tox profiles (savedata), as well as arbitrary data. | +| `DHT_bootstrap` | Executable | libtoxcore | Cross-platform | A simple DHT bootstrap node. | +| `tox-bootstrapd` | Executable | libtoxcore, libconfig | Unix-like | Highly configurable DHT bootstrap node daemon (systemd, SysVinit, Docker). | | `cmp` | Library | | Cross-platform | C implementation of the MessagePack serialization format. [https://github.com/camgunz/cmp](https://github.com/camgunz/cmp) | #### Secondary -There are some programs that are not built by default which you might find interesting. You need to pass `-DBUILD_FUN_UTILS=ON` to cmake to build them. +There are some programs that are not built by default which you might find +interesting. You need to pass `-DBUILD_FUN_UTILS=ON` to cmake to build them. ##### Vanity key generators Can be used to generate vanity Tox Ids or DHT bootstrap node public keys. -| Name | Type | Dependencies | Platform | Description | -|---------------------------|------------|-----------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `cracker` | Executable | libsodium, OpenMP | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which starts with a specified byte sequence. Multi-threaded. | -| `cracker_simple` | Executable | libsodium | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which starts with a specified byte sequence. Single-threaded. | -| `strkey` | Executable | libsodium | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which contains a specified byte sequence at a specified or any position at all. Single-threaded. | +| Name | Type | Dependencies | Platform | Description | +| ---------------- | ---------- | ----------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `cracker` | Executable | libsodium, OpenMP | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which starts with a specified byte sequence. Multi-threaded. | +| `cracker_simple` | Executable | libsodium | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which starts with a specified byte sequence. Single-threaded. | +| `strkey` | Executable | libsodium | Cross-platform | Tries to find a curve25519 key pair, hex representation of the public key of which contains a specified byte sequence at a specified or any position at all. Single-threaded. | ##### Key file generators -Useful for generating Tox profiles from the output of the vanity key generators, as well as generating random Tox profiles. +Useful for generating Tox profiles from the output of the vanity key generators, +as well as generating random Tox profiles. | Name | Type | Dependencies | Platform | Description | -|---------------------------|------------|-----------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| ------------------------- | ---------- | --------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `make-funny-savefile` | Script | python | Cross-platform | Generates a Tox profile file (savedata file) with the provided key pair. | | `create_bootstrap_keys` | Executable | libsodium | Cross-platform | Generates a keys file for tox-bootstrapd with either the provided or a random key pair. | | `create_minimal_savedata` | Executable | libsodium | Cross-platform | Generates a minimal Tox profile file (savedata file) with either the provided or a random key pair, printing the generated Tox Id and secret & public key information. | @@ -69,10 +78,10 @@ Useful for generating Tox profiles from the output of the vanity key generators, ##### Other -| Name | Type | Dependencies | Platform | Description | -|---------------------------|------------|-----------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `bootstrap_node_info` | Script | python3 | Cross-platform | Prints version and Message Of The Day (MOTD) information of the specified DHT bootstrap node, given the node doesn't have those disabled. | -| `sign` | Executable | libsodium | Cross-platform | Signs a file with a ed25519 key. | +| Name | Type | Dependencies | Platform | Description | +| --------------------- | ---------- | ------------ | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `bootstrap_node_info` | Script | python3 | Cross-platform | Prints version and Message Of The Day (MOTD) information of the specified DHT bootstrap node, given the node doesn't have those disabled. | +| `sign` | Executable | libsodium | Cross-platform | Signs a file with a ed25519 key. | ## Building @@ -80,33 +89,47 @@ Useful for generating Tox profiles from the output of the vanity key generators, #### Library dependencies -Library dependencies are listed in the [components](#components) table. The dependencies need to be satisfied for the components to be built. Note that if you don't have a dependency for some component, e.g. you don't have `libopus` installed required for building `toxav` component, building of that component is silently disabled. +Library dependencies are listed in the [components](#components) table. The +dependencies need to be satisfied for the components to be built. Note that if +you don't have a dependency for some component, e.g. you don't have `libopus` +installed required for building `toxav` component, building of that component is +silently disabled. - -Be advised that due to the addition of `cmp` as a submodule, you now also need to initialize the git submodules required by toxcore. This can be done by cloning the repo with the addition of `--recurse-submodules` or by running `git submodule update --init` in the root directory of the repo. +Be advised that due to the addition of `cmp` as a submodule, you now also need +to initialize the git submodules required by toxcore. This can be done by +cloning the repo with the addition of `--recurse-submodules` or by running +`git submodule update --init` in the root directory of the repo. #### Compiler requirements The supported compilers are GCC, Clang and MinGW. -In theory, any compiler that fully supports C99 and accepts GCC flags should work. +In theory, any compiler that fully supports C99 and accepts GCC flags should +work. -There is a partial and experimental support of Microsoft Visual C++ compiler. We welcome any patches that help improve it. +There is a partial and experimental support of Microsoft Visual C++ compiler. We +welcome any patches that help improve it. -You should have a C99 compatible compiler in order to build the main components. The secondary components might require the compiler to support GNU extensions. +You should have a C99 compatible compiler in order to build the main components. +The secondary components might require the compiler to support GNU extensions. #### Build system requirements -To build the main components you need to have CMake of at least 2.8.6 version installed. You also need to have pkg-config installed, the build system uses it to find dependency libraries. +To build the main components you need to have CMake of at least 2.8.6 version +installed. You also need to have pkg-config installed, the build system uses it +to find dependency libraries. -There is some experimental accommodation for building natively on Windows, i.e. without having to use MSYS/Cygwin and pkg-config, but it uses exact hardcoded paths for finding libraries and supports building only of some of toxcore components, so your mileage might vary. +There is some experimental accommodation for building natively on Windows, i.e. +without having to use MSYS/Cygwin and pkg-config, but it uses exact hardcoded +paths for finding libraries and supports building only of some of toxcore +components, so your mileage might vary. ### CMake options There are some options that are available to configure the build. | Name | Description | Expected Value | Default Value | -|------------------------|-----------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|---------------------------------------------------| +| ---------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------- | | `AUTOTEST` | Enable autotests (mainly for CI). | ON or OFF | OFF | | `BOOTSTRAP_DAEMON` | Enable building of tox-bootstrapd, the DHT bootstrap node daemon. For Unix-like systems only. | ON or OFF | ON | | `BUILD_FUZZ_TESTS` | Build fuzzing harnesses. | ON or OFF | OFF | @@ -150,11 +173,12 @@ cmake \ ``` ### Building tests + In addition to the integration tests ("autotests") and miscellaneous tests enabled by cmake variables described above, there are unit tests which will be built if the source distribution of gtest (the Google Unit Test framework) is -found by cmake in `c-toxcore/third_party`. This can be achieved by running -'git clone https://github.com/google/googletest` from that directory. +found by cmake in `c-toxcore/third_party`. This can be achieved by running 'git +clone https://github.com/google/googletest` from that directory. ### Build process @@ -176,13 +200,20 @@ make install ###### Microsoft Visual Studio's Developer Command Prompt -In addition to meeting the [requirements](#requirements), you need a version of Visual Studio (the [community edition](https://www.visualstudio.com/vs/visual-studio-express/) is enough) and a CMake version that's compatible with the Visual Studio version you're using. +In addition to meeting the [requirements](#requirements), you need a version of +Visual Studio (the +[community edition](https://www.visualstudio.com/vs/visual-studio-express/) is +enough) and a CMake version that's compatible with the Visual Studio version +you're using. -You must also ensure that the msvc versions of dependencies you're using are placed in the correct folders. +You must also ensure that the msvc versions of dependencies you're using are +placed in the correct folders. -For libsodium that is `c-toxcore/third_party/libsodium`, and for pthreads-w32, it's `c-toxcore/third_party/pthreads-win32` +For libsodium that is `c-toxcore/third_party/libsodium`, and for pthreads-w32, +it's `c-toxcore/third_party/pthreads-win32` -Once all of this is done, from the **Developer Command Prompt for VS**, simply run +Once all of this is done, from the **Developer Command Prompt for VS**, simply +run ``` mkdir _build @@ -193,22 +224,23 @@ msbuild ALL_BUILD.vcxproj ###### MSYS/Cygwin -Download Cygwin ([32-bit](https://cygwin.com/setup-x86.exe)/[64-bit](https://cygwin.com/setup-x86_64.exe)) +Download Cygwin +([32-bit](https://cygwin.com/setup-x86.exe)/[64-bit](https://cygwin.com/setup-x86_64.exe)) Search and select exactly these packages in Devel category: - - mingw64-i686-gcc-core (32-bit) / mingw64-x86_64-gcc-core (64-bit) - - mingw64-i686-gcc-g++ (32-bit) / mingw64-x86_64-gcc-g++ (64-bit) - - make - - cmake - - libtool - - autoconf - - automake - - tree - - curl - - perl - - yasm - - pkg-config +- mingw64-i686-gcc-core (32-bit) / mingw64-x86_64-gcc-core (64-bit) +- mingw64-i686-gcc-g++ (32-bit) / mingw64-x86_64-gcc-g++ (64-bit) +- make +- cmake +- libtool +- autoconf +- automake +- tree +- curl +- perl +- yasm +- pkg-config To handle Windows EOL correctly run the following in the Cygwin Terminal: @@ -221,18 +253,24 @@ set -o igncr Download toxcore source code and extract it to a folder. -Open Cygwin Terminal in the toxcore folder and run `./other/windows_build_script_toxcore.sh` to start the build process. - -Toxcore build result files will appear in `/root/prefix/` relatively to Cygwin folder (default `C:\cygwin64`). +Open Cygwin Terminal in the toxcore folder and run +`./other/windows_build_script_toxcore.sh` to start the build process. -Dependency versions can be customized in `./other/windows_build_script_toxcore.sh` and described in the section below. +Toxcore build result files will appear in `/root/prefix/` relatively to Cygwin +folder (default `C:\cygwin64`). +Dependency versions can be customized in +`./other/windows_build_script_toxcore.sh` and described in the section below. ##### Cross-compiling from Linux -These cross-compilation instructions were tested on and written for 64-bit Ubuntu 16.04. You could generalize them for any Linux system, the only requirements are that you have Docker version of >= 1.9.0 and you are running 64-bit system. +These cross-compilation instructions were tested on and written for 64-bit +Ubuntu 16.04. You could generalize them for any Linux system, the only +requirements are that you have Docker version of >= 1.9.0 and you are running +64-bit system. -The cross-compilation is fully automated by a parameterized [Dockerfile](/other/docker/windows/Dockerfile). +The cross-compilation is fully automated by a parameterized +[Dockerfile](/other/docker/windows/Dockerfile). Install Docker @@ -243,17 +281,18 @@ apt-get install docker.io Get the toxcore source code and navigate to `other/docker/windows`. -Build the container image based on the Dockerfile. The following options are available to customize the building of the container image. +Build the container image based on the Dockerfile. The following options are +available to customize the building of the container image. -| Name | Description | Expected Value | Default Value | -|-----------------------|----------------------------------------------------------------|-------------------------------------|---------------| -| `SUPPORT_ARCH_i686` | Support building 32-bit toxcore. | "true" or "false" (case sensitive). | true | -| `SUPPORT_ARCH_x86_64` | Support building 64-bit toxcore. | "true" or "false" (case sensitive). | true | -| `SUPPORT_TEST` | Support running toxcore automated tests. | "true" or "false" (case sensitive). | false | -| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | true | -| `VERSION_OPUS` | Version of libopus to build toxcore with. | Numeric version number. | 1.3.1 | -| `VERSION_SODIUM` | Version of libsodium to build toxcore with. | Numeric version number. | 1.0.18 | -| `VERSION_VPX` | Version of libvpx to build toxcore with. | Numeric version number. | 1.11.0 | +| Name | Description | Expected Value | Default Value | +| --------------------- | --------------------------------------------------- | ----------------------------------- | ------------- | +| `SUPPORT_ARCH_i686` | Support building 32-bit toxcore. | "true" or "false" (case sensitive). | true | +| `SUPPORT_ARCH_x86_64` | Support building 64-bit toxcore. | "true" or "false" (case sensitive). | true | +| `SUPPORT_TEST` | Support running toxcore automated tests. | "true" or "false" (case sensitive). | false | +| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | true | +| `VERSION_OPUS` | Version of libopus to build toxcore with. | Numeric version number. | 1.3.1 | +| `VERSION_SODIUM` | Version of libsodium to build toxcore with. | Numeric version number. | 1.0.18 | +| `VERSION_VPX` | Version of libvpx to build toxcore with. | Numeric version number. | 1.11.0 | Example of building a container image with options @@ -265,16 +304,17 @@ docker build \ . ``` -Run the container to build toxcore. The following options are available to customize the running of the container image. +Run the container to build toxcore. The following options are available to +customize the running of the container image. -| Name | Description | Expected Value | Default Value | -|----------------------|--------------------------------------------------------------------------------------------|-------------------------------------|--------------------------------------------------------------------| -| `ALLOW_TEST_FAILURE` | Don't stop if a test suite fails. | "true" or "false" (case sensitive). | `false` | -| `ENABLE_ARCH_i686` | Build 32-bit toxcore. The image should have been built with `SUPPORT_ARCH_i686` enabled. | "true" or "false" (case sensitive). | `true` | -| `ENABLE_ARCH_x86_64` | Build 64-bit toxcore. The image should have been built with `SUPPORT_ARCH_x86_64` enabled. | "true" or "false" (case sensitive). | `true` | -| `ENABLE_TEST` | Run the test suite. The image should have been built with `SUPPORT_TEST` enabled. | "true" or "false" (case sensitive). | `false` | -| `EXTRA_CMAKE_FLAGS` | Extra arguments to pass to the CMake command when building toxcore. | CMake options. | `-DTEST_TIMEOUT_SECONDS=90` | -| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | `true` | +| Name | Description | Expected Value | Default Value | +| -------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------- | --------------------------- | +| `ALLOW_TEST_FAILURE` | Don't stop if a test suite fails. | "true" or "false" (case sensitive). | `false` | +| `ENABLE_ARCH_i686` | Build 32-bit toxcore. The image should have been built with `SUPPORT_ARCH_i686` enabled. | "true" or "false" (case sensitive). | `true` | +| `ENABLE_ARCH_x86_64` | Build 64-bit toxcore. The image should have been built with `SUPPORT_ARCH_x86_64` enabled. | "true" or "false" (case sensitive). | `true` | +| `ENABLE_TEST` | Run the test suite. The image should have been built with `SUPPORT_TEST` enabled. | "true" or "false" (case sensitive). | `false` | +| `EXTRA_CMAKE_FLAGS` | Extra arguments to pass to the CMake command when building toxcore. | CMake options. | `-DTEST_TIMEOUT_SECONDS=90` | +| `CROSS_COMPILE` | Cross-compiling. True for Docker, false for Cygwin. | "true" or "false" (case sensitive). | `true` | Example of running the container with options @@ -288,10 +328,17 @@ docker run \ toxcore ``` -After the build succeeds, you should see the built toxcore libraries in `/path/to/where/output/build/result`. +After the build succeeds, you should see the built toxcore libraries in +`/path/to/where/output/build/result`. ## Pre-built binaries ### Linux -Toxcore is packaged by at least by the following distributions: ALT Linux, [Arch Linux](https://www.archlinux.org/packages/?q=toxcore), [Fedora](https://apps.fedoraproject.org/packages/toxcore), Mageia, openSUSE, PCLinuxOS, ROSA and Slackware, [according to the information from pkgs.org](https://pkgs.org/download/toxcore). Note that this list might be incomplete and some other distributions might package it too. +Toxcore is packaged by at least by the following distributions: ALT Linux, +[Arch Linux](https://www.archlinux.org/packages/?q=toxcore), +[Fedora](https://apps.fedoraproject.org/packages/toxcore), Mageia, openSUSE, +PCLinuxOS, ROSA and Slackware, +[according to the information from pkgs.org](https://pkgs.org/download/toxcore). +Note that this list might be incomplete and some other distributions might +package it too. diff --git a/README.md b/README.md index cb62273fe5..853a371516 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,14 @@ # ![Project Tox](https://raw.github.com/TokTok/c-toxcore/master/other/tox.png "Project Tox") -**Current Coverage:** [![coverage](https://codecov.io/gh/TokTok/c-toxcore/branch/master/graph/badge.svg?token=BRfCKo02De)](https://codecov.io/gh/TokTok/c-toxcore) +**Current Coverage:** +[![coverage](https://codecov.io/gh/TokTok/c-toxcore/branch/master/graph/badge.svg?token=BRfCKo02De)](https://codecov.io/gh/TokTok/c-toxcore) -[**Website**](https://tox.chat) **|** [**Wiki**](https://wiki.tox.chat/) **|** [**Blog**](https://blog.tox.chat/) **|** [**FAQ**](https://wiki.tox.chat/doku.php?id=users:faq) **|** [**Binaries/Downloads**](https://tox.chat/download.html) **|** [**Clients**](https://wiki.tox.chat/doku.php?id=clients) **|** [**Compiling**](/INSTALL.md) +[**Website**](https://tox.chat) **|** [**Wiki**](https://wiki.tox.chat/) **|** +[**Blog**](https://blog.tox.chat/) **|** +[**FAQ**](https://wiki.tox.chat/doku.php?id=users:faq) **|** +[**Binaries/Downloads**](https://tox.chat/download.html) **|** +[**Clients**](https://wiki.tox.chat/doku.php?id=clients) **|** +[**Compiling**](/INSTALL.md) ## What is Tox @@ -16,29 +22,33 @@ and privacy easy to obtain for regular users. It uses ### ![Danger: Experimental](other/tox-warning.png) This is an **experimental** cryptographic network library. It has not been -formally audited by an independent third party that specializes in -cryptography or cryptanalysis. **Use this library at your own risk.** +formally audited by an independent third party that specializes in cryptography +or cryptanalysis. **Use this library at your own risk.** The underlying crypto library [libsodium](https://doc.libsodium.org/) provides reliable encryption, but the security model has not yet been fully specified. -See [issue 210](https://github.com/TokTok/c-toxcore/issues/210) for a -discussion on developing a threat model. See other issues for known weaknesses -(e.g. [issue 426](https://github.com/TokTok/c-toxcore/issues/426) describes -what can happen if your secret key is stolen). +See [issue 210](https://github.com/TokTok/c-toxcore/issues/210) for a discussion +on developing a threat model. See other issues for known weaknesses (e.g. +[issue 426](https://github.com/TokTok/c-toxcore/issues/426) describes what can +happen if your secret key is stolen). ## Toxcore Development Roadmap -The roadmap and changelog are generated from GitHub issues. You may view them -on the website, where they are updated at least once every 24 hours: +The roadmap and changelog are generated from GitHub issues. You may view them on +the website, where they are updated at least once every 24 hours: -- Changelog: https://toktok.ltd/changelog/c-toxcore -- Roadmap: https://toktok.ltd/roadmap/c-toxcore +- Changelog: https://toktok.ltd/changelog/c-toxcore +- Roadmap: https://toktok.ltd/roadmap/c-toxcore ## Installing toxcore Detailed installation instructions can be found in [INSTALL.md](INSTALL.md). -Be advised that due to the addition of `cmp` as a submodule, you now also need to initialize the git submodules required by toxcore. This can be done by cloning the repo with the following command: `git clone --recurse-submodules https://github.com/Toktok/c-toxcore` or by running `git submodule update --init` in the root directory of the repo. +Be advised that due to the addition of `cmp` as a submodule, you now also need +to initialize the git submodules required by toxcore. This can be done by +cloning the repo with the following command: +`git clone --recurse-submodules https://github.com/Toktok/c-toxcore` or by +running `git submodule update --init` in the root directory of the repo. In a nutshell, if you have [libsodium](https://github.com/jedisct1/libsodium) installed, run: @@ -74,17 +84,17 @@ if (err_new != TOX_ERR_NEW_OK) { } ``` -Here, we simply exit the program, but in a real client you will probably want -to do some error handling and proper error reporting to the user. The `NULL` +Here, we simply exit the program, but in a real client you will probably want to +do some error handling and proper error reporting to the user. The `NULL` argument given to the first parameter of `tox_new` is the `Tox_Options`. It -contains various write-once network settings and allows you to load a -previously serialised instance. See [toxcore/tox.h](tox.h) for details. +contains various write-once network settings and allows you to load a previously +serialised instance. See [toxcore/tox.h](tox.h) for details. ### Setting up callbacks -Toxcore works with callbacks that you can register to listen for certain -events. Examples of such events are "friend request received" or "friend sent -a message". Search the API for `tox_callback_*` to find all of them. +Toxcore works with callbacks that you can register to listen for certain events. +Examples of such events are "friend request received" or "friend sent a +message". Search the API for `tox_callback_*` to find all of them. Here, we will set up callbacks for receiving friend requests and receiving messages. We will always accept any friend request (because we're a bot), and diff --git a/docs/Group-Chats.md b/docs/Group-Chats.md index 2935c7f5d9..b2199e769a 100644 --- a/docs/Group-Chats.md +++ b/docs/Group-Chats.md @@ -2,77 +2,122 @@ Group chats. Note: we assume everyone in the chat trusts each other. -These group chats work by temporarily adding the 4 "closest" people defined by a distance function -in group.c in order to form a circle of connected peers. These peers then relay messages to each other. - -A friend invites another friend to a group chat by sending them an invite packet. The friend either ignores -the invite or responds with a response packet if he wants to join the chat. The friend invite contains the type -of groupchat (text only, A/V) the friend is being invited to. +These group chats work by temporarily adding the 4 "closest" people defined by a +distance function in group.c in order to form a circle of connected peers. These +peers then relay messages to each other. +A friend invites another friend to a group chat by sending them an invite +packet. The friend either ignores the invite or responds with a response packet +if he wants to join the chat. The friend invite contains the type of groupchat +(text only, A/V) the friend is being invited to. TODO(irungentoo): write more of this. ## Protocol Invite packets: -Invite packet: + +### Invite packet: + +``` [uint8_t id 96][uint8_t id 0][uint16_t group chat number][33 bytes group chat identifier[1 byte type][32 bytes id]] +``` -Response packet +### Response packet + +``` [uint8_t id 96][uint8_t id 1][uint16_t group chat number(local)][uint16_t group chat number to join][33 bytes group chat identifier[1 byte type][32 bytes id]] +``` +### Peer online packet: -Peer online packet: +``` [uint8_t id 97][uint16_t group chat number (local)][33 bytes group chat identifier[1 byte type][32 bytes id]] +``` + +### Peer leave packet: -Peer leave packet: +``` [uint8_t id 98][uint16_t group chat number][uint8_t id 1] +``` -Peer query packet: +### Peer query packet: + +``` [uint8_t id 98][uint16_t group chat number][uint8_t id 8] +``` + +### Peer response packet: + +``` +[uint8_t id 98][uint16_t group chat number][uint8_t id 9][Repeated times number of peers: [uint16_t peer num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key][uint8_t name length][name]] +``` -Peer response packet: -[uint8_t id 98][uint16_t group chat number][uint8_t id 9][Repeated times number of peers: [uint16_t peer num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key][uint8_t name length][name]] +### Title response packet: -Title response packet: +``` [uint8_t id 98][uint16_t group chat number][uint8_t id 10][title] +``` -Message packets: +### Message packets: + +``` [uint8_t id 99][uint16_t group chat number][uint16_t peer number][uint32_t message number][uint8_t with a value representing id of message][data] +``` + +### Lossy Message packets: -Lossy Message packets: +``` [uint8_t id 199][uint16_t group chat number][uint16_t peer number][uint16_t message number][uint8_t with a value representing id of message][data] +``` -Group chat types: -0: text -1: AV +## Group chat types: +- 0: text +- 1: AV Note: the message number is increased by 1 for each sent message. -message ids: -0 - ping -sent every ~60 seconds by every peer. -No data. +## message ids: + +### 0 - ping + +sent every ~60 seconds by every peer. No data. + +### 16 - new_peer -16 - new_peer Tell everyone about a new peer in the chat. + +``` [uint16_t peer_num][uint8_t 32bytes real public key][uint8_t 32bytes temp DHT public key] +``` + +### 17 - kill_peer -17 - kill_peer +``` [uint16_t peer_num] +``` -48 - name change +### 48 - name change + +``` [uint8_t name[namelen]] +``` + +### 49 - groupchat title change -49 - groupchat title change +``` [uint8_t title[titlelen]] +``` -64 - chat message -[uint8_t message[messagelen]] +### 64 - chat message -65 - action (/me) +``` [uint8_t message[messagelen]] +``` +### 65 - action (/me) - +``` +[uint8_t message[messagelen]] +``` diff --git a/docs/apidsl.md b/docs/apidsl.md deleted file mode 100644 index 96ff5d0de9..0000000000 --- a/docs/apidsl.md +++ /dev/null @@ -1,51 +0,0 @@ -This folder contains the input file (``tox.in.h``) that has to be used to generate the ``tox.h`` api with: https://github.com/TokTok/apidsl - -# Minimal requirements - -There are some minimal requirements to contribute to ``tox.h``: -* unix environment -* ``astyle`` ``>=2.03`` -* [``apidsl``](https://github.com/TokTok/apidsl) (you can use provided service with curl instead) - -## Quick way - -If you want to do it quickly and you don't have time for anything other than copypasting commands, you should have ``curl`` installed. - - -1. Make sure that you have ``curl`` and ``>=astyle-2.03`` installed -2. Modify [``tox.api.h``](/toxcore/tox.api.h) -3. Run command below ↓ - -Command to run from ``toxcore`` directory (quick way, involves using curl): -```bash -# For tox.h: -curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \ - < toxcore/tox.api.h \ - | astyle --options=other/astyle/astylerc \ - > toxcore/tox.h -# For toxav.h: -curl -X POST --data-binary @- https://apidsl.herokuapp.com/apidsl \ - < toxav/toxav.api.h \ - | astyle --options=other/astyle/astylerc \ - > toxav/toxav.h -``` - -You may want to make sure with ``git diff`` that changes made in ``tox.h`` reflect changes in ``tox.in.h``. - -And you're done. - - -## Manually - -If you prefer to have more control over what is happening, there are steps below: - -1. Install [``apidsl``](https://github.com/TokTok/apidsl) -2. Install ``astyle``, version 2.03 or later. -3. Modify [``tox.api.h``](/toxcore/tox.api.h) -4. Use ``apidsl`` ``??`` -5. Parse generated ``tox.h`` with astyle, minimal command for it would be: -```bash -astyle --options=other/astyle/astylerc toxcore/tox.h -``` - -**Always pass output from ``apidsl`` through astyle.** diff --git a/docs/av_api.md b/docs/av_api.md index a087cdadce..733c91f882 100644 --- a/docs/av_api.md +++ b/docs/av_api.md @@ -8,38 +8,36 @@ phone_t* initPhone(uint16_t _listen_port, uint16_t _send_port); ``` -function initializes sample phone. _listen_port and _send_port are variables only meant -for local testing. You will not have to do anything regarding to that since -everything will be started within a messenger. +function initializes sample phone. `_listen_port` and `_send_port` are variables +only meant for local testing. You will not have to do anything regarding to that +since everything will be started within a messenger. - -Phone requires one msi session and two rtp sessions ( one for audio and one for -video ). +Phone requires one msi session and two rtp sessions (one for audio and one for +video). ``` msi_session_t* msi_init_session( void* _core_handler, const uint8_t* _user_agent ); ``` -initializes msi session. -Params: +initializes msi session. Params: ``` void* _core_handler - pointer to an object handling networking, const uint8_t* _user_agent - string describing phone client version. ``` -Return value: -msi_session_t* - pointer to a newly created msi session handler. +Return value: `msi_session_t*` - pointer to a newly created msi session handler. + +### `msi_session_t` reference: -### msi_session_t reference: +How to handle msi session: Controlling is done via callbacks and action +handlers. First register callbacks for every state/action received and make sure +NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every +callback is being called directly from event loop. You can find examples in +phone.c. -How to handle msi session: -Controlling is done via callbacks and action handlers. -First register callbacks for every state/action received and make sure -NOT TO PLACE SOMETHING LIKE LOOPS THAT TAKES A LOT OF TIME TO EXECUTE; every callback is being called -directly from event loop. You can find examples in phone.c. +Register callbacks: -Register callbacks: ``` void msi_register_callback_call_started ( MCALLBACK ); void msi_register_callback_call_canceled ( MCALLBACK ); @@ -55,10 +53,9 @@ void msi_register_callback_recv_error ( MCALLBACK ); void msi_register_callback_requ_timeout ( MCALLBACK ); ``` -MCALLBACK is defined as: void (*callback) (void* _arg) -msi_session_t* handler is being thrown as \_arg so you can use that and \_agent_handler to get to your own phone handler -directly from callback. - +MCALLBACK is defined as: `void (*callback) (void* _arg)` `msi_session_t*` +handler is being thrown as `_arg` so you can use that and `_agent_handler` to +get to your own phone handler directly from callback. Actions: @@ -66,80 +63,93 @@ Actions: int msi_invite ( msi_session_t* _session, call_type _call_type, uint32_t _timeoutms ); ``` -Sends call invite. Before calling/sending invite msi_session_t::_friend_id is needed to be set or else -it will not work. _call_type is type of the call ( Audio/Video ) and _timeoutms is how long -will poll wait until request is terminated. +Sends call invite. Before calling/sending invite `msi_session_t::_friend_id` is +needed to be set or else it will not work. `_call_type` is type of the call ( +Audio/Video ) and `_timeoutms` is how long will poll wait until request is +terminated. ``` int msi_hangup ( msi_session_t* _session ); ``` + Hangs up active call ``` int msi_answer ( msi_session_t* _session, call_type _call_type ); ``` -Answer incoming call. _call_type set's callee call type. + +Answer incoming call. `_call_type` set's callee call type. ``` int msi_cancel ( msi_session_t* _session ); ``` + Cancel current request. ``` int msi_reject ( msi_session_t* _session ); ``` -Reject incoming call. +Reject incoming call. ### Now for rtp: -You will need 2 sessions; one for audio one for video. -You start them with: +You will need 2 sessions; one for audio one for video. You start them with: + ``` rtp_session_t* rtp_init_session ( int _max_users, int _multi_session ); ``` Params: + ``` int _max_users - max users. -1 if undefined int _multi_session - any positive number means uses multi session; -1 if not. ``` Return value: + ``` rtp_session_t* - pointer to a newly created rtp session handler. ``` ### How to handle rtp session: + Take a look at + ``` void* phone_handle_media_transport_poll ( void* _hmtc_args_p ) in phone.c ``` + on example. Basically what you do is just receive a message via: + ``` struct rtp_msg_s* rtp_recv_msg ( rtp_session_t* _session ); ``` -and then you use payload within the rtp_msg_s struct. Don't forget to deallocate it with: -void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg ); +and then you use payload within the `rtp_msg_s` struct. Don't forget to +deallocate it with: +`void rtp_free_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg );` Receiving should be thread safe so don't worry about that. When you capture and encode a payload you want to send it ( obviously ). first create a new message with: + ``` struct rtp_msg_s* rtp_msg_new ( rtp_session_t* _session, const uint8_t* _data, uint32_t _length ); ``` and then send it with: + ``` int rtp_send_msg ( rtp_session_t* _session, struct rtp_msg_s* _msg, void* _core_handler ); ``` -_core_handler is the same network handler as in msi_session_s struct. - +`_core_handler` is the same network handler as in `msi_session_s` struct. ## A/V initialization: + ``` int init_receive_audio(codec_state *cs); int init_receive_video(codec_state *cs); @@ -156,39 +166,62 @@ In the future, VP8 should be used directly and ffmpeg should be dropped from the The variable bps is the required bitrate in bits per second. ``` - ### A/V encoding/decoding: + ``` void *encode_video_thread(void *arg); ``` -Spawns the video encoding thread. The argument should hold a pointer to a codec_state. -This function should only be called if video encoding is supported (when init_send_video returns 1). -Each video frame gets encoded into a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe is encoded. + +Spawns the video encoding thread. The argument should hold a pointer to a +`codec_state`. This function should only be called if video encoding is +supported (when `init_send_video` returns 1). Each video frame gets encoded into +a packet, which is sent via RTP. Every 60 frames a new bidirectional interframe +is encoded. + ``` void *encode_audio_thread(void *arg); ``` -Spawns the audio encoding thread. The argument should hold a pointer to a codec_state. -This function should only be called if audio encoding is supported (when init_send_audio returns 1). -Audio frames are read from the selected audio capture device during initialisation. This audio capturing can be rerouted to a different device on the fly. -Each audio frame is encoded into a packet, and sent via RTP. All audio frames have the same amount of samples, which is defined in AV_codec.h. + +Spawns the audio encoding thread. The argument should hold a pointer to a +`codec_state`. This function should only be called if audio encoding is +supported (when `init_send_audio` returns 1). Audio frames are read from the +selected audio capture device during initialisation. This audio capturing can be +rerouted to a different device on the fly. Each audio frame is encoded into a +packet, and sent via RTP. All audio frames have the same amount of samples, +which is defined in `AV_codec.h`. + ``` int video_decoder_refresh(codec_state *cs, int width, int height); ``` -Sets the SDL window dimensions and creates a pixel buffer with the requested size. It also creates a scaling context, which will be used to convert the input image format to YUV420P. + +Sets the SDL window dimensions and creates a pixel buffer with the requested +size. It also creates a scaling context, which will be used to convert the input +image format to YUV420P. ``` void *decode_video_thread(void *arg); ``` -Spawns a video decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised video decoder. -This function reads video packets and feeds them to the video decoder. If the video frame's resolution has changed, video_decoder_refresh() is called. Afterwards, the frame is displayed on the SDL window. + +Spawns a video decoding thread. The argument should hold a pointer to a +`codec_state`. The `codec_state` is assumed to contain a successfully +initialised video decoder. This function reads video packets and feeds them to +the video decoder. If the video frame's resolution has changed, +`video_decoder_refresh()` is called. Afterwards, the frame is displayed on the +SDL window. + ``` void *decode_audio_thread(void *arg); ``` -Spawns an audio decoding thread. The argument should hold a pointer to a codec_state. The codec_state is assumed to contain a successfully initialised audio decoder. -All received audio packets are pushed into a jitter buffer and are reordered. If there is a missing packet, or a packet has arrived too late, it is treated as a lost packet and the audio decoder is informed of the packet loss. The audio decoder will then try to reconstruct the lost packet, based on information from previous packets. -Audio is played on the default OpenAL output device. +Spawns an audio decoding thread. The argument should hold a pointer to a +`codec_state`. The `codec_state` is assumed to contain a successfully +initialised audio decoder. All received audio packets are pushed into a jitter +buffer and are reordered. If there is a missing packet, or a packet has arrived +too late, it is treated as a lost packet and the audio decoder is informed of +the packet loss. The audio decoder will then try to reconstruct the lost packet, +based on information from previous packets. Audio is played on the default +OpenAL output device. -If you have any more qustions/bug reports/feature request contact the following users on the irc channel #tox-dev on irc.freenode.net: -For RTP and MSI: mannol +If you have any more qustions/bug reports/feature request contact the following +users on the irc channel #tox-dev on irc.freenode.net: For RTP and MSI: mannol For audio and video: Martijnvdc diff --git a/docs/minpgc.md b/docs/minpgc.md index a09895bd35..6389bb50c5 100644 --- a/docs/minpgc.md +++ b/docs/minpgc.md @@ -3,29 +3,29 @@ This document describes the "minpgc" simple persistent conferences implementation of PR #1069. -Many of the ideas derive from isotoxin's persistent conferences -implementation, PR #826. +Many of the ideas derive from isotoxin's persistent conferences implementation, +PR #826. ## Specification of changes from pre-existing conference specification + We add one new packet type: Rejoin Conference packet -| Length | Contents | -|:-------|:--------------------------------| -| `1` | `uint8_t` (0x64) | -| `33` | Group chat identifier | - +| Length | Contents | +| :----- | :-------------------- | +| `1` | `uint8_t` (0x64) | +| `33` | Group chat identifier | -A peer times out from a group if it has been inactive for 60s. When a peer -times out, we flag it as _frozen_. Frozen peers are disregarded for all -purposes except those discussed below - in particular no packets are sent to -them except as described below, they are omitted from the peer lists sent to -the client or in a Peer Response packet, and they are not considered when -determining closest peers for establishing direct connections. +A peer times out from a group if it has been inactive for 60s. When a peer times +out, we flag it as _frozen_. Frozen peers are disregarded for all purposes +except those discussed below - in particular no packets are sent to them except +as described below, they are omitted from the peer lists sent to the client or +in a Peer Response packet, and they are not considered when determining closest +peers for establishing direct connections. -A peer is considered to be active if we receive a group message or Rejoin -packet from it, or a New Peer message for it. +A peer is considered to be active if we receive a group message or Rejoin packet +from it, or a New Peer message for it. If a frozen peer is seen to be active, we remove its 'frozen' flag and send a Name group message. (We can hold off on sending this message until the next @@ -40,77 +40,83 @@ message. (This is current behaviour; it's mentioned here because it's important and not currently mentioned in the spec.) If we receive a Rejoin packet from a peer we update its DHT pubkey, add a -temporary groupchat connection for the peer, and, once the connection is -online, send out a New Peer message announcing the peer, and a Name message. +temporary groupchat connection for the peer, and, once the connection is online, +send out a New Peer message announcing the peer, and a Name message. -Whenever we make a new friend connection, we check if the public key is that -of any frozen peer. If so, we send it a Rejoin packet, add a temporary -groupchat connection for it, and, once the connection is online, send the -peer a Peer Query packet. +Whenever we make a new friend connection, we check if the public key is that of +any frozen peer. If so, we send it a Rejoin packet, add a temporary groupchat +connection for it, and, once the connection is online, send the peer a Peer +Query packet. -We do the same with a peer when we are setting it as frozen if we have a -friend connection to it. +We do the same with a peer when we are setting it as frozen if we have a friend +connection to it. The temporary groupchat connections established in sending and handling Rejoin packets are not immediately operational (because group numbers are not known); rather, an Online packet is sent when we handle a Rejoin packet. -When a connection is set as online as a result of an Online packet, we ping -the group. +When a connection is set as online as a result of an Online packet, we ping the +group. When processing the reply to a Peer Query, we update the DHT pubkey of an -existing peer if and only if it is frozen or has not had its DHT pubkey -updated since it last stopped being frozen. +existing peer if and only if it is frozen or has not had its DHT pubkey updated +since it last stopped being frozen. When we receive a Title Response packet, we set the title if it has never been set or if at some point since it was last set, there were no unfrozen peers (except us). ## Discussion + ### Overview -The intention is to recover seamlessly from splits in the group, the most -common form of which is a single peer temporarily losing all connectivity. -To see how this works, first note that groups (even before the changes -discussed here) have the property that for a group to be connected in the -sense that any peer will receive the messages of any other peer and have them -in their peerlist, it is necessary and sufficient that there is a path of -direct group connections between any two peers. +The intention is to recover seamlessly from splits in the group, the most common +form of which is a single peer temporarily losing all connectivity. + +To see how this works, first note that groups (even before the changes discussed +here) have the property that for a group to be connected in the sense that any +peer will receive the messages of any other peer and have them in their +peerlist, it is necessary and sufficient that there is a path of direct group +connections between any two peers. Now suppose the group is split into two connected components, with each member -of one component frozen according to the members of the other. Suppose there -are two peers, one in each component, which are using the above protocol, and +of one component frozen according to the members of the other. Suppose there are +two peers, one in each component, which are using the above protocol, and suppose they establish a friend connection. Then each will rejoin the other, forming a direct group connection. Hence the whole group will become connected (even if all other peers are using the unmodified protocol). The Peer Query packet sent on rejoining hastens this process. -Peers who leave the group during a split will not be deleted by all peers -after the merge - but they will be set as frozen due to ping timeouts, which -is sufficient. +Peers who leave the group during a split will not be deleted by all peers after +the merge - but they will be set as frozen due to ping timeouts, which is +sufficient. ### Titles -If we have a split into components each containing multiple peers, and the -title is changed in one component, then peers will continue to disagree on the -title after the split. Short of a complicated voting system, this seems the -only reasonable behaviour. + +If we have a split into components each containing multiple peers, and the title +is changed in one component, then peers will continue to disagree on the title +after the split. Short of a complicated voting system, this seems the only +reasonable behaviour. ### Implementation notes -Although I've described the logic in terms of an 'frozen' flag, it might -actually make more sense in the implementation to have a separate list for + +Although I've described the logic in terms of an 'frozen' flag, it might +actually make more sense in the implementation to have a separate list for frozen peers. ## Saving + Saving is implemented by simply saving all live groups with their group numbers and full peer info for all peers. On reload, all peers are set as frozen. Clients needs to support this by understanding that groups may exist on start-up. Clients should call `tox_conference_get_chatlist` to obtain them. A -group which is deleted (with `tox_conference_delete`) is removed permanently -and will not be saved. +group which is deleted (with `tox_conference_delete`) is removed permanently and +will not be saved. ## Limitations + If a peer disconnects from the group for a period short enough that group timeouts do not occur, and a name change occurs during this period, then the name change will never be propagated. @@ -120,9 +126,8 @@ requesting missed group messages. But this is considered out of scope of this PR. If a peer changes its DHT pubkey, the change might not be properly propagated -under various circumstances - in particular, if connections do not go down -long enough for the peer to become frozen. +under various circumstances - in particular, if connections do not go down long +enough for the peer to become frozen. -One way to deal with this would be to add a group message announcing the -sending peer's current DHT pubkey, and treat it analogously to the Name -message. +One way to deal with this would be to add a group message announcing the sending +peer's current DHT pubkey, and treat it analogously to the Name message.