Skip to content

Commit

Permalink
Merge pull request #1469 from vircadia/webapp
Browse files Browse the repository at this point in the history
Enable WebRTC connections to be made to the Domain, i.e., add Web client support.
  • Loading branch information
digisomni authored Nov 26, 2021
2 parents 57167cd + 8764e6f commit aabcdea
Show file tree
Hide file tree
Showing 61 changed files with 2,908 additions and 224 deletions.
10 changes: 9 additions & 1 deletion CODING_STANDARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -948,7 +948,7 @@ In an international environment English is the preferred language.
#### [4.3.2] Use // for all comments, including multi-line comments.
An exception to this rule applies for jsdoc or Doxygen comments.
An exception to this rule applies to JSDoc and Doxygen comments.
```cpp
// Comment spanning
Expand Down Expand Up @@ -1008,3 +1008,11 @@ These types of comments are explicitly not allowed. If you need to break up sect
//--------------------------------------------------------------------------------
```

#### [4.3.6] Doxygen comments should use "///"

Use the `///` style of [Doxygen](https://www.doxygen.nl/index.html) comments when documenting public interfaces.

Some editors can automatically create a Doxygen documentation stub if you type `///` in the line above the item to be
documented.

**Visual Studio:** To configure Visual Studio's Doxygen commenting behavior, search for "Doxygen" in Tools > Options.
60 changes: 58 additions & 2 deletions assignment-client/src/AssignmentClient.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

#include <assert.h>

#include <QJsonDocument>
#include <QProcess>
#include <QSharedMemory>
#include <QThread>
Expand Down Expand Up @@ -84,7 +85,7 @@ AssignmentClient::AssignmentClient(Assignment::Type requestAssignmentType, QStri
_assignmentServerHostname = assignmentServerHostname;
}

_assignmentServerSocket = SockAddr(_assignmentServerHostname, assignmentServerPort, true);
_assignmentServerSocket = SockAddr(SocketType::UDP, _assignmentServerHostname, assignmentServerPort, true);
if (_assignmentServerSocket.isNull()) {
qCCritical(assignment_client) << "PAGE: Couldn't resolve domain server address" << _assignmentServerHostname;
}
Expand Down Expand Up @@ -119,7 +120,8 @@ AssignmentClient::AssignmentClient(Assignment::Type requestAssignmentType, QStri

// did we get an assignment-client monitor port?
if (assignmentMonitorPort > 0) {
_assignmentClientMonitorSocket = SockAddr(DEFAULT_ASSIGNMENT_CLIENT_MONITOR_HOSTNAME, assignmentMonitorPort);
_assignmentClientMonitorSocket = SockAddr(SocketType::UDP, DEFAULT_ASSIGNMENT_CLIENT_MONITOR_HOSTNAME,
assignmentMonitorPort);
_assignmentClientMonitorSocket.setObjectName("AssignmentClientMonitor");

qCDebug(assignment_client) << "Assignment-client monitor socket is" << _assignmentClientMonitorSocket;
Expand All @@ -132,6 +134,18 @@ AssignmentClient::AssignmentClient(Assignment::Type requestAssignmentType, QStri
PacketReceiver::makeUnsourcedListenerReference<AssignmentClient>(this, &AssignmentClient::handleCreateAssignmentPacket));
packetReceiver.registerListener(PacketType::StopNode,
PacketReceiver::makeUnsourcedListenerReference<AssignmentClient>(this, &AssignmentClient::handleStopNodePacket));

#if defined(WEBRTC_DATA_CHANNELS)
auto webrtcSocket = nodeList->getWebRTCSocket();

// Route inbound WebRTC signaling messages from the Domain Server.
packetReceiver.registerListener(PacketType::WebRTCSignaling,
PacketReceiver::makeUnsourcedListenerReference<AssignmentClient>(this, &AssignmentClient::handleWebRTCSignalingPacket));
connect(this, &AssignmentClient::webrtcSignalingMessageFromUserClient, webrtcSocket, &WebRTCSocket::onSignalingMessage);

// Route outbound WebRTC signaling messages via the Domain Server to the user client.
connect(webrtcSocket, &WebRTCSocket::sendSignalingMessage, this, &AssignmentClient::sendSignalingMessageToUserClient);
#endif
}

void AssignmentClient::stopAssignmentClient() {
Expand Down Expand Up @@ -340,3 +354,45 @@ void AssignmentClient::assignmentCompleted() {

_isAssigned = false;
}

#if defined(WEBRTC_DATA_CHANNELS)

void AssignmentClient::handleWebRTCSignalingPacket(QSharedPointer<ReceivedMessage> message) {
auto messageString = message->readString();
auto json = QJsonDocument::fromJson(messageString.toUtf8()).object();
if (json.keys().contains("echo")) {
// Echo message back to sender.

if (!json.keys().contains("to") || !json.keys().contains("from")) {
qCDebug(assignment_client) << "Invalid WebRTC signaling echo message received.";
return;
}

// Swap to/from.
auto to = json.value("to");
json.insert("to", json.value("from"));
json.insert("from", to);

// Send back to sender via the Domain Server.
auto packetList = NLPacketList::create(PacketType::WebRTCSignaling, QByteArray(), true, true);
packetList->writeString(QJsonDocument(json).toJson(QJsonDocument::Compact));
auto nodeList = DependencyManager::get<NodeList>();
auto domainServerAddress = nodeList->getDomainHandler().getSockAddr();
nodeList->sendPacketList(std::move(packetList), domainServerAddress);

} else {
// WebRTC signaling message.
emit webrtcSignalingMessageFromUserClient(json);
}
}

// Sends a signaling message from the assignment client to the user client via the Domain Server.
void AssignmentClient::sendSignalingMessageToUserClient(const QJsonObject& json) {
auto packetList = NLPacketList::create(PacketType::WebRTCSignaling, QByteArray(), true, true);
packetList->writeString(QJsonDocument(json).toJson(QJsonDocument::Compact));
auto nodeList = DependencyManager::get<NodeList>();
auto domainServerAddress = nodeList->getDomainHandler().getSockAddr();
nodeList->sendPacketList(std::move(packetList), domainServerAddress);
}

#endif
19 changes: 14 additions & 5 deletions assignment-client/src/AssignmentClient.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
#include <QtCore/QPointer>
#include <QtCore/QSharedPointer>

#include <shared/WebRTC.h>

#include "ThreadedAssignment.h"

class QSharedMemory;
Expand All @@ -30,19 +32,26 @@ class AssignmentClient : public QObject {
bool disableDomainPortAutoDiscovery);
~AssignmentClient();

public slots:
void aboutToQuit();

private slots:
void sendAssignmentRequest();
void assignmentCompleted();
void handleAuthenticationRequest();
void sendStatusPacketToACM();
void stopAssignmentClient();

public slots:
void aboutToQuit();

private slots:
void handleCreateAssignmentPacket(QSharedPointer<ReceivedMessage> message);
void handleStopNodePacket(QSharedPointer<ReceivedMessage> message);
#if defined(WEBRTC_DATA_CHANNELS)
void handleWebRTCSignalingPacket(QSharedPointer<ReceivedMessage> message);
void sendSignalingMessageToUserClient(const QJsonObject& json);
#endif

signals:
#if defined(WEBRTC_DATA_CHANNELS)
void webrtcSignalingMessageFromUserClient(const QJsonObject& json);
#endif

private:
void setUpStatusToMonitor();
Expand Down
2 changes: 1 addition & 1 deletion cmake/ports/webrtc/CONTROL
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Source: webrtc
Version: 20190626
Version: 20210105
Description: WebRTC
215 changes: 215 additions & 0 deletions cmake/ports/webrtc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
# WebRTC

WebRTC Information:
- https://webrtc.org/
- https://webrtc.googlesource.com/src
- https://webrtc.googlesource.com/src/+/refs/heads/master/docs/native-code/index.md
- https://webrtc.googlesource.com/src/+/refs/heads/master/docs/native-code/development/prerequisite-sw/index.md
- https://webrtc.googlesource.com/src/+/refs/heads/master/docs/native-code/development/index.md
- https://www.chromium.org/developers/calendar
- https://github.com/microsoft/winrtc
- https://docs.microsoft.com/en-us/winrtc/getting-started
- https://groups.google.com/g/discuss-webrtc \
See "PSA" posts for release information.
- https://bugs.chromium.org/p/webrtc/issues/list
- https://stackoverflow.com/questions/27809193/webrtc-not-building-for-windows
- https://github.com/aisouard/libwebrtc/issues/57

## Windows - M84

WebRTC's M84 release is currently used because it corresponded to Microsoft's latest WinRTC release at the time of development,
and WinRTC is a source of potentially useful patches.

The following notes document how the M84-based Windows VCPKG was created, using Visual Studio 2019.

### Set Up depot_tools

Install Google's depot_tools.
- Download depot_tools.zip.
- Extract somewhere.
- Add the extracted directory to the start of the system `PATH` environment variable.

Configure depot_tools.
- Set an environment variable `DEPOT_TOOLS_WIN_TOOLCHAIN=0`
- Set an environment variable `GYP_MSVS_VERSION=2019`

Initialize depot_tools.
- VS2019 developer command prompt in the directory where the source tree will be created.
- `gclient`

### Get the Code

Fetch the code into a *\src* subdirectory. This may take some time!
- `fetch --nohooks webrtc`

Switch to the M84 branch.
- `cd src`
- `git checkout branch-heads/4147`

Fetch all the subrepositories.
- `gclient sync -D -r branch-heads/4147`

### Patch the Code

#### Modify compiler switches
- Edit *build\config\win\BUILD.gn*:
- Change all `/MT` to `/MD`, and `/MTd` to `/MDd`.
- Change all `cflags = [ "/MDd" ]` to `[ "/MDd", "-D_ITERATOR_DEBUG_LEVEL=2", "-D_HAS_ITERATOR_DEBUGGING=1" ]`.
- Edit *build\config\compiler\BUILD.gn*:\
Change:
```
if (is_win) {
if (is_clang) {
cflags = [ "/Z7" ] # Debug information in the .obj files.
} else {
cflags = [ "/Zi" ] # Produce PDB file, no edit and continue.
}
```
to:
```
if (is_win) {
if (is_clang) {
cflags = [ "/Z7", "/std:c++17", "/Zc:__cplusplus" ] # Debug information in the .obj files.
} else {
cflags = [ "/Zi", "/std:c++17", "/Zc:__cplusplus" ] # Produce PDB file, no edit and continue.
}
```

#### H265 Codec Fixes
https://bugs.webrtc.org/9213#c13
- Edit the following files:
- *modules\video_coding\codecs\h264\h264_color_space.h*
- *modules\video_coding\codecs\h264\h264_decoder_impl.h*
- *modules\video_coding\codecs\h264\h264_encoder_impl.h*
In each, comment out the following lines:
```
#if defined(WEBRTC_WIN) && !defined(__clang__)
#error "See: bugs.webrtc.org/9213#c13."
#endif
```
- Edit *third_party\ffmpeg\libavcodec\fft_template.c*:\
Comment out all of `ff_fft_init` except the fail clause at the end.
- Edit *third_party\ffmpeg\libavcodec\pcm.c*:\
Comment out last line, containing `PCM Archimedes VIDC`.
- Edit *third_party\ffmpeg\libavutil\x86\imgutils_init.c*:\
Add the following method to the end of the file:
```
void avpriv_emms_asm(void) {} // Fix missing symbol in FFMPEG.
```

#### Exclude BoringSSL
A separate OpenSSL VCPKG is used for building Vircadia.
The following patches are needed even though SSL is excluded in the `gn gen` build commands.
- Rename *third_party\boringssl* to *third_party\boringssl-NO*.
- Edit *third_party\libsrtp\BUILD.gn:\
Change:
```
public_deps = [
"//third_party/boringssl:boringssl",
]
```
To:
```
public_deps = [
# "//third_party/boringssl:boringssl",
]
```

- Edit *third_party\usrsctp\BUILD.gn*:\
Change:
```
deps = [ "//third_party/boringssl" ]
```
To:
```
deps = [
# "//third_party/boringssl"
]
```
- Edit *base\BUILD.gn*:\
In the code under:
```
# Use the base implementation of hash functions when building for
# NaCl. Otherwise, use boringssl.
```
Change:
```
if (is_nacl) {
```
To:
```
# if (is_nacl) {
if (true) {
```
- Edit *rtc_base\BUILD.gn*:\
Change:
```
if (rtc_build_ssl) {
deps += [ "//third_party/boringssl" ]
} else {
```
To:
```
if (rtc_build_ssl) {
# deps += [ "//third_party/boringssl" ]
} else {
```

### Set Up OpenSSL

Do one of the following to provide OpenSSL for building against:
a. If you have built Vircadia, find the **HIFI_VCPKG_BASE** subdirectory used in your build and make note of the path to and
including the *installed\x64-windows\include* directory (which includes an *openssl* directory).
a. Follow https://github.com/vircadia/vcpkg to install *vcpkg* and then *openssl*. Make note of the path to and including the
*packages\openssl-windows_x64-windows\include* directory (which includes an *openssl* directory).

Copy the *\<path\>\openssl* directory to the following locations (i.e., add as *openssl* subdirectories):
- *third_party\libsrtp\crypto\include*
- *third_party\usrsctp\usrsctplib\usrsctplib*

Also use the path in the `gn gen` commands, below, making sure to escape `\`s as `\\`s.

### Build and Package

Use a VS2019 developer command prompt in the *src* directory.

Create a release build of the WebRTC library:
- `gn gen --ide=vs2019 out\Release --filters=//:webrtc "--args=is_debug=false is_clang=false use_custom_libcxx=false libcxx_is_shared=true symbol_level=2 use_lld=false rtc_include_tests=false rtc_build_tools=false rtc_build_examples=false proprietary_codecs=true rtc_use_h264=true enable_libaom=false rtc_enable_protobuf=false rtc_build_ssl=false rtc_ssl_root=\"<path>\""`
- `ninja -C out\Release`

Create a debug build of the WebRTC library:
- `gn gen --ide=vs2019 out\Debug --filters=//:webrtc "--args=is_debug=true is_clang=false use_custom_libcxx=false libcxx_is_shared=true enable_iterator_debugging=true use_lld=false rtc_include_tests=false rtc_build_tools=false rtc_build_examples=false proprietary_codecs=true rtc_use_h264=true enable_libaom=false rtc_enable_protobuf=false rtc_build_ssl=false rtc_ssl_root=\"<path>\""`
- `ninja -C out\Debug`

Create VCPKG file:
- Assemble files in VCPKG directory structure: Use the *copy-VCPKG-files-win.cmd* batch file per instructions in it.
`cd ..`\
`copy-VCPKG-files-win`
- Zip up the VCPKG *webrtc* directory created by the batch file.
`cd vcpkg`\
`7z a -tzip webrtc-m84-yyyymmdd-windows.zip webrtc`
- Calculate the SHA512 of the zip file. E.g., using a Windows PowerShell command window:\
`Get-FileHash <filename> -Algorithm SHA512 | Format-List`
- Convert the SHA512 to lower case. E.g., using Microsoft Word, select the SHA512 text and use Shift-F3 to change the case.
- Host the zip file on the Web.
- Update *CONTROL* and *portfile.cmake* with version number (revision date), zip file Web URL, and SHA512.

### Tidying up

Disable the depot_tools:
- Rename the *depot_tools* directory so that these tools don't interfere with the rest of your development environment for
other work.


## Linux - M81

The original, High Fidelity-provided WebRTC VCPKG library is used for AEC (audio echo cancellation) only.

**TODO:** Update to M84 and include WebRTC components per Windows WebRTC.


## MacOS - M78

The original, High Fidelity-provided WebRTC VCPKG library is used for AEC (audio echo cancellation) only.

**TODO:** Update to M84 and include WebRTC components per Windows WebRTC.
Loading

0 comments on commit aabcdea

Please sign in to comment.