A structured discussions platform — brings together the main features from StackOverflow, Slack, Discourse, Reddit/HackerNews, and Disqus blog comments.
- Save time. People find help "instantly", in Question & Answers topics.
- Crowdsource feedback and ideas. Discuss, upvote, choose the right things to do.
- Collaborate, in team chats.
- Talk with your blog readers, in embedded comments.
For your co-workers / customers / students / volunteers / contributors / users.
You can use Slack and Mattermost etc for day to day teamwork, combined with Talkyard for the more important discussions that you want to find in a nice-to-read format, months and years later.
Or use Talkyard for customer support, or for schools and students helping each other.
How does Talkyard compare with
StackOverflow for Teams, Discourse, Slack, Facebook Groups, Disqus?
— Find out here.
This Git repository is for development.
Installation instructions are elsewhere: https://github.com/debiki/talkyard-prod-one.
There's a hosting service: https://www.talkyard.io.
Support forum here (& live "demo"), at Talkyard.io — and report bugs there too.
You'll find Docker image build files in: ./images/image-name/
This is beta software; there might be bugs.
The good answers surface to the top.
See if others disagree with something, so you can avoid following the wrong advice.
You leave for lunch, or go home over the night — and return to a Question-Answers topic the next day. Did someone post more replies, when you were away? You want to find and read?
You can directly find the most recent answers and replies: Open the sidebar, click the Recent tab, and find the most recent replies, click to scroll.
(But at StackOverflow, Reddit, HackerNews etc, it's hard to find them (since the discussions are threaded). You need to carefully scan the whole discussion from top to bottom.)
Notifications via email, as of now. Some time later, there'll be a PWA mobile app with push notifications.
Like Disqus — but lightweight, no ads, no tracking. Configure in the Admin Area, the Settings tab, the Embedded Comments sub tab. — Read more about blog comments, and an optional hosting service, here: https://www.talkyard.io/blog-comments
There's a Disqus importer — talk with us in the forum if you want to migrate from Disqus to Talkyard.
Planned, not yet implemented: Let your students ask anonymous questions. Maybe they feel ashamed for not knowing? — Or let your co-workers submit ideas and feedback, anonymously. Maybe they feel worried their ideas has crossed the border from Creative to Crazy? They can un-anonymize themselves later if they want to. (Anonymous posting first needs to be enabled, by admins. Here you can read more: https://www.talkyard.io/-239/is-there-anonymous-messages-support-for-the-full-talkyard )
Say hello in the community: https://www.talkyard.io/forum/.
How to translate to a new language: i18n-README (step 1 and 2 only). — Open any translations pull request to the ty-translations repo: https://github.com/debiki/ty-translations.
You need to agree to this Contributor License Agreement. You do that by reading it (there's a human friendly intro) and appending a single line paragraph with your real name (no pseudonyms) and the following text, to all your Git commit messages: (note: a Git commit message is not the same thing as a GitHub comment)
I, Your Full Name <your@email.address>, agree to the Contributor License Agreement, docs/CLA-v2.txt.
Please squash your commits to just one (unless you're doing something that's easier to review in separate commits).
This repo is for development. To install and use the software, instead go here: talkyard-prod-one
As operating system, you can use Debian 10, probably Debian 11, Ubuntu 20.04 should work too. We use Debian 9 and 10, soon Debian 11.
You need about 6 GB RAM for the development environment (whereas the production environment needs about 2 GB). And an internet connection — you'll download perhaps 1 GB Docker images and other files.
Install Docker and Docker-Compose, see: https://docs.docker.com/compose/install/. On Linux, you can:
sudo -i
curl -fsSL https://get.docker.com -o install-docker.sh
sh install-docker.sh
curl -L "https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version # should print "docker-compose version ... build ..."
Read A brief intro to Docker-Compose — unless you know how to use docker-compose already.
-
Append some settings to the system config, so ElasticSearch will work. ElasticSearch will get downloaded later, in a Docker image (don't install it yourself). — Run this as one single command, not one line at a time:
sudo tee -a /etc/sysctl.conf <<EOF ################################################################### # Talkyard settings # Up the max backlog queue size (num connections per port), default = 128 net.core.somaxconn=8192 # ElasticSearch requires (at least) this, default = 65530 # Docs: https://www.kernel.org/doc/Documentation/sysctl/vm.txt vm.max_map_count=262144 # VSCode and IntelliJ Idea want to watch many files — without this, there'll be # a "Unable to watch for file changes in this large workspace" error in VSCode. # Also an "User limit of inotify watches reached" error can happen, when # tailing logs, if too few watches. (The default is sometimes only 8192.) fs.inotify.max_user_watches=524288 EOFReload the system config:
sudo sysctl --system -
You need Git, Make, cURL,
jqfor viewing logs,gpg2for checking signatures, and a file change notifier:sudo apt install git make curl jq gpg2 inotify-tools -
(Optional step, for now.) Install Nix-shell, see https://nixos.org/download.html#nix-verify-installation. Nix gives you all build tools, no need to modify your host OS. For example, Nodejs 14. And later, Deno, and Rust build stuff.
Get the installation script and signature:
curl -o install-nix-2.3.15 https://releases.nixos.org/nix/nix-2.3.15/install curl -o install-nix-2.3.15.asc https://releases.nixos.org/nix/nix-2.3.15/install.ascImport the public signing key:
# Try this: gpg2 --recv-keys B541D55301270E0BCF15CA5D8170B4726D7198DE # Or, if that won't work, the key is in the vendors/ dir: gpg2 vendors/nixos-signing-pub-key.gpg # should print: B541D55301270E0BCF15CA5D8170B4726D7198DE # then: gpg2 --import vendors/nixos-signing-pub-key.gpgVerify the signature — this should say "Good signature", and print:
Primary key fingerprint: B541 D553 0127 0E0B CF15 CA5D 8170 B472 6D71 98DE:gpg2 --verify ./install-nix-2.3.15.ascInstall Nix and, optionally, Niv. Either 1/2: First create the
/nixdirectory for yourself, and then install — no need to be root: (see https://nixos.wiki/wiki/Nix_Installation_Guide)sudo install --directory --mode=755 --owner=$(id -u) --group=$(id -g) /nix sh ./install-nix-2.3.15 # not root # nix-env -iA nixpkgs.niv # optionally, if you want to upgrade Ty's build tools(To uninstall Nix, remove
/nix, a line in~/.profile, and:~/{.nix-channels,.nix-defexpr,.nix-profile,.config/nixpkgs})Or alternatively, 2/2: Do a multi-user installation of Nix: https://nixos.org/manual/nix/stable/#sect-multi-user-installation (then I'd think it's best to use the above
./install-nix-...script whose signature you've verified). -
Clone the Talkyard repository:
git clone https://github.com/debiki/talkyard.git talkyard cd talkyard -
Download dependencies — they're in Git submodules: (this might take long — you'll be downloading about 1 GB files)
git submodule update --init -
Start Talkyard: (
s/tydis a Talkyard development helper script)s/tyd upAnd have a coffee — it takes a while before Talkyard starts the first time: Typescript, Stylus and Scala code gets compiled and packaged, Docker images get built. To view log messages, type
s/tyd logs -f(ordocker-compose logs -f).Wait for these "Server started" log messages to appear:
app_1 | app_1 | --- (Running the application, auto-reloading is enabled) --- app_1 | app_1 | [info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000 app_1 | [info] p.c.s.NettyServer - Listening for HTTPS on /0:0:0:0:0:0:0:0:9443 app_1 | app_1 | (Server started, use Ctrl+D to stop and go back to the console...) app_1 | -
Point your browser to http://localhost/. This sends a request to the Docker container named 'web', in which Nginx listens on port 80. Nginx sends the request to Play Framework in the 'app' container, port 9000. Play Framework then starts compiling more Scala files — this take a little while; the browser might show a 502 Bad Gateway error message.
Soon, when done compiling, Play Framework will start. Then this gets logged:
app_1 | [info] application - Starting... [TyMHELLO] ... ... app_1 | [info] application - Started. [TyMSTARTED]If you don't see these messages (maybe they scroll past too fast), you can continue with the next step just below anyway — just keep reloading the browser page until any "is starting" message in the browser window disappears.
-
Create a forum
Reload the browser at http://localhost/. A page with a button should appear. Sign up as admin with this email:
[email protected](must be that email). As username and password you can typeadminandpublic1234.
Shut down everything like so: s/tyd kill. (To remove all containers: s/tyd down.)
To edit Typescript code (in client/app-*/) you can use VSCode.
To automatically recompile code you've edited, so the changes appear in your browser after reload, do this:
s/tyd tw # 'tw' means "transpile" and "watch"
But wait five seconds, after you've edited any Typescript code, before you reload the page in the browser — otherwise the Typescript code might not yet have been transpiled. (Details: There's a Docker container with Node.js installed, which recompile Typescript and Stylus CSS.)
To edit Scala code (in app/*), you can use IntelliJ IDEA, the free community edition.
Don't forget to install a Java Development Kit (JDK) — in Debian 9:
sudo apt install default-jdk # installs JDK 8
There's a container, named app, which runs the Play Framework application server, and looks for changes to Scala files, recompiles and reloads.
Unfortunately, if you keep editing and reloading Scala files many many times, then eventually
Play Framework runs out of memory. Restart it like so: s/tyd ra ('ra' means "restart app").
Do this: (make is GNU Make)
vi version.txt # type a new vesion number
vi .env # change DOCKER_REPOSITORY to your own repository
make prod-images # this runs tests and builds production images
make tag-and-push-latest-images tag=... # pushes images to your repo
You can type make to see help about the Makefile targets.
All this has been tested in Ubuntu and Mint Linux only, with Bash. If you're on Windows, probably you'll need Cygwin or MinGW.
To use the images in your own Docker-Compose installation, have a look here: https://github.com/debiki/talkyard-prod-swarm
Here you can read about the various images in the Talkyard stack: about-the-talkyard-images.md.
See tips.md.
The end-to-end tests are written in TypeScript and uses Selenium and Webdriver.io.
See the end-to-end tests readme.
And, you also need to make *.localhost addresses work,
because the e2e tests creates test sites at *.localhost sub domains.
The security tests are written in TypeScript and use Tape = test-anything-protocol for Node.js. See the security tests readme.
Stop everything: sudo docker-compose down and then: s/cli then type test + hit Enter.
Install Scala SBT, see http://www.scala-sbt.org/download.html. On Linux:
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
sudo apt-get update
sudo apt-get install sbt
Append to /etc/security/limits.conf ... hmm but now with Docker-Compose, which container?
your_login_name hard nofile 65535
your_login_name soft nofile 65535
Configure very high max-requests-per-ip-per-second etc Nginx limits — otherwise during the performance test Nginx will start to rate limit stuff and reply 503 Service Not Available:
sudo docker-compose -f docker-compose.yml -f docker-compose-no-limits.yml up -d
- Client: React.js, TypeScript, Webdriver.io.
- Server: Scala and Play Framework. OpenResty, some Lua. React.js in Java's Nashorn Javascript engine.
- Databases: PostgreSQL, Redis, ElasticSearch.
This project looks like so:
server/
|
+-Makefile <-- You can build Talkyard, using GNU Make
| (work in progress, as of Nov 2018)
|
+-docker-compose.yml <-- Tells Docker how to run Talkyard
|
+-client/ <-- Javascript, CSS, React.js components
| +-app/ <-- Client side code
| +-server/ <-- React.js components rendered server side,
| : usually softlinks to ../app/
| :
|
+-app/ <-- Scala code — a Play Framework 2 application
|
+-tests/
| +-app/ <-- Unit tests and functional tests, for the app server
| +-e2e/ <-- End-to-end tests
| +-security/ <-- Security tests
|
+-modules/
| +-ty-dao-rdb/ <-- A database access object (DAO), for PostgreSQL
| +-ed-core/ <-- Code shared by the DAO and by the ./app/ code
| +-ed-prod-one-test/ <-- A production installation, for automatic tests
| |
| ...Third party modules
|
+-to-talkyard/ <-- For converting e.g. phpBB or Disqus export files
| to Talkyard's JSON format, so they can be imported
| into Talkyard. This is a stand-alone Node.js app.
|
+-images/
| +-web/ <-- For building the 'web' Docker image, runs Nginx
| | +-Dockerfile
| | +-assets/ <-- Typescript and Stylus compiled to JS and CSS
| | +-modules/ <-- Nginx (OpenResty) and Lua modules
| | +-openresty/ <-- OpenResty source code (we build from soruce)
| | ...
| |
| +-gulp/ <-- An image that runs Node.js and bundles JS and CSS
| |
| +-... <-- More images, see: docs/about-the-talkyard-images.md
|
+-volumes/
| +-rdb-data <-- Mounted as a volume in the Postgres container
| +-gulp-home/ <-- Gulp container home-dir = disk cache
| +-uploads <-- Mounted read-write in the Play container, but
| | read-only in Nginx (to serve static files)
| ...
|
+-d/ <-- Docker scripts, e.g. d/c is docker-compose, and
| 'd/n a-script' runs a-script in a Nodejs Docker container
| ('docker-compose' is soo loong to type).
| Many scripts still in s/* instead, could move to here?
|
+-s/ <-- Utility Bash scripts (typing "scripts/" is so long),
| e.g. s/run-e2e-tests.sh, or, to run in a container:
| 'd/n s/run-e2e-tests.sh' — combining the d/* and s/*
| scripts.
|
+-conf/ <-- App server (Play Framework) config files
| +-my.conf <-- You can add your localhost config here. Ignored by Git.
Example: s_P_By_FN-Gst. Here, s_ is a prefix used for all classes, and
it means "some". For ids we use t_ instead, means "the". P means Post. By means
who-was-it-written-By. FN means Full Name. Gst means Guest.
So, this is BEM (Block Element Modifier) with a few tweaks: Blocks/elements are separated with
only one underscore, and modifiers with only one dash. Blocks, elems and modifiers always
start with uppercase — because then it's easy to tell if we're dealing with an abbreviation
or not. For example, FN (full name) is an abbreviation. But By is not (since it continues with
lowercase letters).
Another example: s_Dfs_Df_Ttl — this means the title (Ttl), of a draft (Df),
in a list of drafts (Dfs). You'll find abbreviations like Ttl and Df, in
bem-blocks.txt.
For stuff with otherwise no class or id, and that should be clicked in end-to-end tests,
we use classes only, and the prefix e_ (instead of s_ or t_).
In Typescript (and any Javascript), use single quotes for strings the computer cares about,
like CSS classes or ids, e.g. className: 's_P' or reactRenderMethod = 'hydrate',
or React component display names.
For texts that humans read, instead use double quotes, like: Button({ ...}, "Undo").
When doing this, you can be fairly certain that if you edit a single quote string,
without knowing what you're doing, something will break.
Whilst if you edit a double quoted string and fix e.g. a spelling errors: the computer
won't care, but humans like it.
Some parts of a software system, knows how other parts of the software system works,
sometimes in not-obvious ways. Make such otherwise hidden duplicated knowledge visible,
by tagging the code with tags like: [1ABCDE2].
Example: // Also done here: [4JKAM7] when deleting pages..
Or there's a 3rd partly lib bug workaround in one source code file, for a problem that happens
in a different file, and an end-to-end test that got affected, example: [5QKBRQ].
Tag those three places with the same tag.
Just type a number, random uppercase letters, and another number, to create a tag.
And copy-paste it to where the related code is.
Log messages, and plain text messages sent back to the browser, start with TyM if it's
an info message, and TyE if it's an error. Like, "Started. [TyMSTARTED]" (a log message).
These messsage codes helps you instantly find the relevat source code, if there's
an error message anywhere. Otherwise, it can be terribly annoying,
when the browser says "Not found", and you have no idea where that message comes from.
For example, Nginx didn't find a location handler? Or a user is missing? Or a page? Or a post?
Or a client side route is missing? Or the hostname is wrong? Or ...?
And you search for "Not found" and find 1 000 matches.
Now, instead, you'll see "Not found [TyE123ABC]" — and you then search for "TyE123ABC"
and find the relevant source code.
Some message codes sent to the browser are checked for in end to end tests. They shall
have an underscore _ at the end (because it's called end to end tests). So, if you see a
message code like: "TyM0APPR_" and you change it, you need to search for it
everywhere and update some end-to-end tests too.
Some message codes are checked for by production code Typescript, i.e. frontend code.
They shall have a _ at the beginnign (front) of the error code, and here's how they can be used
server side: throwForbidden("_TyE403BPWD", "Bad username or password") and
client side: if (xhr.responseText.indexOf('_TyE403BPWD') .... — So, when you're looking at the
server side code the _ tells you that the error code is used in the frontend Typescript code,
so you cannot just change it.
Add "as many constraints and foreign keys as you can". Knowing precisely what's in the
database, makes data migrations safer & simpler. Don't forget to index
foreign key columns. Type ix: index-name next to the foreign key constraints,
so one directly sees that there's an index, and its name. All fk constraints should
be deferrable.
Table names: some_table3. The trailing digit "3" is for historical reasons —
maybe change to "_t" instead? Some suffix is needed, so one can search
for table_name3 e.g. pages3 and find only the table, instead of 999 999
other unrelated "table_name" e.g. "pages" search hits.
Unique indexes: tablename_u_columnone_coltwo_three...
Non-unique indexes: tablename_i_colone_coltwo_three...
Foreign keys: tablename_colone_coltwo_..._r_othertable(_col_coltwo_...)
Check constraints: tablename_c_...sth-to-check... e.g. pages_c_id_len (length).
It's nice to have these one letter object types: _u, _i, _c because then they're
more full text searchable — e.g. find all usages of any (unique) indexes on a certain
table, accross the whole code base. And you soon learn to recognize what they mean so you know
what type of thing it is, without having to think or find out. Having them directly
after the table name (without the '3' suffix) makes things nicely aligned & easy to scan,
when typing \d some_table_name in the psql client.
Maybe it'd be nice if all column names ended with "_c"? E.g. username_c.
Because currently they're a bit hard to search for. Instead one needs to search
for the table name, and then look for the column name, in the surrounding code.
Most db things don't follow these naming standards, because I (KajMagnus) had different ideas in the past. Maybe one day, time to rename everything to the correct names?
Source code comments should be concise, but writing "he or she" everywhere, when referring to e.g. a user, becomes a bit verbose (because "he or she" is three words). There's a short Swedish word that means "he or she", namely "hen". Let's start using it in English.
So: "hen" = either "he or she", or "him or her", depending on context. And "hens" = "his or her", and "hen's" = "he or she is".
To refer to many hen = many-he-or-she, write "people". "Hens" however means "his or her", just like "its" means, well, "its" (but not "things").
What about the bird previously called "hen"? Let's call it "henbird" instead.
So, hereafter, the word "hen" means "he or she". And the henbird, which I cannot remember having mentioned or even thought about the past year, no longer gets to occupy the short and useful word "hen".
We're building & using a smaller version of Lodash, like so: (this makes slim-bundle.min.js.gz 8kb = 4% smaller, as of September 2016)
# COULD create custom typedef for this custom Lodash, so won't accidentally use
# Lodash fns that have been excluded here?
node_modules/lodash-cli/bin/lodash \
include="assign,assignIn,before,bind,chain,clone,cloneDeep,compact,concat,create,debounce,defaults,defer,delay,each,escape,every,filter,find,findLast,flatten,flattenDeep,forEach,forOwn,groupBy,has,head,includes,identity,indexOf,isArguments,isArray,isBoolean,isDate,isEmpty,isEqual,isFinite,isFunction,isNaN,isNull,isNumber,isObject,isRegExp,isString,isUndefined,iteratee,keys,last,map,mapValues,matches,max,min,mixin,negate,noConflict,noop,once,partition,pick,reduce,remove,result,size,slice,some,sortBy,sumBy,take,tap,throttle,thru,toArray,uniq,uniqBy,uniqueId,value,values" \
--output client/third-party/lodash-custom.js
# If you upgrade Lodash, afterwards, have a look at the diff of `lodash-custom.js`
# and scroll down towards the bottom. Look at the line `lodash.cloneDeep =` and below:
# did any function unexpectedly disappear? If so, include it in the long list above
# and regenerate `lodash-custom.js`.
- For security reasons, we checkin only the resulting
.jsfile (but not the.min.js) file into source control (so that you can read the source code and see what it does). - There are some Gulp plugins that builds Lodash but one seems abandonend (gulp-lodash-builder) and the other (gulp-lodash-custom) analyzes all .js files, I guess that'd slow down the build rather much + won't immediately work with Typescript?
Old code from before January 2015 is available here: https://github.com/debiki/debiki-server-old. That repo, squashed, is in this repo.
Copyright (c) 2010-2021 Kaj Magnus Lindberg and contributors.
Talkyard is multi licensed under 1) AGPLv3 or later, see LICENSE.txt, and 2) Business Source License v1.1, with change license GPLv2 or later, and change date 2026-01-01 (we bump the change date yearly, so it's between 4 and 5 years into the future) — see LICENSE-BUSLv1.1.txt. And 3) possibly other licenses.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the above-mentioned licence texts for more details.
vim: list et ts=2 sw=2 tw=0 fo=r