The official Singapore government link shortener.
Go.gov.sg is the official Singapore government link shortener, built by the Open Government Products team in GovTech.
There are multiple reasons why we built an official government link shortener:
- URLs are too long to fit into tweets or SMSes, and difficult to remember
- Email clients might block other commercial link shorteners if they are listed as spam on their site
- Citizens are afraid of phishing when receiving a shortened link and unsure of where it goes
With Go.gov.sg, citizens are safe in the knowledge that the links are official and safe. Any public officer can log in with their government emails and immediately create short links with the official gov.sg domain.
Start by cloning the repository and installing dependencies.
# Clone the repository
git clone [email protected]:opengovsg/GoGovSG.git gogovsg
# Go inside the directory
cd gogovsg
# Install dependencies
npm installMake sure you have docker-compose version >= 1.23.1 and Docker version >= 18.09.0 installed. Then run:
npm run devDocker-compose will spin up a postgresql database and redis container to be connected to the backend server.
Once the setup is complete, the local version of GoGovSG can be accessed on your browser via localhost:3000.
Note that 3000 is the port number that the webpack dev server listens on; the backend server actually listens
on port 8080 instead.
Because redirects are served directly from the backend, shortlinks can be accessed via localhost:3000/shortlink,
but that is really being proxied to localhost:8080/shortlink. Also, given that GoGovSG will attempt to send
emails directly from your computer when running on localhost, there is a chance that the email might land in
spam or not be sent entirely. To mitigate this, we have set the one-time password for all log-in attempts
on localhost to be 111111.
Much of this step will involve setting up key infrastructure components since we do not have docker-compose to do that for us. On top of running the server, GoGovSG minimally requires the following infrastructure to be available:
- A PostgreSQL database (for storing short-long URL mappings)
- A Redis server (transient storage of sessions, one-time passwords, click statistics and frequently used shortlinks)
After these have been set up, set the environment variables according to the table below:
| Environment Variable | Required | Description/Value |
|---|---|---|
| NODE_ENV | Yes | production |
| DB_URI | Yes | The postgres connection string, e.g. postgres://postgres:postgres@postgres:5432/postgres |
| OG_URL | Yes | The origin url, used for both google analytics and circular-redirect prevention. E.g. https://go.gov.sg |
| AWS_S3_BUCKET | Yes | The bucket name used for storing file uploads. |
| REDIS_OTP_URI | Yes | Redis connection string, e.g. redis://redis:6379/0 |
| REDIS_SESSION_URI | Yes | Redis connection string, e.g. redis://redis:6379/1 |
| REDIS_REDIRECT_URI | Yes | Redis connection string, e.g. redis://redis:6379/2 |
| REDIS_STAT_URI | Yes | Redis connection string, e.g. redis://redis:6379/3 |
| REDIS_SAFE_BROWSING_URI | Yes | Redis connection string, e.g. redis://redis:6379/4 |
| SESSION_SECRET | Yes | For hashing browser sessions, e.g. change-this |
| VALID_EMAIL_GLOB_EXPRESSION | Yes | The glob expression used to test if a provided email address is valid. For safety, we have disabled the use of negations, ext-glob, glob stars (**) and braces, e.g. *@youremaildomain.com |
| GA_TRACKING_ID | No | The Google Analytics tracking ID, e.g. UA-12345678-9 |
| SENTRY_AUTH_TOKEN | No | To get relevant permissions to upload the source maps. |
| SENTRY_DNS | No | The Sentry DNS used for bug and error tracking. e.g. https://[email protected]/12345 |
| SENTRY_ORG | No | Our Sentry organisation name, e.g. example-org |
| SENTRY_PROJECT | No | The relevant Sentry project. e.g. project-prod |
| SENTRY_URL | No | The Sentry url. e.g. https://sentry.io/ |
| LOGIN_MESSAGE | No | A text message that will be displayed on the login page as a snackbar |
| USER_MESSAGE | No | A text message that will be displayed as a banner, once the user has logged in |
| ANNOUNCEMENT_MESSAGE | No | The message in the announcement displayed as a modal to users on login |
| ANNOUNCEMENT_TITLE | No | The title in the announcement displayed as a modal to users on login |
| ANNOUNCEMENT_SUBTITLE | No | The subtitle in the announcement displayed as a modal to users on login |
| ANNOUNCEMENT_URL | No | The hyperlink for the button in the announcement displayed as a modal to users on login |
| ANNOUNCEMENT_IMAGE | No | The image in the announcement displayed as a modal to users on login |
| CSP_REPORT_URI | No | A URI to report CSP violations to. |
| CSP_ONLY_REPORT_VIOLATIONS | No | Only report CSP violations, do not enforce. |
| CLOUDMERSIVE_KEY | No | API key for access to Cloudmersive. |
| SAFE_BROWSING_KEY | No | API key for access to Google Safe Browsing. |
Trigger the typescript compilation and webpack bundling process by calling npm run build.
Finally, start the production server by running npm start.
GoGovSG uses Travis to deploy to AWS Elastic Beanstalk. We also use Sentry.io to track client-side errors.
| Environment Variable | Required | Description/Value |
|---|---|---|
| AWS_ACCESS_KEY_ID | Yes | AWS credential ID used to deploy to Elastic and Modify files on S3 |
| AWS_SECRET_ACCESS_KEY | Yes | AWS credential secret used to deploy to Elastic Beanstalk and Modify files on S3 |
| AWS_EB_ENV_PRODUCTION, AWS_EB_ENV_STAGING | Yes | Elastic Beanstalk environment name |
| AWS_EB_APP_PRODUCTION, AWS_EB_APP_STAGING | Yes | Elastic Beanstalk application name |
| AWS_EB_BUCKET_PRODUCTION, AWS_EB_BUCKET_STAGING | Yes | S3 bucket used to store the application bundle |
| AWS_EB_REGION | Yes | AWS region to deploy to, e.g. ap-southeast-1 |
| EMAIL_RECIPIENT | Yes | Email for Travis notifications |
| PRODUCTION_BRANCH, STAGING_BRANCH | Yes | Name of Git branches for triggerring deployments to production/staging respectively |
| REPO | Yes | Docker container registry URI to push built images to |
| ROTATED_LINKS | No | List of comma separated path of links to rotate on the landing page |
| SENTRY_ORG | No | Sentry.io organisation name |
| SENTRY_PROJECT | No | Sentry.io project name |
| SENTRY_URL | No | Sentry.io URL e.g. https://sentry.io/ |
| SENTRY_DNS | No | Sentry.io endpoint to post client-side errors to |
We have yet to setup travis to automate these steps:
- Update package version
- Update credits opengovsg/credits-generator
- Upload pdf to S3 bucket
Use the following SQL functions defined in scripts folder to safely transfer link ownership.
To transfer a single link to a new email address (must be lowercase):
SELECT migrate_url_to_user('the-short-link', '[email protected]');To transfer all links belonging to an account to another account, specify the email accounts of both (must be in lowercase):
SELECT migrate_user_links('[email protected]','[email protected]');All source code resides in the src directory. Inside src, there is client and server directory. Frontend code (react, css, js and other assets) will be in client directory. Backend Node.js/Express code will be in the server directory.
Babel helps us to write code in the latest version of JavaScript. If an environment does not support certain features natively, Babel will help us to compile those features down to a supported version. It also helps us to convert JSX to Javascript. .babelrc file is used describe the configurations required for Babel. Below is the .babelrc file which I am using.
{
"presets": ["env", "react"]
}Babel requires plugins to do the transformation. Presets are the set of plugins defined by Babel. Preset env allows to use babel-preset-es2015, babel-preset-es2016, and babel-preset-es2017 and it will transform them to ES5. Preset react allows us to use JSX syntax and it will transform JSX to Javascript.
ESLint is a pluggable and configurable linter tool for identifying and reporting on patterns in JavaScript.
.eslintrc.json file (alternatively configurations can we written in Javascript or YAML as well) is used describe the configurations required for ESLint.
I am using Airbnb's Javascript Style Guide which is used by many JavaScript developers worldwide. Since we are going to write both client (browser) and server side (Node.js) code, I am setting the env to browser and node. Optionally, we can override the Airbnb's configurations to suit our needs. I have turned off no-console, comma-dangle and react/jsx-filename-extension rules.
Webpack is a module bundler. Its main purpose is to bundle JavaScript files for usage in a browser.
webpack.config.js file is used to describe the configurations required for webpack.
- entry: entry: ./src/client/index.js is where the application starts executing and webpack starts bundling. Note: babel-polyfill is added to support async/await. Read more here.
- output path and filename: the target directory and the filename for the bundled output
- module loaders: Module loaders are transformations that are applied on the source code of a module. We pass all the js file through babel-loader to transform JSX to Javascript. Fonts and images are loaded through file-loader.
- Dev Server: Configurations for the webpack-dev-server which will be described in coming section.
- plugins: clean-webpack-plugin is a webpack plugin to remove the build folder(s) before building. html-webpack-plugin simplifies creation of HTML files to serve your webpack bundles. It loads the template (public/index.html) and injects the output bundle.
Webpack dev server is used along with webpack. It provides a development server that provides live reloading for the client side code. This should be used for development only.
The devServer section of webpack.config.js contains the configuration required to run webpack-dev-server which is given below.
devServer: {
port: 3000,
open: true,
proxy: {
"/api": "http://localhost:8080"
}
}Port specifies the Webpack dev server to listen on this particular port (3000 in this case). When open is set to true, it will automatically open the home page on startup. Proxying URLs can be useful when we have a separate API backend development server and we want to send API requests on the same domain. In our case, we have a Node.js/Express backend where we want to send the API requests to.
ts-node-dev will monitor for any changes in the server source code and automatically restart the server. This is used in development only. It watches changes to all files required from the entry point onwards, and will restart the node server whenever such files are modified.
ts-node-dev also allows debugging via Inspector. The quickest way to debug the application will be to use Chrome DevTools via chrome://inspect. VSCode users may want to add the following to their launch.json to quickly attach VSCode to the application for debugging:
{
"type": "node",
"request": "attach",
"name": "Inspect",
"protocol": "inspector",
"port": 9229,
"restart": true,
"cwd": "${workspaceFolder}"
}Express is a web application framework for Node.js. It is used to build our backend API's.
src/server/index.ts is the entry point to the server application. This starts a server and listens on port 8080 for connections. It is also configured to serve the static files from dist directory.
Concurrently is used to run multiple commands concurrently. I am using it to run the webpack dev server and the backend node server concurrently in the development environment. Below are the npm/yarn script commands used.
"client": "webpack-dev-server --mode development --devtool inline-source-map --hot",
"server": "nodemon src/server/index.js",
"dev": "concurrently \"npm run server\" \"npm run client\""VSCode is a lightweight but powerful source code editor. ESLint takes care of the code-quality.
Developer Tools to power-up Redux development workflow.
It can be used as a browser extension (for Chrome, Edge and Firefox), as a standalone app or as a React component integrated in the client app.