Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug Fixes, Page Aliases, Redaction, Local Media, Layout Customization, Asset Imports & Case (De)Sensitivity #247

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
fce906e
initial fork;
rj919 Jul 26, 2018
1d270f9
ignoring more local files;
rj919 Jul 26, 2018
2076a53
added media folder, redirects map, redaction regexps and layout confi…
rj919 Jul 27, 2018
e70b25a
added footer customization to layout configs; added layout customizat…
rj919 Jul 27, 2018
856bff7
clarified nomenclature in layout fields to use Width instead of Colum…
rj919 Jul 27, 2018
6facbc8
created module in lib for redaction and redirection routines; added r…
rj919 Jul 28, 2018
9fb1db2
added redaction routines to search, wiki and misc routes; added pageS…
rj919 Jul 28, 2018
39cf96a
added redactContent and pageSummaries to pug templates; fixed bug wit…
rj919 Jul 28, 2018
bb5f159
added case sensitive option to config for search and redirects; added…
rj919 Jul 29, 2018
f801016
added sequentialSections redaction option to config; TODO: add media …
rj919 Jul 29, 2018
8fedec1
added media folder to static path on folder; added mediaPath to gitme…
rj919 Jul 29, 2018
6a86bad
added glyphicons font set for bootstrap; added google font set for ub…
rj919 Jul 29, 2018
f60be55
fixed bug with empty mediaSubdir; fixed bug with solitary RegExp reda…
rj919 Jul 30, 2018
1563699
consolidated redaction fields; removed endRedaction from requirements…
rj919 Jul 30, 2018
f03bcb8
finished documentation of redaction options;
rj919 Jul 30, 2018
77e01dc
finished documentation of redaction options;
rj919 Jul 30, 2018
072b2b4
validated latestValue exists for sequentialSections expressions;
rj919 Jul 30, 2018
6755d12
minor change to documentation;
rj919 Jul 30, 2018
1c18725
change redaction default to false;
rj919 Jul 30, 2018
37008c5
renamed redirects option to aliases for additional clarity;
rj919 Jul 30, 2018
e28be04
created unittest for redactor module; added additional config options…
rj919 Jul 30, 2018
8975058
fixed syntax using standardjs; renamed variables using camelCase; upd…
rj919 Jul 31, 2018
063d78a
fixed bug with sidebar rendering in layout.pug
rj919 Jul 31, 2018
2a8cd3a
restored favicon.ico acquisition process since favicon url setting ca…
rj919 Aug 2, 2018
1c78fc2
added assets import support for css and js files found in mediaSubdir…
rj919 Aug 4, 2018
40c56ae
fixed bug associated with quotation marks in name of article when che…
rj919 Aug 21, 2018
87e3afd
extended caseSensitive option to all page retrieval; renamed caseSens…
rj919 Aug 27, 2018
3aaf9d1
updated documentation for case sensitivity disabling; TODO: integrate…
rj919 Aug 27, 2018
e9a812a
updated documentation for new pull request; TODO: integrate aliases i…
rj919 Aug 27, 2018
0e4a2d7
removed console log debugging lines; TODO: integrate aliases into pag…
rj919 Aug 28, 2018
63df6c3
added option to lift html block elements outside of paragraphs during…
rj919 Aug 30, 2018
1ca60d9
added extension field to config file
rj919 Jul 19, 2023
37b0644
fixed search to ensure only results after redactions are served;
rj919 Jul 20, 2023
ed7cfa8
added redaction capacity to Sidebar.md;
rj919 Aug 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,8 @@ node_modules/*
config.yaml
config.yml
data/*
.idea
__pycache__/
Dockerfile
.dockerignore
package-lock.json
106 changes: 106 additions & 0 deletions README_FORK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
Intro
-----
This is a fork of Jingo (at the time of v1.8.5). Modifications to the source code have been made in order to add a number of features to Jingo and fix a couple bugs.

Merge Status: __pending__

ChangeLog
---------
### List of Modifications
- fixed bug in gitExec when repo path contains one or more spaces
- fixed bug with markMissingPagesAsAbsent request for pages with ' in the name
- added redirection of synonyms to official page toggled in config
- added sidebar, main and footer column width customization to config
- added options to use regexp to redact exerpts of information for anonymous users to config
- added option to redact commit messages for anonymous users to config
- added option to redact entire pages from search and list for anonymous users to config
- added option to include initial paragraph in page list
- added media folder to static routes on page
- added override of standard favicon with any favicon in root of media folder
- added option to serve files from local version rather than CDN
- added option to disable case sensitivity in page links and aliases
- added option to lift block elements outside of paragraphs during markdown rendering

### Mod Strategy
All changes and/or additions have been marked with comments which begin with "// MOD" so as to make it easier to hunt down changes. Existing code has been preserved whenever possible through the use of configuration flags to enable the additional/optional functionality of this fork. Also, additional methods and modules have been employed using naming conventions so as to least possibly conflict with future changes to the main branch.

Additional Configuration Options
--------------------------------

#### application.mediaSubdir (string: "")

If you have media files you would like to host locally, such as a favicon, logo, images or css or js dependencies, inside a directory of the repository, specify its name here.

#### application.serveLocal (boolean: false)

With this option, you can use a local copy of a resource instead of a CDN. Currently, this only applies to the Ubuntu font families requested in the head of each page.

#### application.percolateBlocks (string: "div, blockquote")

With this option, you can add other block elements to the list of block elements which are lifted out of their location in paragraphs during markdown rendering. Separate each element tag type you would like to be placed outside of paragraphs with a comma (and optional space). Without this option, only table blocks are lifted outside of paragraphs.

#### features.pageSummaries (boolean: true)

With this option, the html rendered version of the first paragraph of each page will be included its listing when a user browses all the pages.

#### features.caseSensitive (boolean: false)

With this option, you can force aliases and page linking to only match a page if the cases match. For example, if you want to have any link to "Introduction" to redirect to Home.md, when this option is disabled, links to "introduction" would also redirect to Home.md.

#### assets.css (string: '')

With the assets.css string, you can add the file path to any number of css sheets to be imported on page rendering in the \<head\> element prior to the customization style. In this way, you can add dependencies to the customization style. Separate stylesheets should be delineated with a comma and all such sheets must be located in the mediaSubdir folder or they will not be imported.

#### assets.js (string: '')

With the assets.js string, you can add the file path to any number of js files to be appended to the bottom of the \<body\> element on page rendering prior to the customization script. In this way, you can add local dependencies to the customization script. Separate javascript files should be delineated with a comma and all such files must be located in the mediaSubdir folder or they will not be imported.

#### aliases (map)

If you would like to setup one or more aliases for a wiki entry, which will redirect to that page, you can include each alias and its associated page as a key:value pair in this map. Whenever a page request to an alias value is made, the server will return the page it is associated with and also include a line underneath the title of the page that indicates that the requested term redirects to this page. Since URLs cannot contain a space, all aliases with a word break need to replace that word break with a '-'. Whereas the alias should not contain .md, the page it redirects to must include .md in its name.

#### layout.sidebarWidth (integer: 2)

With this layout field, you can change the width of the sidebar content in desktop view between 0 and 10. The default width for Jingo is 2. The sidebarWidth + mainWidth (see below) cannot exceed 12.

#### layout.mainWidth (integer: 8)

With this layout field, you can change the width of the main page in desktop view between 2 and 12. The default width for Jingo is 8. The sidebarWidth (see above) + mainWidth cannot exceed 12.

#### layout.footerWidth (integer: 8)

With this layout field, you can change the width of the footer content in desktop view between 0 and 12. The default width for Jingo is 8.

#### layout.sidebarMobile (boolean: true)

With this layout field, you can disable the sidebar content (from appearing on top of the page) when viewed from mobile devices.

#### redaction.enabled (boolean: false)

When redaction is enabled, it is possible to remove certain content from the view generated for anonymous users. Redaction also removes the commit records and history pages from all content rendered for anonymous users. A number of different techniques are provided for redacting content (see below) using regular expression syntax. When redaction is enabled, all pages and searches will look for content which matches the regular expressions provided and remove it from anonymous views. A global, multi-line search is the default option for all regular expression patterns, so if matching new lines is important, they must be included in the regular expression. If any part of a redaction regular expression is grouped, then all parts that are not inside a grouping will be redacted for both anonymous and authenticated users. Regular expression occurs before pages go through HTML rendering and so will match the markdown as it appears for each entry.

#### redaction.hiddenPage (string: "^<!(--\s?Hidden[\s\S]*?--)>")

If any part of the document matches the regexp in hidden page, then the entire page will be removed from view from anonymous users and will return a 404 page error. In addition, the page will not show up in any search or list function. This is the only redaction expression that is not global, as it only requires one match to trigger.

#### redaction.privateContent (string: "<!(--\s?Private[\s\S]*?--)>([\s\S]*?)<!(--\s?End\s?--)>")

Any sections of the document which match a privateContent regexp will be redacted from the document. Content inside the redaction will not show up in any search result. If no grouping is found in the regexp, then the entire regexp will be returned for the authenticated user. Otherwise, only the content inside groupings will be returned to the authenticated user.

#### redaction.futureContent (string: "<!(--\s?\d{4}\.\d{2}\.\d{2}[\s\S]*?--)>([\s\S]*?)<!(--\s?End\s?--)>")

Any section of the document which matches an futureContent regexp will be tested to see if it should be redacted from anonymous users. This technique will construct a date from the first (4 to 10)integers it finds inside the matched content using the following order: YYYYMMDDHH and then see if that date exists beyond the current date. If the date it finds is in the future, it will redact the content from anonymous users. So, for example, an excerpt such as <!-- 2099.09.09 -->Future stuff<!-- End --> will be treated as content not to be revealed until on or after 20990909. If the date is in the past, it will reveal the content it finds in either the first group (if there is only one group in the regexp) or the second group (if there are two or more groupings). This technique requires at least one grouping to be specified in the regexp or it will have no effect.

#### redaction.sectionContent[0].expression (string: "<!(--\s?chapter-\d+[\s\S]*?--)>([\s\S]*?)<!(--\s?End\s?--)>")

Any section of the document which matches an expression in the list of sectionContent will be tested to see if it should be redacted from anonymous users. This technique will look for the first group of integers it finds inside each match and compare that integer to the one provided in the associated current value. If the integer it finds is greater than the current value, the section will be redacted from anonymous users. So, for example, an excerpt such as <!-- chapter-1000 -->A late chapter<!-- End --> would be redacted if the current chapter (current) is 9. Like the futureContent technique, if the current value is equal to or greater than the integer found, it will reveal the content it finds in either the first group (if there is only one group in the regexp) or the second group (if there are two or more groupings). This technique requires at least one grouping to be specified in the regexp or it will have no effect.

#### redaction.sectionContent[0].current (integer: 0)

This field is included in order to evaluate the highest value in an associated sectionContent expression which is public. Any value that is found to be higher than the current value will be redacted from anonymous users.






1 change: 1 addition & 0 deletions jingo
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ var refspec = config.get('application').remote.split(/\s+/)
Git.setup(config.get('application').git,
config.get('application').repository,
config.get('application').docSubdir,
config.get('application').mediaSubdir, // MOD add media subdirectory in config
refspec, function (err, version) {
if (err) {
console.log(err)
Expand Down
87 changes: 76 additions & 11 deletions lib/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ var gravatar = require('gravatar')
var passport = require('passport')
var methodOverride = require('method-override')
var flash = require('express-flash')
var fs = require('fs') // MOD import dependency for media folder mapping

var app

Expand All @@ -47,6 +48,30 @@ module.exports.initialize = function (config) {
app.locals.baseUrl = config.get('server').baseUrl
}

// MOD add assets to locals
app.locals.assets = {
css: [],
js: []
}
var cssArray = config.get('assets').css.split(',')
for (var i = 0; i < cssArray.length; i++){
if (cssArray[i].trim() && fs.existsSync(Git.mediaPath(cssArray[i].trim()))){
app.locals.assets.css.push(cssArray[i].trim())
}
}
var jsArray = config.get('assets').js.split(',')
for (var i = 0; i < jsArray.length; i++){
if (jsArray[i].trim() && fs.existsSync(Git.mediaPath(jsArray[i].trim()))){
app.locals.assets.js.push(jsArray[i].trim())
}
}
if (app.locals.assets.css){
console.log('Adding styles ' + JSON.stringify(app.locals.assets.css))
}
if (app.locals.assets.js){
console.log('Adding scripts ' + JSON.stringify(app.locals.assets.js))
}

// View helpers
app.use(function (req, res, next) {
res.locals = {
Expand Down Expand Up @@ -84,6 +109,43 @@ module.exports.initialize = function (config) {
}
return config.get('application').favicon.trim()
},
// MOD add layout configuration to locals
get sidebarWidth () {
return config.get('layout').sidebarWidth.toString()
},
get mainWidth () {
return config.get('layout').mainWidth.toString()
},
get footerWidth () {
return config.get('layout').footerWidth.toString()
},
get footerSidebarWidth () {
return (12 - config.get('layout').footerWidth).toString()
},
get sidebarMobile () {
return config.get('layout').sidebarMobile
},
// MOD add redaction and page summary toggles
get redactEnabled () {
return config.get('redaction').enabled
},
get redactContent () {
return config.get('redaction').enabled && !req.user
},
get pageSummaries () {
return config.get('features').pageSummaries
},
// MOD add serve local toggle
get serveLocal () {
return config.get('application').serveLocal
},
// MOD add asset imports
get cssAssets () {
return app.locals.assets.css
},
get jsAssets () {
return app.locals.assets.js
},

isAnonymous: function () {
return !req.user
Expand Down Expand Up @@ -143,7 +205,10 @@ module.exports.initialize = function (config) {
return method
}
}))

// MOD add files in media path to static files
if (Git.mediaPath()) {
app.use(express.static(Git.mediaPath()))
}
app.use(express.static(path.join(__dirname + '/../', 'public'))) // eslint-disable-line no-path-concat
app.use(cookieParser())
app.use(cookieSession({
Expand All @@ -152,11 +217,11 @@ module.exports.initialize = function (config) {
cookie: { maxAge: 30 * 24 * 60 * 60 * 1000 }
}))
app.use(session({ name: 'jingosid',
secret: config.get('application').secret,
cookie: { httpOnly: true },
saveUninitialized: true,
resave: true
}))
secret: config.get('application').secret,
cookie: { httpOnly: true },
saveUninitialized: true,
resave: true
}))
app.use(flash())
app.use(expValidator())

Expand All @@ -173,7 +238,7 @@ module.exports.initialize = function (config) {
if (/^\/auth\//.test(req.url) ||
/^\/misc\//.test(req.url) ||
(/^\/login/.test(req.url) && !config.get('authorization').anonRead)
) {
) {
return next()
}

Expand Down Expand Up @@ -212,10 +277,10 @@ module.exports.initialize = function (config) {
app.use('/wiki', wikiStatic.configure())

app.use(require('../routes/wiki'))
.use(require('../routes/pages'))
.use(require('../routes/search'))
.use(require('../routes/auth'))
.use(require('../routes/misc'))
.use(require('../routes/pages'))
.use(require('../routes/search'))
.use(require('../routes/auth'))
.use(require('../routes/misc'))

// Server error
app.use(function (err, req, res, next) {
Expand Down
Loading