Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have URLs periodically checked #673

Merged
merged 85 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
79b61f6
Testing git on terminal
Sep 7, 2023
f84d7bf
Revert changes
Sep 7, 2023
46dd84c
Initial pass at URL checker
Sep 7, 2023
2747750
Test URL checker by running every 2 mint
Sep 7, 2023
df75156
Fix syntax
Sep 7, 2023
2a74a4f
Syntax
Sep 7, 2023
4be7989
chatgpt?
Sep 7, 2023
80ec10d
Fix spacing
howardbaik Sep 12, 2023
3f8d0e5
Remove "shell:bash"
howardbaik Sep 12, 2023
407e83d
Please work
howardbaik Sep 12, 2023
d60a9d8
Add spacing and indents
cansavvy Sep 20, 2023
30b104d
Fix indentation
cansavvy Sep 20, 2023
4f35605
Tryna find this indentation
cansavvy Sep 20, 2023
e154c6b
Add config option
cansavvy Sep 20, 2023
fa79700
Add workflow dispatch button too
cansavvy Sep 20, 2023
77ab7e7
temporarily have a pull_request trigger
cansavvy Sep 20, 2023
1a4c12a
Add URL config
cansavvy Sep 20, 2023
a4e0f1f
Add a delete step
cansavvy Sep 20, 2023
fa608e5
Fix spec
cansavvy Sep 20, 2023
7decb8e
Update for branch problem
cansavvy Sep 20, 2023
65df96d
Add issue creating infrastructure
cansavvy Sep 20, 2023
6aa1c3a
Add issue template
cansavvy Sep 20, 2023
e747bbc
Add steps:
cansavvy Sep 20, 2023
6242dc0
Declare docker
cansavvy Sep 20, 2023
4a8942e
get rid of dash
cansavvy Sep 20, 2023
2df94c5
Move if
cansavvy Sep 20, 2023
2b947cc
Add a needs argument
cansavvy Sep 20, 2023
b84b2d5
Add an option for it branch exists
cansavvy Sep 20, 2023
26b840f
missing paranthesis
cansavvy Sep 20, 2023
46ca898
Make sure its checked out
cansavvy Sep 20, 2023
38091e7
Try mapfile?
cansavvy Sep 20, 2023
8dec796
Add a file
cansavvy Sep 20, 2023
a1fa04a
Didn't fix template issue
cansavvy Sep 20, 2023
3a4dac5
Put file link
cansavvy Sep 20, 2023
7cd6c56
Fix link
cansavvy Sep 21, 2023
05563a1
Fix less than or greater than
cansavvy Sep 21, 2023
da60ae3
Add in a spelling error to check if this works
cansavvy Sep 21, 2023
2a9b34b
Add spelling errors to test part 2
cansavvy Sep 21, 2023
91e68e3
Try to fix
cansavvy Sep 21, 2023
418686c
Reverse order or < and =
cansavvy Sep 21, 2023
6ed286d
Change branch name
cansavvy Sep 21, 2023
500506c
Make sure branch is checked out
cansavvy Sep 21, 2023
c3f6ac8
Look for an issue
cansavvy Sep 21, 2023
641875a
Trying a different issue finder
cansavvy Sep 21, 2023
3c24629
Gonna make my own issue finder
cansavvy Sep 22, 2023
590933f
Add find issue bits
cansavvy Sep 22, 2023
feb31b8
Add CRAN mirror
cansavvy Sep 22, 2023
1e50a2d
Fix syntax
cansavvy Sep 22, 2023
e08c9e1
Delete spelling errors
cansavvy Sep 22, 2023
fd5e7f3
Merge remote-tracking branch 'origin/main' into 501-have-urls-periodi…
cansavvy Oct 24, 2023
a3e2dd9
Add a missing githubr::
cansavvy Oct 24, 2023
3df3e40
Install from github
cansavvy Oct 24, 2023
c640fe3
Fix GH_PAT
cansavvy Oct 24, 2023
ebc0d47
Install from CRAN again
cansavvy Oct 24, 2023
1580cc8
Install from github but use devtools
cansavvy Oct 24, 2023
cde9b61
dependencies = TRUE
cansavvy Oct 24, 2023
0d97f4a
Install from CRAN instead
cansavvy Oct 24, 2023
7784806
Test variables
cansavvy Oct 25, 2023
a04720a
Retrigger a run for GHA
cansavvy Nov 2, 2023
891d5be
Print out session info
cansavvy Nov 2, 2023
918083d
Actually what I did won't work!
cansavvy Nov 2, 2023
3b2dc08
use cat() instead
cansavvy Nov 2, 2023
3dde518
unlist
cansavvy Nov 2, 2023
cc733ba
I need cat() too
cansavvy Nov 2, 2023
7d6e766
Why is GHA not printing this out? Try message()
cansavvy Nov 2, 2023
4c48938
Save session info to a file and archive upload it
cansavvy Nov 2, 2023
8403a7e
Shut off the other things so this step completes
cansavvy Nov 2, 2023
8a51b45
Comment out one more step
cansavvy Nov 2, 2023
b8775cb
Merge remote-tracking branch 'origin/main' into 501-have-urls-periodi…
cansavvy Nov 21, 2023
cff4724
Merge branch 'main' into 501-have-urls-periodically-checked
cansavvy Feb 29, 2024
d76574b
Order dictionary.txt
cansavvy Feb 29, 2024
7baf79b
Add broken URL
cansavvy Feb 29, 2024
45eb1a1
Make it so blank is the logic
cansavvy Feb 29, 2024
f8d555e
Just retrigger the check
cansavvy Feb 29, 2024
a2ffbce
Try again
cansavvy Feb 29, 2024
18ee7b4
fix true
cansavvy Feb 29, 2024
dc6c371
drop quotes
cansavvy Feb 29, 2024
b85042a
Let's try this
cansavvy Feb 29, 2024
e364016
repo_name -> repo
cansavvy Feb 29, 2024
3093c4c
change per_page
cansavvy Feb 29, 2024
526c917
mysterious error
cansavvy Feb 29, 2024
2405c61
Print out clues
cansavvy Feb 29, 2024
5107f2d
Update script
cansavvy Feb 29, 2024
92a64fa
Fix logic
cansavvy Feb 29, 2024
3bf5ac4
fix so runs once a month
cansavvy Feb 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/url-error.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: Broken URLs found in the course!
labels: url-error
---
URL's in this course were just checked and some broken URLs were found.

**Number of errors:** {{ env.ERROR_NUM }}
**File where errors are:** [url checks file here]({{ env.FILE_URL }})
107 changes: 107 additions & 0 deletions .github/workflows/check-url.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
name: Periodic URL Check

on:
workflow_dispatch:
schedule:
- cron: '0 0 1 * *'

jobs:
set-up:
name: Load user automation choices
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

# Use the yaml-env-action action.
- name: Load environment from YAML
uses: doughepi/[email protected]
with:
files: config_automation.yml # Pass a space-separated list of configuration files. Rightmost files take precedence.
outputs:
toggle_url_check_periodically: "${{ env.URL_CHECK_PERIODICALLY }}"

url-check:
name: Check URLs
needs: set-up
if: ${{needs.set-up.outputs.toggle_url_check_periodically == 'yes'}}
runs-on: ubuntu-latest
container:
image: jhudsl/base_ottr:main

steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

# Delete the branch if this has been run before
- name: Delete branch locally and remotely
run: git push origin --delete preview-spell-error || echo "No branch to delete"

# Make the branch fresh
- name: Make the branch fresh
run: |
git config --global --add safe.directory $GITHUB_WORKSPACE
git config --global user.email "[email protected]"
git config --global user.name "jhudsl-robot"

echo branch doesnt exist
git checkout -b preview-spell-error || echo branch exists
git push --set-upstream origin preview-spell-error || echo echo branch exists remotely
shell: bash

- name: Run the check
uses: jhudsl/ottr-reports@main
id: check_results
continue-on-error: true
with:
check_type: urls
error_min: 1

- name: Declare file path and time
id: check-report
run: |
error_num=$(cat check_reports/url_checks.tsv | wc -l)
error_num="$((error_num-1))"
echo "error_num=$error_num" >> $GITHUB_OUTPUT
echo "error_url=https://github.com/${GITHUB_REPOSITORY}/blob/preview-spell-error/check_reports/url_checks.tsv" >> $GITHUB_OUTPUT
shell: bash

- name: Stop if failure
if: steps.check_results.outcome == 'failure'
run: exit 1

- name: Print out error variables
run: |
echo ${{ steps.check-report.outputs.error_url }}
echo ${{ steps.check-report.outputs.error_num }}

- name: Find issues
id: find-issue
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: |
echo "$GH_PAT"
echo "$GITHUB_REPOSITORY"
issue_exists=$(Rscript --vanilla scripts/find_issue.R --repo $GITHUB_REPOSITORY --git_pat $GH_PAT)
echo URL issue exists: $issue_exists
echo "issue_existence=$issue_exists" >> $GITHUB_OUTPUT

- name: If too many URL errors, then make an issue
if: ${{ steps.check-report.outputs.error_num >= 1 && steps.find-issue.outputs.issue_existence == 0}}
uses: JasonEtco/create-an-issue@v2
with:
filename: .github/ISSUE_TEMPLATE/url-error.md
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FILE_URL: ${{ steps.check-report.outputs.error_url }}
ERROR_NUM: ${{ steps.check-report.outputs.error_num }}

- name: If no URL errors than delete the branch we made
if: ${{ steps.check-report.outputs.error_num < 1 }}
run: |
git config --system --add safe.directory "$GITHUB_WORKSPACE"
git push origin --delete preview-spell-error || echo "No branch to delete"
2 changes: 1 addition & 1 deletion .github/workflows/pull_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
branch_name='preview-${{ github.event.pull_request.number }}'
echo branch doesnt exist
git checkout -b $branch_name || echo branch exists
git push --set-upstream origin $branch_name
git push --set-upstream origin $branch_name || echo echo branch exists remotely
shell: bash


Expand Down
1 change: 1 addition & 0 deletions 01-intro.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ ottrpal::set_knitr_image_path()
```

# Introduction
https://ahuidshiushiufhs.com


## Motivation
Expand Down
17 changes: 17 additions & 0 deletions OTTR_template.Rproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Version: 1.0

RestoreWorkspace: Default
SaveWorkspace: Default
AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: knitr
LaTeX: pdfLaTeX

AutoAppendNewline: Yes

BuildType: Website
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,3 @@ _This template and guide helps you_:
- Check out our [course on using Leanpub](https://hutchdatascience.org/Using_Leanpub/index.html) for help publishing on this platform.
- Please take a look at the [code of conduct](./code_of_conduct.md).
- If you encounter any problems or have ideas for improvements to this template repository or this getting started guide, please [file an issue here](https://github.com/jhudsl/OTTR_Template/issues/new/choose)! Your feedback is very much appreciated.

2 changes: 2 additions & 0 deletions config_automation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ spell-check: yes
style-code: yes
# Test build the docker image if any docker-relevant files have been changed
docker-test: no
# Should URLs be tested periodically?
url-check-periodically: yes

##### Renderings run upon merge to main branch #####
# Rendering each platform's content
Expand Down
42 changes: 38 additions & 4 deletions resources/dictionary.txt
Original file line number Diff line number Diff line change
@@ -1,33 +1,67 @@
AnVIL
Audiographer
Audiography
automagic
BIPOC
Bloomberg
Bookdown
chapt
Coursera
creativecommons
css
Datatrail
DataTrail
DISCOVERABLE
Dockerfile
Dockerhub
dropdown
enforceability
faq
favicon
FirstName
Funder
fyi
GDSCN
GitHub
Github
Generis
GH
Github
GitHub
https
ies
iframe
immunities
impactful
ITCR
itcrtraining
ITN
fyi
LastName
Leanpub
Licensor
LICENSOR
Markua
mentorship
MERCHANTABILITY
Muschelli
NCI
NHGRI
OTTR
ottrpal
ottrproject
Pandoc
png
pre
reproducibility
rmarkdown
RMarkdown
Savonen
sexualized
socio
synched
tada
UE
UE5
reproducibility
underserved
Videographer
Videography
WIPO
www
55 changes: 55 additions & 0 deletions scripts/find_issue.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/usr/bin/env Rscript

# Written by Candace Savonen Sept 2021

if (!("optparse" %in% installed.packages())){
install.packages("optparse")
}

library(optparse)

option_list <- list(
optparse::make_option(
c("--repo"),
type = "character",
default = NULL,
help = "GitHub repository name, e.g. jhudsl/OTTR_Template",
),
optparse::make_option(
c("--git_pat"),
type = "character",
default = NULL,
help = "GitHub personal access token",
)
)

# Read the arguments passed
opt_parser <- optparse::OptionParser(option_list = option_list)
opt <- optparse::parse_args(opt_parser)

repo <- opt$repo
git_pat <- opt$git_pat

if (!is.character(repo)) {
repo <- as.character(repo)
}

# Github api get
result <- httr::GET(
paste0("https://api.github.com/repos/", repo, "/issues"),
httr::add_headers(Authorization = paste0("Bearer ", git_pat)),
httr::accept_json()
)

if (httr::status_code(result) != 200) {
httr::stop_for_status(result)
}

# Process and return results
result_content <- httr::content(result, "text")
result_list <- jsonlite::fromJSON(result_content)

issue_exists <- length(grep('Broken URLs found in the course!', result_list$title))

# Print out the result
write(issue_exists, stdout())
Loading