Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish peripheral data as well, even if we don't use it ourselves #99

Merged
merged 4 commits into from
Jul 1, 2024
Merged

Publish peripheral data as well, even if we don't use it ourselves #99

merged 4 commits into from
Jul 1, 2024

Conversation

muffato
Copy link
Member

@muffato muffato commented Jun 4, 2024

TOLIT-2200

This PR is to publish more output files from the pipeline. Those things were not originally included because we don't need them ourselves (we'll get the files via other pipelines) but someone running the pipeline may find those interesting.

At the same time, I found that the full-test didn't pass because of some recent changes I did.

Keeping it as a draft for now until the base branch is merged. I also need to document the changes in docs/output.md

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@muffato muffato added bug Something isn't working enhancement Improvement of the existing features labels Jun 4, 2024
@muffato muffato requested a review from gq1 June 4, 2024 13:59
@muffato muffato self-assigned this Jun 4, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit aef0532

+| ✅ 133 tests passed       |+
#| ❔  24 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

❔ Tests ignored:

  • files_exist - File is ignored: CODE_OF_CONDUCT.md
  • files_exist - File is ignored: assets/nf-core-blobtoolkit_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_dark.png
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • files_exist - File is ignored: conf/igenomes.config
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
  • files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/nf-core-blobtoolkit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_dark.png
  • files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/blobtoolkit/blobtoolkit/.github/workflows/awstest.yml
  • template_strings - template_strings
  • merge_markers - merge_markers

✅ Tests passed:

Run details

  • nf-core/tools version 2.11
  • Run at 2024-07-01 16:13:42

@muffato muffato force-pushed the more_published_files branch 3 times, most recently from 24e2ca2 to 9449a46 Compare June 4, 2024 22:20
@muffato muffato marked this pull request as ready for review June 5, 2024 09:20
@muffato
Copy link
Member Author

muffato commented Jun 5, 2024

It's ready for review

gq1
gq1 reviewed Jun 5, 2024
View reviewed changes
docs/output.md Outdated Show resolved Hide resolved
@gq1
Copy link
Member

gq1 commented Jun 5, 2024

I can see master still being used in this ci?
https://github.com/sanger-tol/blobtoolkit/blob/dev/.github/workflows/branch.yml

if you haven't changed anywhere else, I can do a quick PR.

@muffato
Copy link
Member Author

muffato commented Jun 5, 2024

I can see master still being used in this ci? https://github.com/sanger-tol/blobtoolkit/blob/dev/.github/workflows/branch.yml

if you haven't changed anywhere else, I can do a quick PR.

Yes, please

gq1
gq1 reviewed Jun 5, 2024
View reviewed changes
docs/output.md Show resolved Hide resolved
@gq1
Copy link
Member

gq1 commented Jun 5, 2024

I have successfully run both test and test_raw profile on the farm as a new users.

Except the error with the two newly pulled singularity images:
docker.io-pacificbiosciences-pyyaml-5.3.1.img
docker.io-genomehubs-blobtk-0.5.1.img

And the error message:

Command error:
  FATAL:   container creation failed: mount /proc/self/fd/12->/software/singularity/3.11.4/var/singularity/mnt/session/rootfs error: while mounting image /proc/self/fd/12: kernel reported a bad superblock for squashfs image partition, possible causes are that your kernel doesn't support the compression algorithm or the image is corrupted

I have tried different version of singualrity on the new farm, all have the same problem. This is probably caused by a combination of farm22 and docker image?

muffato
muffato commented Jun 5, 2024
View reviewed changes
docs/output.md Outdated Show resolved Hide resolved
@muffato muffato force-pushed the more_published_files branch from 7363ac0 to 4c57872 Compare July 1, 2024 09:05
@muffato muffato changed the base branch from draft_assemblies to dev July 1, 2024 09:05
@muffato
Copy link
Member Author

muffato commented Jul 1, 2024

Since #97 is taking a while, I've rebased this branch onto dev

@muffato muffato force-pushed the more_published_files branch from 4c57872 to f27e1af Compare July 1, 2024 15:46
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 1, 2024

Python linting (black) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

  • Install black: pip install black
  • Fix formatting errors in your pipeline: black .

Once you push these changes the test should pass, and you can hide this comment 👍

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!

@muffato muffato force-pushed the more_published_files branch from f27e1af to aef0532 Compare July 1, 2024 16:12
@muffato muffato merged commit 7547ead into dev Jul 1, 2024
6 checks passed
@muffato muffato deleted the more_published_files branch July 1, 2024 16:25
@muffato muffato mentioned this pull request Jul 18, 2024
Merged
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gq1 gq1 gq1 approved these changes

Assignees

@muffato muffato

Labels
bug Something isn't working enhancement Improvement of the existing features
Projects
Genome After Party
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@muffato @gq1