Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update KrakenUniq: Add SE/PE to suffixes as cannot control by typical ext.prefix #7023

Closed
wants to merge 2 commits into from

Conversation

jfy133
Copy link
Member

@jfy133 jfy133 commented Nov 19, 2024

Due to the nature of the construction of the module (for efficiency loading the database then aligning all samples in one process in the same process), we found that as we cannot control the suffix in the normal way via ext.args, we often get file-name collisions or overwrites at a pipeline results level when the leading prefix is the same between a single-end and a paired-end library (/end longest sentence of the week).

This PR hardcodes .se and .pe to file output suffxes to ensure they do not overwrite each other on publishing by providing a unique name.

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@jfy133 jfy133 marked this pull request as ready for review November 19, 2024 10:38
@Midnighter
Copy link
Contributor

I was wondering if this is not an artificial problem? Wouldn't under normal circumstances the run accession always be different if there was a paired-end and a single-end sequencing run of the same sample?

@Midnighter
Copy link
Contributor

Additional thoughts:

  • You have not configured the language server with Harshil-alignment? 😱
  • I haven't seen any other modules or pipelines adding a distinction for the filenames based on PE or SE. I don't think this part is special to KrakenUniq and would be a problem for every module, no?

@jfy133
Copy link
Member Author

jfy133 commented Nov 21, 2024

I was wondering if this is not an artificial problem? Wouldn't under normal circumstances the run accession always be different if there was a paired-end and a single-end sequencing run of the same sample?

Yes it somewhat is based on how we do things in nf-core/taxprofiler, but I don't see a way around it, as this module is NOT technically following the nf-core standard where a pipeline developer can control a file name by adjusting ext.prefix.

We cannot use ext.prefix here as we are having to infer the prefix ourselves within the module because, well you know why.

Another way would be to prevent upstream that the problem of unique sampel IDs.
However, in the pipeline schema for taxprofiler, we can't conditionally check the uniqueness based on the presence of another column. By this I mean: we can't enforce uniqueness on sample ID if there is no run_accession column, and then enforce another type of uniqueness (the combination) if it is present.

Regardless, the tests in taxprofiler are failing so it needs to be fixed 😅

Additional thoughts:

* You have not configured the language server with Harshil-alignment? 😱

* I haven't seen any other modules or pipelines adding a distinction for the filenames based on PE or SE. I don't think this part is special to KrakenUniq and would be a problem for every module, no?
  • I forgot to turn back on the format-on-save since they fixed it...
  • See point above

@jfy133 jfy133 marked this pull request as draft November 21, 2024 08:34
@jfy133 jfy133 marked this pull request as ready for review November 21, 2024 08:55
@jfy133
Copy link
Member Author

jfy133 commented Nov 21, 2024

Had a huddle with @Midnighter - actually we can use the same logic as the typical nf-core ext.prefix - so we don't have to hardcode -the diffference is actually jkust that the the prefix comes from a n input channel rather than ext., so I can close this.

If it comes up again, we can have a follow up PR to updateth the meta.yaml to give explicit guidance on this

@jfy133 jfy133 closed this Nov 21, 2024
@jfy133 jfy133 deleted the krakekuniq-suffix-fix branch November 21, 2024 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants