Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Two new plugins, customizable subtitle plugin and all-in-one video/audio/subtitle. #354

Open
wants to merge 45 commits into
base: master
Choose a base branch
from

Conversation

tehniemer
Copy link

@tehniemer tehniemer commented Nov 26, 2022

The SUBS plugin will extract/copy/remove embedded text and image-based subtitles according to user defined preferences in one pass inside Tdarr. S_TEXT/WEBVTT subtitles will be removed as ffmpeg does not handle them properly.

The AIO script combines the SUBS script with video and audio conversion of files in an all-in-one solution. Added sanity checks that will flag errors for misconfigured options and invalid files.

@tws101
Copy link
Contributor

tws101 commented Nov 28, 2022

I am testing this now....

So far excellent,

I am going to use it to replace my modified DRpepper extractor...

* Add sanity check to SUBS and AIO (#11)

* Delete Tdarr_Plugin_TN10_SUBS.js

* Create Tdarr_Plugin_TN10_AIO.js

* Update Tdarr_Plugin_TN10_AIO.js

* Update Tdarr_Plugin_TN10_AIO.js

* Update Tdarr_Plugin_TN10_AIO.js

* Update Tdarr_Plugin_TN10_AIO.js

* Create Tdarr_Plugin_TN10_SUBS.js

* Update Tdarr_Plugin_TN10_AIO.js

Update defaultValues for number types

* Update Tdarr_Plugin_TN10_AIO.js

Updates per checks

* Update Tdarr_Plugin_TN10_AIO.js

removed erroneous character

* Update Tdarr_Plugin_TN10_AIO.js

check fixes

* Update Tdarr_Plugin_TN10_AIO.js

* Update Tdarr_Plugin_TN10_SUBS.js

* Update Tdarr_Plugin_TN10_AIO.js

* Add sanity check

* Add sanity check

* Update Tdarr_Plugin_TN10_AIO.js

Fixes from checks

* Update Tdarr_Plugin_TN10_SUBS.js

Fixed from checks

* Update Tdarr_Plugin_TN10_SUBS.js

Fixes from checks

* Update Tdarr_Plugin_TN10_AIO.js

Added infolog entry for removing all subtitles.

* Update Tdarr_Plugin_TN10_AIO.js

Update id:

* Update Tdarr_Plugin_TN10_AIO.js

Fixed ffmpeg command order for subtitle extraction.

* Add error flags to sanity checks

* Update Tdarr_Plugin_TN10_AIO.js

Convert tabs to spaces
@tehniemer tehniemer changed the title New subtitle plugin with extensive options. Two new plugins, customizable subtitle plugin and all-in-one video/audio/subtitle. Nov 30, 2022
@tehniemer
Copy link
Author

I am testing this now....

So far excellent,

I am going to use it to replace my modified DRpepper extractor...

Good to hear. I've added a few additional checks so the plugin will error instead of exiting with a transcode not required if it's misconfigured.

@tws101
Copy link
Contributor

tws101 commented Dec 6, 2022

I am still using the subtitle plugin... I dont have any issues to report with it... which one to you test out errors in?

@tehniemer
Copy link
Author

I noticed some issues in the AIO that could present themselves in the SUBS script under very specific scenarios. I decided to pull the plug temporarily until I feel more confident. I have not made any changes to the SUBS script as of this comment.

* Multiple changes (#16)

* Multiple changes
Add check for default audio stream.
Add catch for infinite loops.
Implement trigger to keep all audio streams.
Add check for mediaInfo and remux if missing.
@tehniemer
Copy link
Author

Issues seem to be resolved now and some more features added. Should be good to go now.

@tehniemer tehniemer reopened this Dec 16, 2022
Fixed two small issues.
replace isNaN with Number.isNaN
Reinstated isNaN instead of Number.isNaN
* Add code to update stats if missing from MKV

* Clean up code

Clean up how process checks for stats and mediainfo, add threshold for updating stats.
@tws101
Copy link
Contributor

tws101 commented Jun 16, 2023

SUB plugin I have been running constantly since December...

Have you considered adding also extract ASS subs ?

@tehniemer
Copy link
Author

I could look into it, but haven't run across files that have that subtype

@tws101
Copy link
Contributor

tws101 commented Jun 19, 2023

I could look into it, but haven't run across files that have that subtype

Anime,

In my effort to reduce transcoding server side to zero and have all clients direct play everything. Those ASS anime subs which are text based can be an issue. When I am lucky bazarr will just use the embeded subtitle extractor and extract and convert them for me I am not always so lucky.

@tehniemer
Copy link
Author

That does sound familiar, do they have fonts that go along with the subs? I think that might be an issue. I'll see if I can find a file with them and play around a bit.

@tehniemer
Copy link
Author

@HaveAGitGat any chance of getting this one merged? I've been using them exclusively without issue since the last commit. Or are classic plugins going to be abandoned with the introduction of flows?

@HaveAGitGat
Copy link
Owner

@tehniemer thanks ideally we should not be using // tdarrSkipTest and should add some basic tests so various inputs are checked. So would be case of creating files with the same name here:

https://github.com/HaveAGitGat/Tdarr_Plugins/tree/master/tests/Community

Then copy content of one of the other test files and modify input/output until npm run test works fine

pull bot and others added 2 commits December 18, 2023 12:06
* Use node 18

* Fixed issue with video files missing language tag on audio track so it
assumes the audio track is a desired audio track. Changed default
target framerate to 30 to prevent issues with videos that have a
framerate of 29.97.

* Add av1 option to checkVideoCodec flow plugin

* Adding tests for the plugin

* Working on updating the tests and cleaning up any formatting issues

* Update desc wording

* Check Audio Codec

* Remove for now as can use Tdarr_Plugin_00td_action_add_audio_stream_codec

* fix(HaveAGitGat#277): Update Tdarr_Plugin_sdd3_Remove_Commentary_Tracks.js

* Add Ensure Audio Stream

* Add better estimates for 2 pass hb encoding

* More accurate hb 2 pass logic

* Update ts

* Add other codecs

* Update index.ts

Set the default for the listbox to "==" as per my Discord post

* Update default to ==

* Trim input

* Add more examples

* Add logging if file size does not match

* Use lf

* Add opus support to EnsureAudioStream

* Use fileMoveOrCopy

* Add auto compile tsc

* Remove error script

* Apply auto-build changes

* Fix lint errors regarding unused vars

* Add Basic Video or Audio Settings

* Fix lint error

* Apply auto-build changes

* Add Basic Video or Audio Settings template

* Apply auto-build changes

* Disable tryMvdir

* Apply auto-build changes

* Create index.ts

* Apply auto-build changes

* Update basic flow template

* Format wording

* Apply auto-build changes

* Add reasoning for why no GPU encoding

* Apply auto-build changes

* Add -strict -2 for truehd

* Fix spacing

* Add Copy/Move Folder Content

* Apply auto-build changes

* Fix defaults

* Apply auto-build changes

* Add labels

* Add labels

* Apply auto-build changes

* HandBrake Custom Arguments

* Use original library file

* Apply auto-build changes

* Add bitrate option

* Add samplerate

* Add check bitrate

* Apply auto-build changes

* Clarify switch input type

* Apply auto-build changes

* Add deleteParentFolderIfEmpty option

* Apply auto-build changes

* Kill flow subthread on worker exit

* Apply auto-build changes

* Refactor thread kill

* Apply auto-build changes

* Clean up listener after cli run

* Apply auto-build changes

---------

Co-authored-by: HaveAGitGat <[email protected]>
Co-authored-by: Zach Gelnett <[email protected]>
Co-authored-by: Andrew Kennedy <[email protected]>
Co-authored-by: Alexander Gomez <[email protected]>
Co-authored-by: Kedryn <[email protected]>
Co-authored-by: HaveAGitGat <[email protected]>
@tehniemer
Copy link
Author

@HaveAGitGat understood, but being new to the tests I'm unsure of what they are actually testing for, and I don't see any documentation about setting them up. Do you have any pointers?

@HaveAGitGat
Copy link
Owner

@tehniemer just checking what outputs happen when you enter certain inputs. The first test is typically with the input left blank so that the default inputs are used and then add as many other tests as needed to check the other inputs are working correctly.

E.g. simple remux plugin:
https://github.com/HaveAGitGat/Tdarr_Plugins/blob/master/tests/Community/Tdarr_Plugin_MC93_Migz1Remux.js

Inputs in first test:

inputs: {},

Inputs in second test:

      inputs: {
        container: 'mp4',
        force_conform: 'true',
      },

@tehniemer
Copy link
Author

@HaveAGitGat Thanks for the explanation, I think I'm catching on now. In short, I need to manually build the expected output for a given input set based on the data from the file(s) located in sampleData\media . Is that correct?

@HaveAGitGat
Copy link
Owner

HaveAGitGat commented Dec 22, 2023

@tehniemer yeah that's right, you don't need to use all the sample files. For example one is h264 and the other is h265, another has many audio streams with different languages and channel counts (good for checking audio related plugins) so just case of using the relevant ones. These plugins do a bit of everything so could use each one. In some cases a little hack like this can be used to modify the sample file data so you can test the plugin in a certain way (instead of adding a whole new sample file for one small thing):
https://github.com/HaveAGitGat/Tdarr_Plugins/blob/master/tests/Community/Tdarr_Plugin_MC93_MigzImageRemoval.js#L25

* Update qsv check

* Stack log errors

---------

Co-authored-by: HaveAGitGat <[email protected]>
HaveAGitGat and others added 3 commits May 8, 2024 08:04
* Use templating

* Add brackets

* Add compiled changes

* CPU and GPU Workers with Classic Plugins

* Update fork check

* Apply auto-build changes

* Set Flow Variable

* Ignore lint errors

* Fix Check Video Resolution typo

* Apply auto-build changes

* Adjust desc

* Apply auto-build changes

* Pause/Unpause All Nodes

* Apply auto-build changes

* Set original after adding typeIndex

* Apply auto-build changes

* Add Run CLI

* Apply auto-build changes

* Update logic

* Apply auto-build changes

* Update var to userCli

* Apply auto-build changes

* Add Apprise

* Format tooltip

* Apply auto-build changes

* Auto build

* Apply auto-build changes

* Add name

* Fix yml syntax error

* Format description

* Apply auto-build changes

* Add  'Auto accept successful transcodes' note

* Apply auto-build changes

* Add 2nd classic transcode plugin output for if no processing occurred

* Apply auto-build changes

* Remove default

* Update tooltip

* Apply auto-build changes

* Check mediaInfo properties

* Update description

* Apply auto-build changes

* Add additional compare logging

* Format text

* Apply auto-build changes

* Add useCustomCliPath and customCliPath inputs

* Use destructuring

* Apply auto-build changes

* Format classic plugin names

* Use Node scan function if available

* Apply auto-build changes

* Add types

* Add version comment

* Flow Run Health Check updates stats and queues

* Apply auto-build changes

* Fix error message with cliPath

* Apply auto-build changes

* Add files via upload

* Add files via upload

* Update worker with hb fps if available

* Apply auto-build changes

* Use fsp for promises

* Ensure es6 import

* Use async fileExists

* Promisify fs.unlinkSync

* Promisify copyFileSync

* Apply auto-build changes

* Fix rename-undo bug when using rename plugin

* Apply auto-build changes

* Use simpler map trim() for values

* Fix comment typo

* Fix genpts comment typo

* Add tags plugins

* Update inferfaces

* Apply auto-build changes

* Don't use scale_qsv=format=p010le on Windows

* Apply auto-build changes

* Tags subset fix

* Apply auto-build changes

* Update tag plugin descriptions

* Apply auto-build changes

* Trim and filter tags

* Apply auto-build changes

* Compare File Size Ratio Live

* Apply auto-build changes

* Update tooltip

* Apply auto-build changes

* Add Basic HEVC Video Flow - Migz 50%

* Apply auto-build changes

* Add error variable

* Apply auto-build changes

* Add original container option.

* Add toggles for preset and quality

* Apply auto-build changes

* Adjust input labels

* Apply auto-build changes

* Log CLI exit code

* Apply auto-build changes

* Add ffmpegCommand.init to ensure "Begin" command has been used

* Add checkFfmpegCommandInit(() to ffmpeg command plugins

* Apply auto-build changes

* Use checkFfmpegCommandInit() in Execute plugin

* Apply auto-build changes

* Re-order filepath change logging

* Use macos-14

* Use windows-2022

---------

Co-authored-by: HaveAGitGat <[email protected]>
Co-authored-by: HaveAGitGat <[email protected]>
Co-authored-by: tws101 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants