Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Other ffmprovisr-suitable tools? #310

Open
ablwr opened this issue Mar 8, 2018 · 54 comments
Open

Other ffmprovisr-suitable tools? #310

ablwr opened this issue Mar 8, 2018 · 54 comments
Labels
discussion

Comments

@ablwr
Copy link
Member

ablwr commented Mar 8, 2018 •
edited
Loading

Growing out of #309 and stemming also from this tweet --

What tools are a good fit for ffmprovisr-style sites, potentially to live immediately adjacent to ffmprovisr in this repo? What is complicated enough to need regular scripting reference?

  • Exiftool (Exiftoolkit)?
  • ImageMagick (ImageMagickTricks [thanks @kfrn for that one])?
  • Bash?
  • Pandoc?
  • sox?
  • exiv2 ref?
@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

They all at minimum need punny and hard-to-remember-or-search names just like their parent. ;)

@privatezero
Copy link
Member

I have been tempted to add a few sox commands into ffmprovisr for a couple of the things sox handles better than ffmpeg - but not sure if there are enough to warrant an adjacent site...

@privatezero
Copy link
Member

oh, hahaha I just noticed you had sox on the list already! Sox is good!

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

@EG-tech noted it AND you!

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 8, 2018 via email

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 8, 2018 •
edited
Loading

Also the openssl pipe here is invaluable for calculating the weird hashes in a digital cinema package

Edit: link https://en.wikipedia.org/wiki/Digital_Cinema_Package#Packing_list_file_or_PKL_package_key_list

@kieranjol
Copy link
Collaborator

Also https://twitter.com/dericed/status/758275619767582720?s=19 the xxd call in here as a means of analysing binary outputs and determining any potential padding..

@retokromer
Copy link
Member

The use of Rubberband and Tesseract (with and/or without FFmpeg) could also be of interest.

@EG-tech
Copy link
Member

EG-tech commented Mar 8, 2018

  • ImageMagickMagic
  • KnockYourSoxOff
  • ExifThroughTheGifShop

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 8, 2018 via email

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

Yeah that's what I'm thinking about... it would be good to have a misc though, for things related to the same premise but a different tool is better. Trying to think about which ones are complex enough -- ffmpeg really is a BEAST! ImageMagick for sure is weighty.

@retokromer
Copy link
Member

Bash is a (little) beast, I presume.

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

The biggest beast of all!!!!

@bturkus
Copy link
Contributor

bturkus commented Mar 8, 2018 via email

@todrobbins
Copy link
Contributor

maybe a youtube-dl guide for (community) archivists?

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

We are wired in, @bturkus.

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

Ya @todrobbins do you use youtube-dl?

I guess also it comes down to soliciting for commands people use. ffmprovisr got a jump-start from 1 my totally non-useful commands and 2 being during a hackathon and 3 leeching off of a previous AMIA hackathon's ffmpeg-for-archivists guide which had sample commands in it

@privatezero
Copy link
Member

I support all of this! Especially the ❤️ , @EG-tech's proposed names, and another +1 for youtube-dl! I use that hella!

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 8, 2018 via email

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

My secret dream is a hackathon that moves with time zones, like Sunday 10am-5pm but N.Z. gets a head start, and @privatezero is left cleaning up all of the junk commits that have been made, unless we find an a/v archivist with github skills in Hawaii/Alaska. ;)

@kfrn
Copy link
Member

kfrn commented Mar 8, 2018

but N.Z. gets a head start

I'm happy to write some bugs for y'all to sort out 😎

@ablwr
Copy link
Member Author

ablwr commented Mar 8, 2018

wow this just arrived in my inbox https://github.com/learnbyexample/Command-line-text-processing/blob/master/ruby_one_liners.md 😍

@privatezero
Copy link
Member

I'm ok with being a closer as long as y'all are prepared for some <!--Free Cascadia--> comments being slipped in after everyone else has wrapped ;-P

@retokromer
Copy link
Member

The Roman Empire is falling: I was convinced that <!--Free Cascadia--> is a secret instruction for the compiler!

@kfrn
Copy link
Member

kfrn commented Mar 8, 2018

Don't forget what we learned from Philip K Dick ....

THE EMPIRE NEVER ENDED

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 8, 2018 via email

@todrobbins
Copy link
Contributor

todrobbins commented Mar 15, 2018 •
edited
Loading

A potential starting point for a youtube-dl guide: https://github.com/bibanon/tubeup

YouTube → Internet Archive

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018 •
edited
Loading

I'd like to work on this.

Since we aren't sure how big some of these could potentially get (and I'm unsure if any of them could reach the size of ffmpeg, at least for this crowd), how about a compromise like this?

screen shot 2018-03-26 at 22 46 03

(Adding tools to the bottom of ffmprovisr, and in the ToC of course)

If they grow and the size becomes a problem (like @ross-spencer brought up in #309) we can decide to split at that time, and this way we don't block progress.

But what do y'all think?

Also I need to round up some actual useful examples...

@privatezero
Copy link
Member

I think that makes a lot of sense! I can start thinking of some sox things! (Maybe one of the most useful things sox does better than ffmpeg is splitting tracks on silence, and there is a good example right in the sox manual)

@todrobbins
Copy link
Contributor

ToC title could be Similar tools: tips & tricks 🎩🐰

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 27, 2018 via email

@richardpl
Copy link

@privatezero sox is no longer developed for good, what command you speak about?

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018

I did the thing I know best, which is tweet about it https://twitter.com/ablwr/status/978617812079009792

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018

I think wget might be able to justify having its own section too.

rsync isn't as complex, but maybe? I also know I can never ever remember the right way to do it.

@kieranjol
Copy link
Collaborator

kieranjol commented Mar 27, 2018 via email

@richardpl
Copy link

richardpl commented Mar 27, 2018 •
edited
Loading

try curl as alternative to wget

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018

Maybe wget, rsync, and curl are close enough to group together in one section about collection and transferring data.

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018

@retokromer Do you happen to have examples of where and when you use tesseract or rubberband? I'm familiar with tesseract, I think it's super useful and robust enough to have a section. Just need some examples to start! 🎉

@todrobbins
Copy link
Contributor

PS: I'm throwing OpenCV into the fray. Come at me!

@JonnyTech
Copy link

@todrobbins I am not sure about including OpenCV - all of the other tools can be run from a command line but afaik OpenCV is a software component that requires writing code and then compiling and running it, which imho is out of the scope for most non-programmers.

@privatezero
Copy link
Member

@richardpl yes, I think we are all aware that sox is not under active development, and when it either ceases to work or is superseded by other tools then we won't have to consider it. Until then, it remains an effective tool for limited use cases, the aforementioned ability to effectively split files on silence via a single simple command being one of them.

In my experience sox has also supported more reliable capture of audio devices than ffmpeg does. In testing the ffmpeg avfoundation device across a couple different computers it was the common variable in an unacceptable amount of interstitial errors.

@richardpl
Copy link

@privatezero And exact command is?

@retokromer
Copy link
Member

@ablwr Yes, I can make a PR after the SEAPAVAA and FIAF conferences. We do use tesseract very often, mainly to extract intertitles of silent films (BTW: it works quite well also with the Fraktur of German silents!) or more rarely for subtitle extraction, and rubberband for sound manipulations.

@privatezero
Copy link
Member

@richardpl For file splitting? A basic example for splitting a digitized reel at points of silence is sox input.flac output.ogg silence 1 0.50 0.1% 1 2.0 0.1% : newfile : restart

As regarding the avfoundation device, it has been too long to remember, and I am no longer using macOS, but it would have been some variant on ffmpeg -f avfoundation -i 'none:0' -f wav -c:a pcm_s24le -ar 96000..., possibly with some piping involved for the outputs.

@ablwr
Copy link
Member Author

ablwr commented Mar 27, 2018

DAMN LOOK AT THIS http://www.fmwconcepts.com/imagemagick/index.php

@richardpl
Copy link

ImageMagick have many CVEs, and many of that what is listed there should be doable with FFmpeg, and faster. FFmpeg operates on images too, so if you find something doable in ImageMagick and not doable in FFmpeg please report to FFmpeg bug report tracker.

@retokromer
Copy link
Member

Usually, I prefer curl as well. It’s just 20 year old, and still young!

@todrobbins
Copy link
Contributor

@JonnyTech you're right, OpenCV is out of scope. But an GLAM guide utilizing OpenCV would be great. Carry on!

@JonnyTech
Copy link

@todrobbins, forgive my ignorance but I am unfamiliar with the term "GLAM guide".

@kieranjol
Copy link
Collaborator

GLAM= galleries, libraries,archives,museums.

@todrobbins
Copy link
Contributor

What @kieranjol said. Sorry for assuming everybody knows that acronym!

@richardpl
Copy link

@privatezero similar could be implemented in ffmpeg: silencesplit filter, the only limitation is that one would need to know number of split points ahead of time, but one could use same filter in 2 passes.

regarding avfundation, is that issue reported to trac?
IIRC we have active avfundation maintainer. I can not fix it because i do now own such OS.

@ablwr
Copy link
Member Author

ablwr commented Mar 28, 2018

This thread is super big! I am going to make some individual threads soliciting scripts from some of the programs mentioned here. I'm happy to add anything and everything but don't have the power to dig up things that are practical and useful too, so I hope people can chime in and once there's a threshold reached (like... 4...? 5 recipes?), a new section can emerge.

This was referenced Mar 28, 2018
Closed
Open
Open
@ablwr
Copy link
Member Author

ablwr commented Mar 28, 2018

I'll start with those. But feel free to make your own if you have some to start with, or send a PR. Instead of Bash, maybe "Unix tools" is a good general header for tossing in some stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discussion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@todrobbins @richardpl @kieranjol @retokromer @ablwr @JonnyTech @bturkus @privatezero @kfrn @EG-tech