OSError: Too Many Files, when downloading a bigger playlist #1978
Comments
mdawood94
added
the
Bug
|
gretting, since you're using mac, please tell me the value of "open file" using ulimit -a |
|
default number of open file descriptors on mac is 256. This behavior does not goes away when using the |
|
this tutorial solved temporarily the issue: https://gist.github.com/skylock/0117ec637d468f91260927b43b816eda but closing the file after saving it should solve this issue (file stay open in RAM even after song finished) |
|
looking through the source quickly it looks like the project follows the pythonic |
|
Confirming this issue on Linux (using Ubuntu 23). This starts to happen after downloading a couple hundred files. Max open files on my OS is not altered, so should be whatever the default is. Downloaded "Le Roy - See the Light Ahead": https://music.youtube.com/watch?v=Kd5reUZaGEI OSError: [Errno 24] Too many open files AudioProviderError: YT-DLP download error - https://music.youtube.com/watch?v=aEV83SlKyU0 OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' OSError: [Errno 24] Too many open files: 'cookies.txt' |
|
I'm having the same issue when downloading a lot of files from WSL using Ubuntu. For reference, I'm using the command: When I'm running |
|
try running I will investigate this more thoroughly whenever I get some time |
|
Still have this issue on Ubuntu 23.10. |
|
I have the same issue on Ubuntu 22.04 |
|
I was able to fix it by using the docker container, since I believe the Docker VM is setup with a much higher "open file" limit. I tested it on a 1300 song playlist and it was able to download ~400 songs before erroring out. The code definitely still needs a fix to close files after it is done with with them, but as a temporary workaround you can use the docker container and run it a few times with the "sync" command. |
|
I am experiencing this problem on Ubuntu.
As someone else suggested, I'm thinking the file handles are not being closed. If the code is recursive or the file handle isn't explicitly closed then it could cause this error. I have yet to look at the source code. I wrote a little bash script to kill and restart every 30 mins with a crontab. |
|
I'm getting this error, too. I believe this may have something to do with spotipy. Something is preventing requests that the program opens to get data about the song from closing. This is an example of one of the main error messages I'd get. |
|
Still having this issue, file limit is 1024 but it errors at around 200 songs. The error is slightly different, instead saying: (The too many open files error also occurs) I downloaded it from the Arch AUR. Does this have anything to do with the problem? |
|
Getting same error with large lists, is there a way to limit open files? tcp4 0 0 10.0.1.4.64872 74.125.157.8.443 CLOSE_WAIT Over 230 of these in CLOSE_WAIT |
|
I do think this is about open sockets |
|
See #2201 for temporary solution |
System OS
MacOS
Python Version
3.11 (CPython)
Install Source
pip / PyPi
Install version / commit hash
v4.2.4
Expected Behavior vs Actual Behavior
When downloading a larger playlist after about 50-70 tracks have been downloaded the terminal throws an error saying Too many open files, and all subsequent tracks do not get downloaded and it starts saying audio provider error.
Steps to reproduce - Ensure to include actual links!
Simple spotdl command on any large playlist should show this issues
Traceback
Other details
No response
The text was updated successfully, but these errors were encountered: