Timelapse suddenly not completing

[dchamb55]

I have been running Allsky for nearly a year without fail. I haven't changed any software versions during that time. Starting on April 23, 2023, my timelapse videos have been cut short. Sometimes, only a second is showing. I am running on an RPi 3 B+ 1GB with 64GB SD card. The system is showing no throttling and has 37GB of remaining space left on the SD card, and the memory used is 10%.

Checking the images stored for each night shows they are all present so I know they were created properly. I rebooted the RPi and tried to recreate the video file with no success. The FTP is not the problem either since the file on the RPi is the same as on my website where they are transferred to.

Any ideas of how to solve this?

Thanks

Dale

[EricClaeys]

@dchamb55 Dale, Unfortunately I don't have an answer. Someone else reported this about a year ago. You can try searching the Issues and Discussions, and asking on the Allsky Facebook group.
This problem is listed in the troubleshooting page, but as you see in red, I don't have an answer.

We plan to release the next version of Allsky on May 1; I doubt this will resolve the problem since it's not with Allsky - it's with ffmpeg. I should have some time after that to help you look into it.

[dchamb55]

Thanks Eric, I have tried some of the steps on that page. I've also posted the issue on the Allsky Facebook page. I am wondering if the micro SD card could be the problem. Although it has been running spectacularly well for close to a year now without having to reboot. Dale

…

On April 26, 2023 7:59:36 PM EricClaeys ***@***.***> wrote: @dchamb55 Dale, Unfortunately I don't have an answer. Someone else reported this about a year ago. You can try searching the Issues and Discussions, and asking on the Allsky Facebook group. This problem is listed in the troubleshooting page, but as you see in red, I don't have an answer. We plan to release the next version of Allsky on May 1; I doubt this will resolve the problem since it's not with Allsky - it's with ffmpeg. I should have some time after that to help you look into it. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[EricClaeys]

Dale, I supposed anything is possible, but I'd think if the card was bad you'd get lots of errors.
Try looking in some of the other log files in /var/log, like "messages" and "kern.log". You may want to truncate them, then run a timelapse:

sudo truncate -s 0 /var/log/messages /var/log/kern.log /var/log/allsky.log
cd ~/allsky/scripts
generateForDay.sh -t 20230425

when the timelapse finishes, look in those log files for any errors.

[bbillp]

Long ago I had a similar problem and found the file sometimes ended in JPEG, JPG, jpg or  jpeg (I don’t recall the exact details) but its easy to check the video generator script to see what file type it uses for input.  

[EricClaeys]

It uses the extension of the name of the image in the "Image Name" field in the WebUI. For most people that's "image.jpg" so the timelapse creation looks for files ending in ".jpg".

[dchamb55]

I believe I have found the root cause of the problem with why the ffmpeg is failing to complete the mp4 file. It seems that at times during the night, images are captured with a zero file size. I removed the zero-length jpeg files and reran the timelapse script and the full mp4 file was generated. Apparently, ffmpeg chokes when it encounters one of these files and stops. Now I don't know why there are zero-length jpeg files. Going back to 4/13, I found only two other nights where this happened, but the number of times it happens is increasing.

The log files did not shed any light on this. Except the kern.log shows multiple times the RPi is rebooting. Here are the occurrences I've seen:
April 23, at 22:20:35
April 24, at 07:17:06, 16:17:06, 18:17:05, 19:38:24
April 25 at 00:17:05, 04:17:48, 07:36:33, 09:42:08, 11:32:24, 12:21:34, 17:30:01, 18:20:02, 19:17:06, 20:17:06
April 26 at 04:50:24, 05:17:06, 12:17:06, 12:38:25, 13:19:43, 14:47:45, 15:21:25, 16:04:30, 17:12:01, 17:19:55, 18:03:36, 19:38:07, 20:00:57, 21:17:32, 23:17:06
April 27 at 02:17:06, 07:01:19, 07:17:06, 08:36:19, 08:38:45, 09:17:06, 09:52:51, 10:40:49, 12:18:02, 12:21:55, 13:17:06 (of course there may be other times for this day)

The kern.log repeatedly shows "usb 1-1.1.3: reset high-speed USB device number 4 using dwc_otg" before the reboot occurs.

Now the reboot times do not correlate with the creation times of the zero-length jpeg files, but I have to think there is a relationship. Questions are 1.) Can the script delete the zero-length jpeg files before ffmpeg runs? and 2.) What could cause the frequent rebooting: a USB cable or camera issue?

Dale

[dchamb55]

Hi Eric,

I set REMOVE_BAD_IMAGES to "true" and the timelapse mp4 still cut short. There were 8 zero length jpgs in the folder from last night and were not removed. I looked at the scripts but I don't see where any are invoking "removeBadImages.sh". Perhaps I need an updated version of the scripts?

If I run "removeBadImages.sh" on the date, the zero length jpgs are removed and I can successfully regenerate the timelapse. It just doesn't seem that the script is getting run after the night is finished.

Dale

[EricClaeys]

Dale, We're releasing the next version of Allsky on May 1st, so if you can wait a couple days I suggest you grab it. It'll be version v2023.05.01. I seem to recall fixing something with removeBadImages.sh.

Does the video stop just before the first zero-length file?

If you remove the 8 zero length images does the timelapse work?

FYI, I just added a "to do" item to have timelapse.sh check the number of images in the video against the number it expects, and if different, warn the user. I also updated the documentation page to mention zero-length images.

[dchamb55]

Eric, yes it looks like the video is cut off at the point of the first (chronological) zero length jpg. And I ran the "removeBadImages" script which removed all 8 zero length images. The video completed successfully with all 8 removed.
Do you know if the new release will include any new ZWO drivers?

Dale

[EricClaeys]

@dchamb55 Dale, that's great news. The new release (out tomorrow) will remove zero-length files if REMOVE_BAD_IMAGES is on, which it is by default.

It will also include the latest ZWO drivers - I just checked their site.

[dchamb55]

Excellent!

[EricClaeys]

Closing discussion. Problem should be solved in v2023.05.01.

[EricClaeys]

With version v2023.05.01 and newer, REMOVE_BAD_IMAGES should ALWAYS be enabled since it removes 0-length files that cause the timelapse creation to stop.