Automated time-lapse? (SOLVED & WORKING!!)

[archaic0]

I'm still tweaking my process, but I'm getting closer to a final setup. I can' t count the number of times over the years that I've found dozens or hundreds of forum posts about precisely what I'm working on, but those posts end with a cliff-hanger and no resolution. I also often find people who post "fixed!" but then do not share how! Drives me bonkers. I'm glad others appreciate finding full and complete journeys like this one. I hope some day in the distant future some poor soul is able to find this post and that it will help them solve their issue as well. Or maybe someone else will decide that they too would like to create a time-lapse workflow from Blue Iris footage and with this guide they'll have what they need for a complete walk-thru.

I have 1 dedicated sky camera currently, but will have 2 going when I'm done. These cameras do like I said above where they write their video to one folder, then the retention settings move it to another folder and convert to an MP4 time-lapse when they do that.

I'm not happy with the speed of that conversion, but after talking with support it seems there is nothing I can do about that. It appears that FFMPEG is not able to process the BVR files directly today, so I have to rely on Blue Iris to convert them, but Blue Iris does not use much of the CPU resources available during exports. I upgraded my camera server and now have a ton of CPU and RAM available, yet I've only seen my CPU hit maybe 20% load while recording all 9 channels and exporting at the same time. One side note is that I was previously having Blue Iris resize the video from 5MP down to 2MP (1080p) because the next step in my process was limited to 1080. I recently changed that to leave the source 5MP video alone and expected that to slow down the conversion but it did not appear to change at all. Just makes me believe there is a LOT of room for performance improvements with the export process.

In the end, all that means for me is that I'll have to schedule my daily batch process to run later than midnight. I'll probably have to wait until 5 or 6AM each day to process the previous day's time-lapse. FFMPEG should be able to combine the segments for me no problem and then my other steps will take over to get that final video published to YouTube (that part is more of a pain than it should be as well!).

While I'm testing things though, because the export is so slow, I have to turn off the scheduled exports if I want to do any manual exports or else I won't be able to work on things for days. It took me 2 weeks to get all of the footage I wanted from July 4 exported for example, and that entire time I could not have my weather cam time-lapse going because the export queue was already so clogged.

It would be REALLY REALLY nice if I could find an external BVR conversion to MP4 process that was multi-threaded, or at least allowed multiple instances to be running, so I could export with all the performance I have available on the hardware I have.

[archaic0]

For stitching together daily clips, FFMPEG is indeed a powerful tool, but handling such scripts can get technical.

I'm an IT guy, so technical is my bread and butter. For this project, the stitching has to be automated and scripted. I'm not going to spend any time in a video editor.

I'm pretty much done with this part of the project. You can see one of my favorite recent videos here:

https://youtu.be/3W3Nf8yJT68?si=-JQCqn8WLhItvYag

BlueIris is exporting the clips automatically at this point and I'm picking them up the next day and combining them with FFMPEG. Then I'm using SyncBack Pro to move them to my Google Drive where Zapier picks them up and publishes them to YouTube. I'm still tweaking a couple pieces of the scripts, but it's working. Some day if I can find a tool that can convert the BVR files to MP4 outside of BlueIris, I'd like to change that. Not just for this project, but for anytime I want to export an entire day of video from my other cameras. Exporting more than a few clips is PAINFUL currently. REALLY hope I can find a better way to do that someday.

As soon as I can make the time to run a new network line up to my roof, I'll add a second view of the sky.

Thanks again to those with suggestions, and I hope this thread helps someone out in the future!

[AussieInSeattle]

I can' t count the number of times over the years that I've found dozens or hundreds of forum posts about precisely what I'm working on, but those posts end with a cliff-hanger and no resolution

Thanks for putting the effort into detailing your process and updating as you progressed! I am going to setup a new camera today for an ~6 month deck build that I want to time-lapse and will use your process. Has anything changed since your last post? I'm also in tech (full-stack software dev) so everything seems to make sense. I already have a pretty beefy NVR and a Synology that I could use to offload any tasks to. Also wondering if some sort of cloud service (I work with Azure) might be a (more expensive but easier) option.

[archaic0]

Nothing new at the moment. This project got stalled a bit with other priorities.

BlueIris is still working as before, still with clips converting very slowly. My process isn’t fully automated yet as I haven’t had time to finish up the scheduled tasks, but manually running ffmpeg is working ok.

One thing I’m not happy with though, but don’t have a solution for, is that my final Timelapse videos have little frame drops with every splice. I’m sure it’s due to the individual mp4 files having different cadences of key frames. The only way to fix that would be to increase the clip length to put an entire day in one clip, but I don’t think that would be good as the clip would be too massive to work with. Also don’t know if I can make clips different sizes per camera, but I may experiment with that in the future.

[AussieInSeattle]

Nothing new at the moment. This project got stalled a bit with other priorities...[snip]

Thanks for the response - I have your method all working and have experienced the same long conversion times (1hr:24m for 1 hour of footage). I did find this hint that I thought was the golden ticket to getting external ffmpeg working but no luck (setting framerate to create mp4 first, then use ffmpeg to do timelapse - see comment on sept 27):
https://github.com/mifi/lossless-cut/issues/2160


Am continuing to experiment this week as the project I want to timelapse doesn't start until mid next week.

[AussieInSeattle]

Nothing new at the moment. This project got stalled a bit with other priorities...

I have my time lapses up and running - in short this is what I did after running into the same issues as you (it was taking about 1.5 hours to timelapse out a 1 hour bvr using Blue Iris):

• created a new camera in Blue Iris (copied from existing) as I still want "normal control" over the camera

• on the new camera I named it the same as the existing but with "timelapse" in the name - turn off all motion, etc if you had that setup on the one you cloned from - this should be a "record only" camera

• on new camera on the record tab, click "File Format and Compression" button and change from bvr to mp4 and only record the main stream

• i setup my new camera to record 2 hour blocks - the above results in mp4 files getting saved instead of bvr files

• use the ffmpeg command below to create a timelapse that is 7.2 seconds in length for a 2 hour long mp4 file that blue iris is creating - note that my command line params are for an nvidia card - I got most of these params from various ffmpeg posts and don't understand them entirely - to change the length of the timelapse I just changed the 0.001 value (7200 seconds in 2 hours * .001 = 7.2 seconds output) and 250 value (pick every 250th frame? without this I got pixelation)

• for a 2 hour mp4 it is taking about 8 minutes to create the timelapse

• TODO: write some batch scripts that use the ffmpeg command to process multiple 2 hour files

• my timelapse is for a long running construction process that will have some "wait time" for materials throughout - with the above approach I can just delete the 2 hour mp4 files that I dont want and then the batch file (to be created) just reads from the folder to create the resulting timelapse - this approach also allows me to do adjust timelapse speed for each file so I could have a "more detailed/longer" timelapse for interesting parts of the construction process (cranes, slab pour) and faster timelapse for stuff that takes longer (framing)

Code: Select all

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i DCT.20250105_120000.mp4 -vf "select='not(mod(n,250))',setpts=0.001\*PTS" -an -sn -c:v h264_nvenc -preset slow output-tl3.mp4

Thanks for the inspiration!

[PaulDaisy]

Thank you for the detailed write-up. It is super helpful.

I too found that BI is not using the available resources well. My CPU (viewed in Task Man as admin, not in BI) never goes above 50%, while BI is struggling to re-encode. I wonder if that is an I/O issue, but the disk throughput in BI never reaches even 10% of its own benchmark, and I would not really expect it to be, as I tried it on a SSD and also on a M2 that is definitely fast enough. I have no idea where the bottleneck is.