DeepStack AI Not Responding After Windows Update

[nh905g]

I am running Blue Iris 5.9.8.5 as a service and DeepStack 2021.02.1 (Windows package). My last change was upgrading to 5.9.8.5 on October 29. I was getting Blue Iris alerts until yesterday at 15:09 - there were some pending Windows updates (KB5048239, KB5046613, KB5046542) that caused my Blue Server to reboot overnight. Since then, DeepStack is not responding to Blue Iris even though the page at 127.0.0.1:82 comes up. The cameras are all showing 15FPS/1.0 except for one camera showing 15FPS/0.5 (I cannot change the key frame and has not been a problem so far). I ran chkdsk - no errors detected. I have rebooted the server multiple times - I have seen this problem intermittently in the last few months but a reboot always worked.

The Blue Iris log shows a mix of

Code: Select all

3 	2024-11-15 12:00:28.295 PM	Reo1                	Triggered: Motion_A
2 	2024-11-15 12:00:45.076 PM	Reo1                	AI: not responding
2 	2024-11-15 12:00:45.151 PM	Reo1                	AI: not responding
0 	2024-11-15 12:00:45.253 PM	Reo1                	AI: Alert canceled [nothing found] 0ms
3 	2024-11-15 12:04:57.887 PM	Walkout             	Triggered: Motion_A
3 	2024-11-15 12:05:07.375 PM	Garage              	Triggered: Motion_A
2 	2024-11-15 12:05:14.572 PM	Walkout             	AI: not responding
2 	2024-11-15 12:05:14.641 PM	Walkout             	AI: not responding
2 	2024-11-15 12:05:14.915 PM	Walkout             	AI: not responding
2 	2024-11-15 12:05:15.777 PM	Walkout             	AI: not responding
0 	2024-11-15 12:05:15.777 PM	App                 	AI: is being restarted
1 	2024-11-15 12:05:15.788 PM	Walkout             	AI: Alert canceled [AI: not responding] 16044ms
2 	2024-11-15 12:05:15.988 PM	Garage              	AI: not responding
2 	2024-11-15 12:05:16.017 PM	Garage              	AI: not responding
2 	2024-11-15 12:05:16.055 PM	Garage              	AI: not responding
2 	2024-11-15 12:05:16.081 PM	Garage              	AI: not responding
1 	2024-11-15 12:05:16.081 PM	Garage              	AI: Alert canceled [AI: not responding] 8025ms
3 	2024-11-15 12:05:16.788 PM	Walkout             	Re-triggered: Motion_A
2 	2024-11-15 12:05:19.495 PM	Walkout             	AI: not responding
2 	2024-11-15 12:05:19.563 PM	Walkout             	AI: not responding
2 	2024-11-15 12:05:19.836 PM	Walkout             	AI: not responding
0 	2024-11-15 12:05:20.934 PM	App                 	AI: has been restarted

The DeepStack error log shows the following but searching on the errors has not been productive.

Code: Select all

c:\Users\xxxx\AppData\Local\DeepStack\logs>type stderr.txt
Process Process-1:
Traceback (most recent call last):
  File "D:\obj\windows-release\37amd64_Release\msi_python\zip_amd64\process.py", line 297, in _bootstrap
  File "D:\obj\windows-release\37amd64_Release\msi_python\zip_amd64\process.py", line 99, in run
  File "C:\DeepStack\intelligencelayer\shared\detection.py", line 71, in objectdetection
    queue = db.lrange(IMAGE_QUEUE, 0, 0)
  File "C://DeepStack\windows_packages\redis\client.py", line 1975, in lrange
    return self.execute_command('LRANGE', name, start, end)
  File "C://DeepStack\windows_packages\redis\client.py", line 901, in execute_command
    return self.parse_response(conn, command_name, **options)
  File "C://DeepStack\windows_packages\redis\client.py", line 915, in parse_response
    response = connection.read_response()
  File "C://DeepStack\windows_packages\redis\connection.py", line 739, in read_response
    response = self._parser.read_response()
  File "C://DeepStack\windows_packages\redis\connection.py", line 340, in read_response
    raise error
redis.exceptions.BusyLoadingError: Redis is loading the dataset in memory

Does anyone have any ideas? I know I should switch to CodeProject but really do not have the cycles right now.

[BruceH]

Not sure if it will help but you could try running the following in an administer command prompt "SFC /SCANNOW" and see if it finds any errors with the windows files.

[TimG]

Ifirc there was a DS issue where you had to periodically delete the contents of a folder to keep it working. That info may still reside in the help files at the top of the forum page.

[nh905g]

Thanks for the fast responses!

sfc /scannow did report warnings about various folders that were "owned twice or has its security set twice". It repaired the issues and sfc /scannow reports no issues, but DeepStack is still not working even after a reboot.

I found over 3000 files in C:\Users\xxx\AppData\Local\Temp\DeepStack all from the 14th and 15th. I deleted all the files and rebooted. There are now about 60 files (all from today), but I am still getting DeepStack timeouts and redis errors.

It turns out Blue Iris is not generated a lot of false positives even with Deepstack disabled which buys me some time. I will try upgrading to DeepStack 2022.01.01 and if that does not work, I will try CodeProject.ai

[nh905g]

I upgraded from DeepStack 2021.02.1 to 2022.01.1 and DeepStack is working again. There were a lot of changes in 2022.01.1, however none appear to affect the base redis code. I did a search on "redis.exceptions.BusyLoadingError: Redis is loading the dataset in memory" in general, rather than specific to DeepStack/Blue Iris, and found https://www.dragonflydb.io/error-soluti ... -in-memory which claims the error is due to the redis data being accesses/modified while the database is still being loaded. DeepStack\redis\dump.rdb was almost 570MB on the failing instance, but only 322KB on the new 2022.01.1 instance - It has grown to 550KB by this morning.

Pure speculation, but could the size of the compressed dump.rdb have contributed to this issue? If it was a borderline condition, that might explain why some reboots seemed to trigger the problem given all the other things going on when Windows boots.