For your amusement: how the AI's see the world

[Ferguson]

I turn on alerts when I have the telescope set up outside unattended imaging. The camera is set to detect humans and vehicles as well as animals; the AI then confirms based on only humans and vehicles (this way I have clips of any animals but no alerts).

Last night I hear this "Someone is near the telescope" announcement, and here's why. It was a deer (in disguise) but I love what it picked out as a human.

This is actually a problem I wish BI could fix -- when using ONVIF alerts, it would sure be nice if that area (and only that area) could be passed to CPAI.

But more to the point... for your amusement:

deer_human.jpg (51.43 KiB) Viewed 4100 times

[TimG]

If you check a DAT file, does the resolution (sub-stream ?) look good enough to identify objects ? It could mean the difference between a Martian and a tripod

[Ferguson]

If you check a DAT file, does the resolution (sub-stream ?) look good enough to identify objects ? It could mean the difference between a Martian and a tripod

I actually am not using substreams. I got really annoyed by the UI and UI3 showing very low resolution images until you bring up one camera I turned it off.

Is there a benefit to using low resolution other than processing speed (which I'm not troubled by)?

But I'll probably need a different AI training module for "alien" much less distinguishing arbitrary aliens from martians.

[TimG]

In that case you are already sending the best image you can to CPAI. Still worth checking what comes back in the DAT file though, as they used to reduce the resolution if it was too high. If the DAT file shows the same size, then that is old news. One day the help files will catch up !

[Ferguson]

In that case you are already sending the best image you can to CPAI. Still worth checking what comes back in the DAT file though, as they used to reduce the resolution if it was too high. If the DAT file shows the same size, then that is old news. One day the help files will catch up !

I don't see in the dat file the resolution given, but it placed the "human" which is rather in the middle at an X position between 1291 and 1500. The camera is 4mpx so 2688 x 1520, that seems fairly consistent with being near the center. So I think it's passing the full resolution.

I related question of course is how it extracts the image it sends (I presume a still), and how much effort it goes to in that extraction to provide a high quality image vs one with lots of compression artifacts. But I see nothing to set that. File attached for anyone curious.

I've since turned off onvif for animals on the cameras aimed at the telescope to cut down on false positives; it was the animal's motion that triggered the CPAI analysis, not the telescope's. So if the camera doesn't trigger on animals, less likely I'll get woken up for false positives.

Hunters who want to be awakened for a deer in their yard are out of luck.

Code: Select all

[
	{
		"api":"ipcam-combined",
		"found":{
			"message":"Found deer,
			person",
			"count":2,
			"predictions":[
				{
					"confidence":0.5570281744003296,
					"label":"deer",
					"x_min":379,
					"y_min":1025,
					"x_max":558,
					"y_max":1302}
				,
				{
					"confidence":0.6160706281661987,
					"label":"person",
					"x_min":1291,
					"y_min":294,
					"x_max":1500,
					"y_max":644}
				]
			,
			"success":true,
			"processMs":190,
			"inferenceMs":188,
			"moduleId":"ObjectDetectionYOLOv5-6.2",
			"moduleName":"Object Detection (YOLOv5 6.2)",
			"code":200,
			"command":"custom",
			"requestId":"5c29071d-d378-4c71-9751-97455f98a491",
			"inferenceDevice":"GPU",
			"analysisRoundTripMs":241,
			"processedBy":"localhost",
			"timestampUTC":"Tue,
			27 Aug 2024 07:43:39 GMT"}
		}
	]

[TimG]

OK, so I set DAT files on my drive cam, and waited for a valid trigger. Once I saw one I held down Ctrl and double clicked the Alert video. It opened the DAT file, and in the top right hand side you can see this, so that must be the image size processed:

resolution.png (6.62 KiB) Viewed 4014 times

And the actual sub-stream resolution for that camera is 704 x 576:

substream.png (32.64 KiB) Viewed 4014 times

So slightly different on that cam. I have seen it the same before, but I think that was a 640x480 sub-stream. I wonder what resolution these AI models are trained at ? Since I remember Deepstack or early CPAI reducing the resolution of images to be tested, I bet they are small - MikeLud will know since he made the custom models for CPAI.

Oh, and don't forget to turn OFF DAT files when you are done, since as far as I know they will still fill up your Alerts folder and won't be deleted by BI5 so you have to manually delete them. I found out the hard way

[Ferguson]

I can't believe I missed that size up on the top right, i was looking in the data dump itself.

Just checked, and mine matches the camera resolution.

Though I was disappointed to find it found a table end as a human when a skunk walked by. CPAI is really pretty bad (or I have the setup poor, not sure, but there does not seem to be a lot of tuning available other than raising the confidence level).

human.jpg (85.84 KiB) Viewed 3923 times

[Pogo]

"For your amusement: how the AI's see the world..."

Equally as amusing is how AI users seem to see the world.