Custom model not respected - all custom models still used

[PaulDaisy]

I am trying to exclude the daytime models from use at night. Apparently, that can only be done on a per camera basis - or I have not found a global setting for this.

So, for each camera, on the Alert tab, I unchecked "Sync with profile 1", enabled AI, configured AI by entering "ipcam-dark" into the Custom model field for the camera.
When I look at the CPAI console, it shows that BI checked something against a bunch of other custom models - ipcam-general, ipcam-combined, license-plate, actionnetv2.

I thought that maybe the To Confirm field was mismatched, as it by default includes all labels from all models, and even some not present in any model installed by default (ups,fedex etc). So I replaced it with just the list from ipcam-dark. The console still shows that all models are used.

I went and disabled AI in all cameras. That stopped BI requests to the CPAI completely. But as soon as I enable one camera AI lookup with ipcam-dark explicitly specified, the CPAI shows that all custom models are used.

Is this a bug?

Cheers,
PD

[PaulDaisy]

The below log is for one single camera, with only ipcam-dark enabled, and with Bicycle,Bus,Car in the Confirm field. I also tried lowercase labels, thinking those may be case sensitive.
All other cameras have AI turned OFF completely for the Night profile.
Upon studying the CPAI log, I noticed some repetition:

22:15:19:Object Detection (YOLOv5 6.2): Detecting using ipcam-dark
22:15:19:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...6ae8dd) ['Found Car'] took 41ms
22:15:19:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
22:15:19:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...3ef2b9) ['Found vehicle'] took 85ms
22:15:19:Object Detection (YOLOv5 6.2): Detecting using license-plate
22:15:19:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...3f1f18) ['No objects found'] took 63ms

22:17:18:Object Detection (YOLOv5 6.2): Detecting using actionnetv2
22:17:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...bd1440) ['No objects found'] took 380ms
22:17:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-animal
22:17:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...248a91) ['No objects found'] took 41ms
22:17:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-combined
22:17:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...65d9ed) ['Found truck, car'] took 70ms

22:17:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-dark
22:17:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...169cc2) ['Found Car'] took 58ms
22:17:19:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
22:17:19:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...515994) ['Found vehicle'] took 68ms
22:17:19:Object Detection (YOLOv5 6.2): Detecting using license-plate
22:17:19:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...05b699) ['No objects found'] took 62ms

22:19:17:Object Detection (YOLOv5 6.2): Detecting using actionnetv2
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...eca7c8) ['No objects found'] took 274ms
22:19:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-animal
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...a8209f) ['No objects found'] took 46ms
22:19:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-combined
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...955958) ['Found truck, car'] took 73ms

22:19:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-dark
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...6fef2d) ['Found Car'] took 108ms
22:19:18:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...f80704) ['Found vehicle'] took 44ms
22:19:18:Object Detection (YOLOv5 6.2): Detecting using license-plate
22:19:18:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...aad6d2) ['No objects found'] took 53ms

So it appears that ipcam-dark triggers a lookup in ipcam-general, which then triggers a lookup in license-plate. I don't have face recognition or ALPR turned on at all.
What is going on, anyone knows?

[TimG]

Firstly you can see which models are being used per camera by enabling the DAT files. It will show all that are in use on each camera. You can then use the :0 tag to prevent any running that you don't need.

An old DAT file of mine showed this:

Zero.png (131.55 KiB) Viewed 2085 times

That DAT file shows I had the "IPcam-combined" and "license-plate" models running on that camera.

The Help file shows:

By default, all Custom model files are considered, although you may select a subset here by specifying one or more (comma delimited; omit the file extensions). Custom models are enabled on the global Settings AI page. You may instead treat this as an exclusion list if you only list models with an :0 extension. You may also add faces:0 or objects:0 or alpr:0 here to specifically skip face, default object, or license plate detections respectively. Tip: specify a single non-existent model name (“none”) and no custom models will be used. Examples:

a,b,c use only models a,b,c
b:0,c:0 use all models except b and c
a,b:0 use only model a (the b:0 is unnecessary)
a,faces:0 use only model a and skip face recognition
objects:0 use all custom models, but not the default object model

And here are examples from two of my cameras:

One.png (15.16 KiB) Viewed 2085 times

And:

Two.png (14.88 KiB) Viewed 2085 times

Don't forget to turn off DAT files when you have finished, as they are not automatically deleted. I believe the :0 has to go before the active models, but that info may be out of date. My :0 entries do work though.

[PaulDaisy]

Thanks Tim. I too read the help file, and it does say exactly what you copied:

By default, all Custom model files are considered, although you may select a subset here by
specifying one or more (comma delimited; omit the file extensions). Custom models are
enabled on the global Settings AI page. You may instead treat this as an exclusion list if you
only list models with an :0 extension. You may also add faces:0 or objects:0 or alpr:0 here to
specifically skip face, default object, or license plate detections respectively. Tip: specify a
single non-existent model name (“none”) and no custom models will be used. Examples:
a,b,c use only models a,b,c
b:0,c:0 use all models except b and c
a,b:0 use only model a (the b:0 is unnecessary)
a,faces:0 use only model a and skip face recognition
objects:0 use all custom models, but not the default object model

I tried setting license-plate:0 but it still was used.
I set up .DAT files for the night mode but none of the triggers were confirmed so alerts didn't save while I watched, and then I turned AI off in the night mode - it doesn't detect anything well at all in my case. I will try to set confidence level for confirmation to like 5% to save any alert at night, because I did see delivery drivers come to my door while I was playing with it, and the AI processed them and didn't alert. Or I will set it to record alerts regardless of confidence level, just to see the .DAT files.

This all makes me wonder if the models are interdependent, and ipcam-dark references ipcam-combined, and if a "vehicle" is detected, they reference license-plate. I don't know how the models work, and will look on the CPAI forum for that.

[TimG]

Try alpr:0 as that works for me.

[PaulDaisy]

Update.

I went all over all cameras, and set their Night profile AI custom model field to the following:

delivery:0,ipcam-animal:0,ipcam-combined:0,ipcam-general:0,license-plate:0,alpr:0,objects:0,faces:0

This should leave only ipcam-dark active (I deleted the "activity" model that is installed by default, I don't know what it is supposed to detect other than random items as "sleeping").

Here is the output from the AI console after I finished that:

18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...16bf5c) ['No objects found'] took 88ms
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...c93e23) ['No objects found'] took 103ms
18:16:44:Object Detection (YOLOv5 6.2): Detecting using ipcam-dark
18:16:44:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...c322ec) ['No objects found'] took 55ms
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...6a013b) ['No objects found'] took 67ms
18:16:44:Object Detection (YOLOv5 6.2): Detecting using license-plate
18:16:44:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...1257f8) ['No objects found'] took 79ms
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...416058) ['No objects found'] took 56ms
18:16:44:Object Detection (YOLOv5 6.2): Detecting using license-plate
18:16:44:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...a655a5) ['No objects found'] took 55ms
18:16:59:Object Detection (YOLOv5 6.2): Detecting using delivery
18:16:59:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...ad1500) ['No objects found'] took 75ms
18:16:59:Object Detection (YOLOv5 6.2): Detecting using ipcam-animal
18:16:59:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...b94359) ['No objects found'] took 82ms
18:16:59:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...0b3d46) ['No objects found'] took 67ms
18:16:59:Object Detection (YOLOv5 6.2): Detecting using ipcam-combined
18:16:59:Object Detection (YOLOv5 6.2): Detecting using ipcam-dark
18:16:59:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...b0b37e) ['No objects found'] took 119ms
18:16:59:Object Detection (YOLOv5 6.2): Detecting using ipcam-general
18:17:00:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...233d24) ['No objects found'] took 63ms
18:17:00:Object Detection (YOLOv5 6.2): Detecting using license-plate
18:17:00:Response rec'd from Object Detection (YOLOv5 6.2) command 'custom' (...cd6bf6) ['No objects found'] took 64ms

So, either the Custom model field in the Camera settings doesn't wok, or something unexplained is going on.

[PaulDaisy]

Well, I don't know what to make of all of this.
I looked at the night DAT files, and the only model BI says is used is ipcam-dark. However, the CPAI console says during that same time that all kinds of models are being scanned, just like my posts below indicated.
No idea what is going on, but BI processing times from DAT files certainly look faster with only one model. So it could be something under the hood in CPAI that is happening, that BI isn't calling for explicitly.

[PaulDaisy]

And to wrap up this thread...

I spent several hours looking through DAT files of confirmed alerts, cancelled alerts and timed out alerts. I am coming to a conclusion that CPAI may be fun, but not useful for anything reliable.
My wife, in broad daylight, 30 feet from the cameras is 79% "cat", I myself am a 75% dog and 89% person, in the same frame. I am 40 feet away from the camera, on the lawn. I guess, people are supposed to stay on pavement, and if its on the lawn, its a pet?

My neighbor's RV is 95% "bus" (good!) but also is 79% "garbage". I don't think he'd appreciate that.
A static garden post with a light on top of it is 75% "bird".
A walking person on the driveway in the evening is a "dog".
A square sunspot on a brick wall is "UPS".
In broad daylight, my neighbors walking the dog, not even 50 feet straight across from the camera, are "persons", but then the woman is 79% cat (I am noticing a pattern here! His wife, my wife are "cats"...), and their dog is not even analyzed by CPAI.

Sigh. And I was fussing about which model is used at night?...

Since I am 75% dog, I put my tail between my legs, and retreat.

[jrbeddow]

I ran into this as well, the solution seems to be to explicitly specify (all of) the desired models to use in the static detection dialog box (if you have this in use at all). The downside is that (as of 5.9.9.10) this static detection box does not "track" with profile changes, it remains the same in all profiles. Which means that you may have to insert ALL the desired "in-use" models here, regardless of whether that profile uses that model (otherwise the release notes DO indicate that ALL custom models will be used), when in fact that profile may only require a specific model (Night profile corresponding with ipcam-dark, for example). Still, this will likely be far better than running ALL custom models, which it will default to if nothing is specified in this box.

Hopefully this eventually becomes a dialog box that does track/change on a per-profile case basis.