General AI Settings

[TimG]

and those times are cpu/gpu dependant,

That brings up another question...the new readouts on the bottom status bar occasionally (but not always) show a reading for a GPU....but I don't have a graphics card, so I wouldn't think it would find a GPU. (??) Other times it only shows the CPU and memory, it doesn't show the GPU at all.

Is it reading something from the onboard video, which is what I'm using?

new-readouts.png

I checked again, and I have seen this now. I do have hardware accelerated decode OFF with my i7-8700k, so it looks like another question for Ken.

Is it interesting to check "Use main stream if available" in AI ?

I'm guessing that is another one for the list. I read somewhere that CPAI reduces the resolution if the resolution is too high, so feeding higher resolution images to it will slow everything down. I've had a search around the www, and found a post on the CPAI forum by MikeLud for DeepStack that may still be true - See this post. It looks like 640 x 640 (0.4MP) may be the sweet spot for images being sent to CPAI.

That gives us:

1) How long should the typical or baseline "Analyze Time" be?
2) What's the typical or recommended number of "real-time images to analyze"?
3) Why is BI5 now showing my intel gpu is being used on the bottom of the BI5 window?
4) Should we use "Use main stream if available", or does CPAI immediately reduce the resolution?
5) If CPAI does reduce the image resolution, what is the optimum size image/sub stream for BI5 to send to CPAI to prevent time being wasted re-sizing images? Is it 640 x 640?
6) What effect does the "Tiny/Low/Medium/High" setting on the BI5 general settings AI page have?

[louyo]

Maybe some help for the beginner:
I suggest that you do some experimenting with the CP/AI side as well as BI.
I have a rather ancient, slow video card that I only use for BI/CPAI testing. I am a little crippled because the test system is an older workstation running ESXi. I have a W10 VM for BI, and a Debian Linux VM running CP Docker. The NVIDIA card is passed through to the Debian. I am sure there is overhead.

1. Make some jpegs from your camera scenes. I made some with people and vehicles, which is all I am interested in. Got these from triggered recordings.

2. I bring up both the Codeproject main screen and the Explorer (localhost:3268). I do this on both the Debian system and on the BI system. This lets me easily compare detection times with and without the GPU in my setup.

3. Turn off detect static objects in BI. Otherwise the logs run continuously.

4. Stop everything but object detection in CP. I use YOLO, have not tried .Net.

5. OK, in each of the Explorers, I do some testing in "object detection" and "Custom object detection". This will show me what objects AI sees and the confidence level.
Going back to the main CP screen, I can see which model was used and the detect times, in the logs.

My detect times with GPU are in the 60ms range (depending on the model) and a little over 100ms without GPU. Obviously best case because there is no BI involvement in my tests.

If you are on a single system running both BI and AI, you can enable and disable the GPU to test (the 3 dots on the right). I keep everything stopped but the ones I use.

[MikeLud]

and those times are cpu/gpu dependant,

That gives us:

1) How long should the typical or baseline "Analyze Time" be?

This is hard to put a baseline, it depends on hardware and model, below can be used as some baseline, but still your system will vary from the below

2) What's the typical or recommended number of "real-time images to analyze"?

I typically use about 10

3) Why is BI5 now showing my intel gpu is being used on the bottom of the BI5 window?

I am not sure, it could be you are using your intel gpu to decode

4) Should we use "Use main stream if available", or does CPAI immediately reduce the resolution?

For object detection "Use main stream if available" should be unchecked. My new ALPR that is coming out soon will need "Use main stream if available" to be checked

5) If CPAI does reduce the image resolution, what is the optimum size image/sub stream for BI5 to send to CPAI to prevent time being wasted re-sizing images? Is it 640 x 640?

The optimum sub stream would be D1 or VGA

6) What effect does the "Tiny/Low/Medium/High" setting on the BI5 general settings AI page have?

"Tiny/Low/Medium/High" uses different YOLOv5 models see below

Code: Select all

    # see https://github.com/ultralytics/yolov5 for resolution data
    MODEL_SETTINGS = {
        "tiny":   Settings(STD_MODEL_NAME = "yolov5n", RESOLUTION = 256), # 640
        "small":  Settings(STD_MODEL_NAME = "yolov5s", RESOLUTION = 256),
        "medium": Settings(STD_MODEL_NAME = "yolov5m", RESOLUTION = 416),
        "large":  Settings(STD_MODEL_NAME = "yolov5l", RESOLUTION = 640),
        "huge":   Settings(STD_MODEL_NAME = "yolov5x", RESOLUTION = 640)  # Not yet included
    }

    # -------------------------------------------------------------------------
    # Setup values

[TimG]

Quite incredible. Thank you for those responses MikeLud.

PS> WOW.

[TimG]

Some feedback from Support too:

1) How long should the typical or baseline "Analyze Time" be? Do we reduce it to just above t_ave shown in DAT files, or is it something else?

I think something in the range of 100-400msec is reasonable ... it will depend on the low/medium/high complexity chosen and the image size (main stream higher) and hardware acceleration.

2) What is the typical or recommended number of "real-time images to analyze"?

This will depend on requirements. Typically you will want more than 1, as that gives the AI the ability to look further into the trigger for items to confirm/cancel, not just the initial trigger image. some people want to keep looking at images the entire duration of the trigger, some are happy with 2-5 and allow the motion detection to reset and re-trigger.

3) Why is BI5 now showing my Intel i7-8700k gpu is being used on the bottom of the BI5 window - especially when my BI5 hardware decoding has been completely disabled since sub streams have sorted out the cpu load?

This is GPU for the entire system ... even Chrome may use some. you can look at the windows task manager or process explorer for some breakdown potentially.

4) Should we use "Use main stream if available", or does CPAI immediately reduce the resolution?

I know that CPAI reduces the image internally, but I am not sure of that size. If you are using main stream you probably want to use HIGH for the mode/complexity setting as well.

5) If CPAI does reduce the image resolution, what is the optimum image/sub stream resolution for BI5 to send to CPAI to prevent time being wasted re-sizing images? Is it 640 x 640? or 640 x 480?

This might be a question for the CPAI staff/developers. it does make sense to avoid extra resizing if unnecessary.

6) What effect does the "Tiny/Low/Medium/High" setting on the BI5 general settings AI page have?

This is really what you have been asking ... it controls the size of the actual data and models used. however I do not have details on this personally.

7) I originally had the issue when changing from DeepStack to CPAI with ghosted settings for the custom folder in General settings/AI which meant I had to edit the registry settings to point to the CPAI custom folder. It still seems a bit flaky there. Will this be resolved, or do I need to do a pc rebuild to get rid of a legacy setting? I'm likely to be doing that for other reasons over the Xmas holiday so I will find out then.

This setting is editable on Settings/AI in the latest releases even with other checkboxes unchecked.

8) I noted that MikeLud has posted on the forum about his forthcoming CPAI ALPR model. I asked him if it could be capable of OCR, and he responded " The next release will also have an OCR module. You would have to see if Ken can do an integration for the OCR module. He is definitely integrating the ALPR module". FWIW, my idea is that BI5 could OCR the name on the side of a van stopping outside my house, and send that info to my Homeseer installation (via MQTT ?) so that Homeseer can speak through my Alexa speakers that "Ocado is here" (The speech part already works). Can that be done by BI5, or am I asking too much ?

This will be great! I will just eventually need to coordinate how the LP is placed in the JSON results from CPAI. he can just let me know how that will work and I will integrate with it.

[MikeBwca]

Wow. Lots of useful info.
Thanks for posting the replies from Ken.