Convert to QNN for Linux Host on HTP Backend¶

Note

This is Part 3 of the Convert to QNN tutorial for Linux host machines. If you have not completed Part 2, please do so here.

Warning

HTP processors require quantized models instead of full precision models. If you do not have a quantized model, please follow Step 2 of the CNN to QNN tutorial to build one.

Additional HTP Required Setup¶

Running the model on a target device’s HTP requires the generation of a **serialized context*.*

On the Linux Host:

Navigate to the directory where you built the model in the previous steps:
```
cd /tmp/qnn_tmp
```
Users can set the custom options and different performance modes to HTP Backend through the backend config. Please refer to QNN HTP Backend Extensions for various options available in the config.

Refer to the example below for creating a backend config file for the QCS6490/QCM6490 target with mandatory options passed in:

{
    "graphs": [
        {
            "graph_names": [
                "Inception_v3"
            ],
            "vtcm_mb": 2
        }
    ],
    "devices": [
        {
            "htp_arch": "v68"
        }
    ]
}

The above config file with minimum parameters such as backend extensions config specified through JSON is given below:

{
    "backend_extensions": {
        "shared_library_path": "path_to_shared_library",  // give path to shared extensions library (.so)
        "config_file_path": "path_to_config_file"         // give path to backend config
    }
}

To generate the context, update <path to JSON of backend extensions> below with the config you wrote above, then run the command:

"$QNN_SDK_ROOT/bin/${QNN_TARGET_ARCH}/qnn-context-binary-generator" \
    --backend "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnHtp.so" \
    --model "${QNN_MODEL_PATH}" \
    --binary_file "${CONVERTED_MODEL_PATH##*/}.serialized" \
    --config_file <path to JSON of backend extensions>

This creates the serialized context at:
- ${PWD}/output/${CONVERTED_MODEL_PATH##*/}.serialized.bin

Transferring over all relevant files¶

On the target device, open a terminal and make a destination folder by running:

mount -o remount,rw /
mkdir -p /data/local/tmp
cd /data/local/tmp
ln -s /etc/ /data/local/tmp
chmod -R 777 /data/local/tmp
mkdir -p /data/local/tmp/qnn_tutorial

Determine your target device’s SnapDragon architecture by looking up your chipset in the Supported Snapdragon Devices table.
Update the “X” values below and run the commands to set HTP_ARCH to match the version number found in the above table. Only the 2 digits at the end should update, and they should have the same version. Ex. For “V68”, the proper value would be hexagon-v68.
```
$HTP_VERSION="XX"
$HTP_ARCH="hexagon-v${HTP_VERSION}"
```

Use scp to transfer libQnnHtp.so as well as other necessary executables from your host machine to /data/local/tmp/qnn_tutorial on the target device.

scp "$QNN_SDK_ROOT/lib/${QNN_TARGET_ARCH}/libQnnHtp.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/lib/${DSP_ARCH}/unsigned/*" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnHtpV${HTP_VERSION}Stub.so" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/output/Inception_v3.serialized.bin" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"

Check the Backend table to see if there are any other processor-specific executables needed for your target processor (DSP) and your target device’s architecture ($QNN_TARGET_ARCH). Use similar syntax above for scp to transfer any additional .so files listed below your selected target architecture in this table. (There may be none!)

Note

Ensure you transfer the hexagon-v## values (in addition to the other architecture files!)
Use scp to transfer the example built model. Update the x64 folder below to the proper folder for your built model. The folder name depends on your host machine’s architecture.
```
scp "${QNN_MODEL_PATH}"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
```

Transfer the input data, input list, and script from the QNN SDK examples folder into /data/local/tmp/qnn_tutorial on the target device using scp in a similar way:

scp -r "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/target_raw_list.txt"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/imagenet_slim_labels"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
scp "${QNN_SDK_ROOT}/examples/Models/InceptionV3/scripts/show_inceptionv3_classifications.py"  "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"

Transfer qnn-net-run from $QNN_SDK_ROOT/bin/$QNN_TARGET_ARCH/qnn-net-run to /data/local/tmp/qnn_tutorial on the target device:
```
scp "${QNN_SDK_ROOT}/bin/${QNN_TARGET_ARCH}/qnn-net-run" "${TARGET_USER}@${TARGET_IP}:/data/local/tmp/qnn_tutorial"
```

On the target device, set the environment variables:

export LD_LIBRARY_PATH="/data/local/tmp/qnn_tutorial"
export ADSP_LIBRARY_PATH="/data/local/tmp/qnn_tutorial"

Doing inferences on the target device processor¶

Open a terminal instance on the target device.
```
ssh "${TARGET_USER}@${TARGET_IP}"
```

Note

You will have to login with your target device’s login for that username.

On your target device, navigate to the directory containing the test files:
```
cd /data/local/tmp/qnn_tutorial
```

Run the following command on the target device to execute an inference:

./qnn-net-run \
   --backend "libQnnHtp.so" \
   --input_list "target_raw_list.txt" \
   --retrieve_context "<model name>.serialized.bin" \
   --output_dir "./output"

Run the following script on the target device to view the classification results:

Note

You can alternatively copy the output folder back to your host machine with scp and run the following script there to avoid having to install python on your target device.

python3 ".\show_inceptionv3_classifications.py" \
    -i ".\cropped\raw_list.txt" \
    -o "output" \
    -l ".\imagenet_slim_labels.txt"

Verify that the classification results in output match the following: 1. ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/trash_bin.raw 0.777344 413 ashcan 2. ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/chairs.raw 0.253906 832 studio couch 3. ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/plastic_cup.raw 0.980469 648 measuring cup 4. ${QNN_SDK_ROOT}/examples/Models/InceptionV3/data/cropped/notice_sign.raw 0.167969 459 brass