Convert to QNN for Linux Host on Linux / Android / QNX Target

Note

This is Part 2 of the Convert to QNN tutorial for Linux host machines. If you have not completed Part 1, please do so here.

Step 3. Model Build on Linux Host for Linux Target

Once the model has been converted into QNN format, the next step is to build it so it can run on the target device’s operating system with qnn-model-lib-generator.

Based on the operating system and architecture of your target device, choose one of the following build instructions.

Warning

For cases where the “host machine” and “target device” are the same, you will need to adapt the steps to handle files locally instead of transferring them to a remote device.

Note

Please continue to use the same terminal you were using on your host machine from part 1.

  1. Create a directory on your host machine where your newly built files will live by running:

    mkdir -p /tmp/qnn_tmp

  2. Navigate to the new directory:

    cd /tmp/qnn_tmp

  3. Copy over the QNN .cpp and .bin model files to /tmp/qnn_tmp/:

    cp "${CONVERTED_MODEL_PATH}.cpp" "${CONVERTED_MODEL_PATH}.bin"  /tmp/qnn_tmp/

  4. Choose the most relevant supported target architecture from the following list:

    Warning

    If you don’t know which one to choose, you can run the following commands on your target device to get more information: uname -a, cat /etc/os-release, and gcc --version.

    Target Architecture

    Build Instruction

    64-bit Linux targets

    x86_64-linux-clang

    64-bit Android devices

    aarch64-android

    Hexagon Tensor Processor (HTP)

    x86_64-linux-clang

    Qualcomm’s QNX OS

    aarch64-qnx - Note: This architecture is not supported by default in the QNN SDK.

    OpenEmbedded Linux (GCC 11.2)

    aarch64-oe-linux-gcc11.2

    OpenEmbedded Linux (GCC 9.3)

    aarch64-oe-linux-gcc9.3

    OpenEmbedded Linux (GCC 8.2)

    aarch64-oe-linux-gcc8.2

    Ubuntu Linux (GCC 9.4)

    aarch64-ubuntu-gcc9.4

    Ubuntu Linux (GCC 7.5)

    aarch64-ubuntu-gcc7.5

  5. On your host machine, set the target architecture of your target device by setting QNN_TARGET_ARCH to your device’s target architecture:

    export QNN_TARGET_ARCH="your-target-architecture-from-above"

    For example:

    export QNN_TARGET_ARCH="x86_64-linux-clang"

  6. Run the following command on your host machine to generate the model library:

    python3 "${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-model-lib-generator" \
    -c "${CONVERTED_MODEL_PATH##*/}.cpp" \
    -b "${CONVERTED_MODEL_PATH##*/}.bin" \
    -o model_libs \
    -t ${QNN_TARGET_ARCH}

    1. c - This indicates the path to the .cpp QNN model file.

    2. b - This indicates the path to the .bin QNN model file. (b is optional, but at runtime, the .cpp file could fail if it needs the .bin file, so it is recommended).

    3. o - The path to the output folder.

    4. t - Indicate which architecture to build for.

  7. Run ls /tmp/qnn_tmp/model_libs/${QNN_TARGET_ARCH} and verify that an output file ending in .so is inside. - You will use the .so file on the target device to execute inferences. - The output .so file will be located in the model_libs directory, named according to the target architecture.

    • For example: model_libs/x64/Inception_v3.so or model_libs/aarch64/Inception_v3.so.

  8. Once you’ve located your relevant .so file, save the full path of the file to an environment variable:

export QNN_MODEL_PATH="/tmp/qnn_tmp/model_libs/<path to .so>"

Step 4. Use the built model on specific processors

Now that you have an executable version of your model, the next step is to transfer the built model and all necessary files to the target processor, then to run inferences on it.

  1. Install all necessary dependencies from Setup.

  2. Follow the below SSH setup instructions.

  3. Follow the instructions for each specific processor you want to run your model on.

Sub-Step 1: Ensure that you follow the processor-specific Setup instructions for your host machine :doc:`here<../setup>`.

Sub-Step 2: Set up SSH on the target device.

Here we use OpenSSH to copy files with scp later on and run scripts on the target device via ssh. If that does not work for your target device, feel free to use any other method of transferring the files over. (Ex. adb for Android debugging or USB with manual terminal commands on the target device)

  1. Ensure that both the host device and the target device are on the same network for this setup.

    • Otherwise, OpenSSH requires port-forwarding to connect.

  2. On your target device, install OpenSSH Server if it is not already installed.

    • Example for an Ubuntu device:

# Update package lists
sudo apt update

# Install OpenSSH server
sudo apt install openssh-server

# Check SSH service status
sudo systemctl status ssh

# Start SSH service if it's not running
sudo systemctl start ssh

# Enable SSH service to start on boot
sudo systemctl enable ssh

Note

You can turn off the OpenSSH Server service later by running sudo systemctl stop ssh if you want to.

  1. On your target device, run ifconfig to get the IP address of your target device.

  2. On your host machine, set a console variable for your target device’s inet addr from above to see its ipv4 address (replacing 127.0.0.1 below).

export TARGET_IP="127.0.0.1"

  1. Also set the username of the desired account on your target device (you can find it by running whoami on your target device if you are logged into the desired account).

export TARGET_USER="your-linux-account-username"

  1. On your host machine, install OpenSSH Client by running the following commands:

sudo apt update
sudo apt install openssh-client
sudo systemctl status ssh
sudo systemctl start ssh   # Only if the service is not running
sudo systemctl enable ssh

Note

From this point on you should be able to connect to ssh from the terminal. You may need to open another terminal to do so though.

Sub-Step 3: Follow the steps below for whichever processor you would like to run your model on.