genie-app Sample Code Tutorial

Prerequisite

Linux: Building genie-app is independent of libGenie.so and these targets can be built in any order. However, libGenie.so will be required for the successful execution of genie-app

Windows: Building genie-app.exe is dependent on Genie.lib. Prior to building genie-app.exe, we must first copy Genie.lib to the build app’s directory to ensure the linker is able to locate this dependency.

Aside from the target-specific libraries and config files required for any Genie SDK application, genie-app requires a script file. The below section shows an example script file that enables execution of the Llama2-7b model.

Building genie-app

x86

cd ${SDK_ROOT}/examples/Genie/genie-app
make x86

After executing make as shown above, you should be able to see genie-app in bin/x86_64-linux-clang

OE

unset LD_LIBRARY_PATH
source PATH_OF_OE_LINUX_GCC_11.2/environment-setup-armv8a-oe-linux
cd ${SDK_ROOT}/examples/Genie/genie-app
make oe

After executing make as shown above, you should be able to see genie-app in bin/aarch64-oe-linux-gcc11.2

Android

cd ${SDK_ROOT}/examples/Genie/genie-app
make android

After executing make as shown above, you should be able to see genie-app in bin/aarch64-android

Windows

Open the Developer Command Prompt for Visual Studio and run the following commands:

cd ${SDK_ROOT}/examples/Genie/genie-app
mkdir build & cd build
cmake ../ -A <x64, ARM64>
cmake --build ./ --config Release

After executing CMake as shown above, you should be able to see genie-app.exe in bin/Release

Sample script file

# Version
version

# Profile creation
profile create profile1
ls

# Dialog creation
dialog config create config1 llama2-7b-genaitransformer.json
dialog config bind profile config1 profile1
dialog create dialog1 config1
ls

# Queries
dialog query dialog1 "Here's a summary of llamas in 10 words."

# Save profiling data
profile save profile1 vinay.json

# Clean up
dialog config free config1
dialog free dialog1
profile free profile1
ls

# Sleep for 5000 ms
sleep 5000

# Loop through 5 iterations
loop 5

# Profile creation
profile create profile1
ls

# Dialog creation
dialog config create config1 llama2-7b-genaitransformer.json
dialog config bind profile config1 profile1
dialog create dialog1 config1
ls

# Queries
dialog query dialog1 "Here's a summary of llamas in 10 words."

# Save profiling data
profile save profile1 profiling_data.json

# Clean up
dialog config free config1
dialog free dialog1
profile free profile1

# End Loop
endloop

# Exit
exit

Now, we can proceed to executing our script file with genie-app.

Executing sample genie-app

Running genie-app is very straightforward. As shown below, simply provide the script file via the -s argument and that’s it! You should now be able to observe the sequential execution of each command in your script file as well as any helpful comments you may have provided. Congrats!

genie-app -s script_file