genie-app

The genie-app tool is a robust test application that allows for general usage of the Genie library. This tool exposes API-level commands and related utilities to enable a wide range of various configurations that more closely emulates developers’ Genie SDK use cases.

DESCRIPTION:
------------
Tool for text to text inference of LLMs using Genie.


REQUIRED ARGUMENTS:
-------------------
-s or --script                        <FILE>      Script file that contains chronological sequence of script commands

OPTIONAL ARGUMENTS:
-------------------
-h or --help                                      Show this help message and exit. Help message contains all currently supported script commands

SCRIPT COMMANDS:
----------------

dialog config create       CONFIG_NAME FILE.json
dialog config bind profile CONFIG_NAME PROFILE_NAME
dialog config bind log     CONFIG_NAME LOG_NAME
dialog config free         CONFIG_NAME
dialog create              DIALOG_NAME CONFIG_NAME
dialog query               DIALOG_NAME ["PROMPT_STR"/PROMPT_FILE] [OUTPUT_FILE]
dialog rewindQuery         DIALOG_NAME ["PROMPT_STR"/PROMPT_FILE] [OUTPUT_FILE]
dialog embeddingQuery      DIALOG_NAME EMBEDDINGS_PATH EMBEDDINGS_TABLE_PATH
dialog embeddingTokenQuery DIALOG_NAME EMBEDDINGS_PATH EMBEDDINGS_TABLE_PATH
dialog tokenQuery          DIALOG_NAME TOKENS_PATH
dialog applyLora           DIALOG_NAME ENGINE_NAME ADAPTER_NAME
dialog setLoraStrength     DIALOG_NAME ENGINE_NAME TENSOR_NAME_1 ALPHA_1 ...
dialog reset               DIALOG_NAME
dialog save                DIALOG_NAME PATH
dialog restore             DIALOG_NAME PATH
dialog setPerformance      DIALOG_NAME PERFORMANCE_MODE
dialog free                DIALOG_NAME
dialog getValue            DIALOG_NAME GENIE_DIALOG_PARAM
dialog getTokenizer        DIALOG_NAME TOKENIZER_NAME
dialog setMaxNumTokens     DIALOG_NAME MAX_NUM_TOKENS
dialog setStopSequence     DIALOG_NAME ["STOP_SEQUENCE_JSON_STR"/STOP_SEQUENCE_JSON_FILE]

embedding config create       CONFIG_NAME FILE.json
embedding config bind profile CONFIG_NAME PROFILE_NAME
embedding config bind log     CONFIG_NAME LOG_NAME
embedding config free         CONFIG_NAME
embedding create              EMBEDDING_NAME CONFIG_NAME
embedding generate            EMBEDDING_NAME ["PROMPT_STR"/PROMPT_FILE] [OUTPUT_FILE]
embedding setPerformance      EMBEDDING_NAME PERFORMANCE_MODE
embedding free                EMBEDDING_NAME

log create LOG_NAME LEVEL [OUTPUT_FILE.txt]
log free   LOG_NAME

node config create         CONFIG_NAME FILE.json
node config free           CONFIG_NAME
node config bind profile   CONFIG_NAME PROFILE_NAME
node config bind log       CONFIG_NAME LOG_NAME
node create                NODE_NAME CONFIG_NAME
node set text              NODE_NAME NODE_IO TEXT_PROMPT
node set textFile          NODE_NAME NODE_IO TEXT_PROMPT_FILE_PATH
node set image             NODE_NAME NODE_IO IMAGE_PATH
node set embedding         NODE_NAME NODE_IO EMBEDDING_PATH
node set textCallback      NODE_NAME NODE_IO
node set embeddingCallback NODE_NAME
node applyLora             NODE_NAME ENGINE_NAME ADAPTER_NAME
node setLoraStrength       NODE_NAME ENGINE_NAME TENSOR_NAME_1 ALPHA_1 ...
node free                  NODE_NAME

pipeline config create       CONFIG_NAME FILE.json
pipeline config bind profile CONFIG_NAME PROFILE_NAME
pipeline config bind log     CONFIG_NAME LOG_NAME
pipeline config free         CONFIG_NAME
pipeline create              PIPELINE_NAME CONFIG_NAME
pipeline add                 PIPELINE_NAME NODE_NAME
pipeline connect             PIPELINE_NAME PRODUCER_NODE_NAME PRODUCER_NODE_IO CONSUMER_NODE_NAME CONSUMER_NODE_IO
pipeline execute             PIPELINE_NAME [OUTPUT_FILE]
pipeline reset               PIPELINE_NAME
pipeline save                PIPELINE_NAME PATH
pipeline restore             PIPELINE_NAME PATH
pipeline free                PIPELINE_NAME

profile config create CONFIG_NAME FILE.json
profile config free   CONFIG_NAME
profile create        PROFILE_NAME [CONFIG_NAME]
profile save          PROFILE_NAME [OUTPUT_FILE.json]
profile free          PROFILE_NAME

tokenizer encode TOKENIZER_NAME INPUT_STRING
tokenizer decode TOKENIZER_NAME TOKEN_SIZE TOKEN_ID_1 ...

accuracy config create CONFIG_NAME FILE.json
accuracy config free   CONFIG_NAME
accuracy create        ACCURACY_NAME CONFIG_NAME [dialog/node] DIALOG_NAME
accuracy compute       ACCURACY_NAME OUTPUT_FILE.json
accuracy free          ACCURACY_NAME

version
help
script FILE
loop [NUMBER OF ITERATIONS]
endloop
sleep [DURATION in ms]
history [save FILE]
ls
exit

See Tutorials for a reference example on how to use the genie-app tool.

Script Statement to C API Mapping

In genie-app scripts, names are used to represent Genie C API handles. Generally, DIALOG_NAME translates to a GenieDialog_Handle_t, PIPELINE_NAME to a GeniePipeline_Handle_t, and so on. If there is a CONFIG_NAME argument, it would correspond to a configuration handle to the respective API component. For example, the “dialog create” CONFIG_NAME would refer to a GenieDialogConfig_Handle_t.

The following genie-app script statements do not have a corresponding Genie C API function:

• help

• ls

• history

• script

• loop/endloop

• sleep

• exit

The following Genie C APIs are not yet supported in genie-app:

• GenieDialog_tokenQuery

• GenieDialog_getSampler

• GenieDialog_signal

• GenieDialog_setPriority

• GenieDialog_setOemKey

• GenieDialog_bindEngine

• GenieDialog_getEngine

• GenieDialog_getPerformancePolicy

• GenieEmbedding_getPerformancePolicy

• GenieNode_bindEngine

• GenieNode_getEngine

• GenieNode_getSampler

• GenieNode_getTokenizer

• GeniePipeline_setPriority

• GeniePipeline_setOemKey

• GenieSamplerConfig_createFromJson

• GenieSamplerConfig_setParam

• GenieSamplerConfig_free

• GenieSampler_applyConfig

• GenieSampler_registerCallback

• GenieSampler_registerUserDataCallback

• GenieEngineConfig_createFromJson

• GenieEngineConfig_bindProfiler

• GenieEngineConfig_bindLogger

• GenieEngineConfig_free

• GenieEngine_create

• GenieEngine_free

Statement	Genie C API
dialog config create	GenieDialogConfig_createFromJson
dialog config bind profile	GenieDialogConfig_bindProfiler
dialog config bind log	GenieDialogConfig_bindLogger
dialog config free	GenieDialogConfig_free
dialog create	GenieDialog_create
dialog query	GenieDialog_query
dialog rewindQuery	GenieDialog_query
dialog embeddingQuery	GenieDialog_embeddingQuery
dialog embeddingTokenQuery	GenieDialog_embeddingTokenQuery
dialog tokenQuery	GenieDialog_tokenQuery
dialog applyLora	GenieDialog_applyLora
dialog setLoraStrength	GenieDialog_setLoraStrength
dialog reset	GenieDialog_reset
dialog save	GenieDialog_save
dialog restore	GenieDialog_restore
dialog setPerformance	GenieDialog_setPerformancePolicy
dialog free	GenieDialog_free
dialog getTokenizer	GenieDialog_getTokenizer
dialog getValue	GenieDialog_getValue
dialog setMaxNumTokens	GenieDialog_setMaxNumTokens
dialog setStopSequence	GenieDialog_setStopSequence
embedding config create	GenieEmbeddingConfig_createFromJson
embedding config bind profile	GenieEmbeddingConfig_bindProfiler
embedding config bind log	GenieEmbeddingConfig_bindLogger
embedding config free	GenieEmbeddingConfig_free
embedding create	GenieEmbedding_create
embedding generate	GenieEmbedding_generate
embedding setPerformance	GenieEmbedding_setPerformancePolicy
embedding free	GenieEmbedding_free
log create	GenieLog_create
log free	GenieLog_free
node config create	GenieNodeConfig_createFromJson
node config bind profile	GenieNodeConfig_bindProfiler
node config bind log	GenieNodeConfig_bindLogger
node config free	GenieNodeConfig_free
node create	GenieNode_create
node set text	GenieNode_setData
node set textFile	GenieNode_setData
node set image	GenieNode_setData
node set embedding	GenieNode_setData
node set textCallback	GenieNode_setTextCallback
node set embeddingCallback	GenieNode_setEmbeddingCallback
node applyLora	GenieNode_applyLora
node setLoraStrength	GenieNode_setLoraStrength
node free	GenieNode_free
pipeline config create	GeniePipelineConfig_createFromJson
pipeline config bind profile	GeniePipelineConfig_bindProfiler
pipeline config bind log	GeniePipelineConfig_bindLogger
pipeline config free	GeniePipelineConfig_free
pipeline create	GeniePipeline_create
pipeline add	GeniePipeline_addNode
pipeline connect	GeniePipeline_connect
pipeline execute	GeniePipeline_execute
pipeline reset	GeniePipeline_reset
pipeline save	GeniePipeline_save
pipeline restore	GeniePipeline_restore
pipeline free	GeniePipeline_free
profile config create	GenieProfileConfig_createFromJson
profile config free	GenieProfileConfig_free
profile create	GenieProfile_create
profile save	GenieProfile_getJsonData
profile free	GenieProfile_free
tokenizer encode	GenieTokenizer_encode
tokenizer decode	GenieTokenizer_decode
accuracy config create	GenieAccuracyConfig_createFromJson
accuracy config free	GenieAccuracyConfig_free
accuracy create	GenieAccuracy_createFromDialog GenieAccuracy_createFromNode
accuracy compute	GenieAccuracy_compute
accuracy free	GenieAccuracy_free
version	Genie_getApiMajorVersion Genie_getApiMinorVersion Genie_getApiPatchVersion