How to use the CLI

This document will guide you through the usage of Zeblok cli

CLI Usability and commands.

Show-URL

This command just outputs the APP URL and datalake Url configured in zeblok configure command

zeblok showUrl

Snapshot

This command is used to create snapshot of your workstation on which it creates a docker image whose tag is emailed to the user on which that tag can be used to create a new workstation with current configuration

zeblok snapshot
? Enter the name for docker file getting generated : <Docker Image Name>
? Select your DataLake Bucket to Deploy (Use arrow keys) <Minio Bucket Selection>

Brief

Snapshot command copy all the files from your notebook with the same structure as in current one with that it also creates requirments.txt with all the installed packages in your current notebook and installs it when you spawn the new workstation with that docker image to have same configuration as you have currently .

requirments.txt is generated automatically with pip list --format=freeze

Base image for workstation used is minimal-notebook:2023.09.20

Components

  1. Docker Image Name- The image name you want the docker to have

  2. Minio Bucket Selection - Select the bucket where you want to store all the data of your notebook

Openvino

This command is used to serve model using openvino

zeblok openvino
? Enter the path for generated IR model (without / at the end): <IR_OUTPUT_PATH>
? Enter the IR output name : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

  1. IR_OUTPUT_PATH - Folder containing the <IR_OUTPUT_NAME>.xml and the <IR_OUTPUT_NAME>.bin

  2. IR_OUTPUT_NAME - IR output name which is same for both .xml and .bin file in the IR_OUTPUT_PATH

  3. BUCKET_NAME - Select the bucket where you want to store the model files

  4. AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API

  5. NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  6. DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  7. KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  8. PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  9. DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

Ideal File Structure

For openvino model serving to work the output path must contain 2 files with same name with extensions *.xml and *.bin

Bentoml

Bentolml model serving serves a bentoml model served built using bentoml build command . It fetches the model with bentoml list --output json command and lists one which you want to serve .

zeblok bentoml
? Select your Model to Deploy: <MODEL_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

  1. MODEL_NAME - Select the model built using the bentoml build command bentoml build

  2. BUCKET_NAME - Select the bucket where you want to store the model files

  3. AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API

  4. NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  5. DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  6. KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  7. PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  8. DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

The model serving of bentoml needs bentoml installed on your workstation

If there is no model to select in first question then try running bentoml build in the model creation .

Mlflow

Mlflow model serving uses two process to serve a model first is to get model into your directory and second one is to serve that model

Commands

1. Get model

To get a model from mlflow register use command

zeblok mlflow
? Enter the mlflow url for model : <MLFLOW_TRACKING_URL>
? Enter the runId for model : <RUN_ID>

Components

  1. MLFLOW_TRACKING_URL - Mlflow tracking URL where your model is stored to .

  2. RUN_ID - Run id of the model you are willing to serve

To create a new mlflow microservice please refer to documentation of AI-Microcloud .

2. Serve Model

Once the command To get the model is done you will need to run modelserve command to serve model

zeblok modelserve
? Enter the rootPath for model in directory : <DOWNLOADED_MODEL_PATH>
? Enter the IR output name : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

  1. DOWNLOADED_MODEL_PATH- Enter folder name where your model is stored .

  2. IR_OUTPUT_NAME- Enter any name that you want to put for model .

  3. AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API

  4. NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  5. DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  6. KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  7. PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  8. DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

The model serving of mlflow needs mlflow installed on your workstation

LLAMA

This command serves LLAMA model with CPP serving .

zeblok llamacpp
? Enter the path for generated IR model (without / at the end): <IR_OUTPUT_PATH>
? Enter the IR output name with its extention : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN> 
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

  1. IR_OUTPUT_PATH- Enter the path where your model and requirements.txt file is situated .

  2. IR_OUTPUT_NAME - Enter the model name with its extension whether its .gguml or .gguf

  3. BUCKET_NAME - Select the bucket where you want to store the model files

  4. AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API

  5. NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  6. DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  7. KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  8. PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  9. DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

IR_OUTPUT_PATH must contain the model and requirements.txt file to work

Make sure you enter IR_OUTPUT_NAME with its extension .

Depending on the size of model it will take time around 2 to 3 hours

VLLM

This command serves model with Vllm model serving .

zeblok vllm 
? Enter the directory name containing your model : <MODEL_DIRECTORY> 
? Select your DataLake Bucket to Deploy : <BUCKET_NAME> 
? Do you want to auto-deploy? : <AUTO_DEPLOY> 
? Select your namespace: <NAMESPACE> 
? Select your DataCenter: <DATACENTER> 
? Select your Kiosk: <KISOK> 
? Select your Plan: <PLAN> 
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

  1. MODEL_DIRECTORY- Enter the folder name where your model and files are situated inside the workstation .

  2. BUCKET_NAME - Select the bucket where you want to store the model files

  3. AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API

  4. NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  5. DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  6. KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  7. PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES

  8. DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

Make sure MODEL_DIRECTORY you enter contains all the files related to model .

Depending on the size of model it will take time around 1 to 2 hours

PreviousZBL Command Line InterfaceNextDeploy