How to use the CLI

This document will guide you through the usage of Zeblok cli

CLI Usability and commands.

Show-URL

This command just outputs the APP URL and datalake Url configured in zeblok configure command

zeblok showUrl

Snapshot

This command is used to create snapshot of your workstation on which it creates a docker image whose tag is emailed to the user on which that tag can be used to create a new workstation with current configuration

zeblok snapshot
? Enter the name for docker file getting generated : <Docker Image Name>
? Select your DataLake Bucket to Deploy (Use arrow keys) <Minio Bucket Selection>

Brief

Snapshot command copy all the files from your notebook with the same structure as in current one with that it also creates requirments.txt with all the installed packages in your current notebook and installs it when you spawn the new workstation with that docker image to have same configuration as you have currently .

requirments.txt is generated automatically with pip list --format=freeze
Base image for workstation used is minimal-notebook:2023.09.20

Components

Docker Image Name- The image name you want the docker to have
Minio Bucket Selection - Select the bucket where you want to store all the data of your notebook

Model Serving

Zeblok's modelserve command allows you to serve machine learning models using different methods. Whether you have a model trained with OpenVINO, MLflow, BentoML, LLamaCpp, or VLLM, Zeblok provides a seamless way to deploy and serve your models.

zeblok modelserve --type <model_name>

Components

The modelserve command serves your machine learning model using the specified method.
Use the --type option followed by the model's name to specify the method for serving the model.

Options

-t, --type : Specify the serving method. Choose from openvino, mlflow, bentoml, llamacpp, vllm.

Usage

Serve with Specified Method:

Openvino

This command is used to serve model using openvino

zeblok modelserve --type openvino
? Enter the path for generated IR model (without / at the end): <IR_OUTPUT_PATH>
? Enter the IR output name : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

IR_OUTPUT_PATH - Folder containing the <IR_OUTPUT_NAME>.xml and the <IR_OUTPUT_NAME>.bin
IR_OUTPUT_NAME - IR output name which is same for both .xml and .bin file in the IR_OUTPUT_PATH
BUCKET_NAME - Select the bucket where you want to store the model files
AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API
NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

Ideal File Structure

For openvino model serving to work the output path must contain 2 files with same name with extensions *.xml and *.bin

Bentoml

Bentolml model serving serves a bentoml model served built using bentoml build command . It fetches the model with bentoml list --output json command and lists one which you want to serve .

zeblok modelserve --type bentoml
? Select your Model to Deploy: <MODEL_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

MODEL_NAME - Select the model built using the bentoml build command bentoml build
BUCKET_NAME - Select the bucket where you want to store the model files
AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API
NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

The model serving of bentoml needs bentoml installed on your workstation
If there is no model to select in first question then try running bentoml build in the model creation .

Mlflow

Mlflow model serving uses process to serve a model first by getting model into your workstation directory from the MLflow registiry and then serving it onto your workstation

Before utilizing the ML-Flow model serving please ensure the trained model is saved onto your Private ML-flow registory.

ML-Flow registory can be spawned as a Microservice i.e. the ML-flow Microservice within the Microcloud itself.

Please Make sure you already have a ML-Flow Microservice

zeblok modelserve --type mlflow
? Enter the mlflow url for model : <MLFLOW_TRACKING_URL>
? Enter the runId for model : <RUN_ID>
? Enter the rootPath for model in directory : <DOWNLOADED_MODEL_PATH>
? Enter the IR output name : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN>
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

MLFLOW_TRACKING_URL - Mlflow tracking URL where your model is stored to .
RUN_ID - Run id of the model you are willing to serve
DOWNLOADED_MODEL_PATH- Enter folder name where your model is stored .
IR_OUTPUT_NAME- Enter any name that you want to put for model .
AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API
NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

The model serving of mlflow needs mlflow installed on your workstation

LLAMA

This command serves LLAMA model with CPP serving .

zeblok modelserve --type llamacpp
? Enter the path for generated IR model (without / at the end): <IR_OUTPUT_PATH>
? Enter the IR output name with its extention : <IR_OUTPUT_NAME>
? Select your DataLake Bucket to Deploy : <BUCKET_NAME>
? Do you want to auto-deploy? : <AUTO_DEPLOY>
? Select your namespace: <NAMESPACE>
? Select your DataCenter: <DATACENTER>
? Select your Kiosk: <KISOK>
? Select your Plan: <PLAN> 
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

IR_OUTPUT_PATH- Enter the path where your model and requirements.txt file is situated .
IR_OUTPUT_NAME - Enter the model name with its extension whether its .gguml or .gguf
BUCKET_NAME - Select the bucket where you want to store the model files
AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API
NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

IR_OUTPUT_PATH must contain the model and requirements.txt file to work
Make sure you enter IR_OUTPUT_NAME with its extension .
Depending on the size of model it will take time around 2 to 3 hours

VLLM

This command serves model with Vllm model serving .

zeblok modelserve --type vllm 
? Enter the directory name containing your model : <MODEL_DIRECTORY> 
? Select your DataLake Bucket to Deploy : <BUCKET_NAME> 
? Do you want to auto-deploy? : <AUTO_DEPLOY> 
? Select your namespace: <NAMESPACE> 
? Select your DataCenter: <DATACENTER> 
? Select your Kiosk: <KISOK> 
? Select your Plan: <PLAN> 
? Name of your Deployment: <DEPLOYMENT_NAME>

Components

MODEL_DIRECTORY- Enter the folder name where your model and files are situated inside the workstation .
BUCKET_NAME - Select the bucket where you want to store the model files
AUTO_DEPLOY - This is Boolean Input which defines if you want to deploy the model directly to AI-API
NAMESPACE - Namespace where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DATACENTER - Datacenter where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
KISOK - Kisoks where the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
PLAN - Plan in which the ai api to be deployed and asked if AUTO_DEPLOY is marked as YES
DEPLOYMENT_NAME - Name of deployment used to deploy AI-API and asked if AUTO_DEPLOY is marked as YES

Make sure MODEL_DIRECTORY you enter contains all the files related to model .
Depending on the size of model it will take time around 1 to 2 hours

Serve without Specified Method:

If you run the modelserve command without specifying the method, it will prompt you to choose a serving method interactively

zeblok modelserve
? Choose the serving method: (Use arrow Keys)
> openvino
  mlflow
  bentoml
  llamacpp
  vllm

It will prompt: "Choose the serving method:"
Use arrow keys to select the desired method.
Press Enter to confirm your selection.

If the type option is provided, the program serves the model using the specified method.
if no type is provided,the program prompts the user to choose a serving method

PreviousZBL Command Line Interface NextDeploy