# How to use the CLI
## CLI Usability and commands.
### Show-URL
This command just outputs the `APP URL` and `datalake Url` configured in zeblok configure command
```
zeblok showUrl
```
### Snapshot
This command is used to create snapshot of your workstation on which it creates a docker image whose tag is emailed to the user on which that tag can be used to create a new workstation with current configuration
```
zeblok snapshot
? Enter the name for docker file getting generated :
? Select your DataLake Bucket to Deploy (Use arrow keys)
```
#### Brief
Snapshot command copy all the files from your notebook with the same structure as in current one with that it also creates requirments.txt with all the installed packages in your current notebook and installs it when you spawn the new workstation with that docker image to have same configuration as you have currently .
> requirments.txt is generated automatically with `pip list --format=freeze`
>
> Base image for workstation used is `minimal-notebook:2023.09.20`
#### Components
1. `Docker Image Name`- The image name you want the docker to have
2. `Minio Bucket Selection` - Select the bucket where you want to store all the data of your notebook
### Model Serving
Zeblok's `modelserve` command allows you to serve machine learning models using different methods. Whether you have a model trained with OpenVINO, MLflow, BentoML, LLamaCpp, or VLLM, Zeblok provides a seamless way to deploy and serve your models.
```
zeblok modelserve --type
```
#### Components
* The modelserve command serves your machine learning model using the specified method.
* Use the --type option followed by the model's name to specify the method for serving the model.
#### Options
* -t, --type : Specify the serving method. Choose from openvino, mlflow, bentoml, llamacpp, vllm.
## Usage
### Serve with Specified Method:
### Openvino
This command is used to serve model using openvino
```
zeblok modelserve --type openvino
? Enter the path for generated IR model (without / at the end):
? Enter the IR output name :
? Select your DataLake Bucket to Deploy :
? Do you want to auto-deploy? :
? Select your namespace:
? Select your DataCenter:
? Select your Kiosk:
? Select your Plan:
? Name of your Deployment:
```
#### Components
1. `IR_OUTPUT_PATH` - Folder containing the \.xml and the \.bin
2. `IR_OUTPUT_NAME` - IR output name which is same for both .xml and .bin file in the `IR_OUTPUT_PATH`
3. `BUCKET_NAME` - Select the bucket where you want to store the model files
4. `AUTO_DEPLOY` - This is Boolean Input which defines if you want to deploy the model directly to AI-API
5. `NAMESPACE` - Namespace where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
6. `DATACENTER` - Datacenter where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
7. `KISOK` - Kisoks where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
8. `PLAN` - Plan in which the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
9. `DEPLOYMENT_NAME` - Name of deployment used to deploy AI-API and asked if `AUTO_DEPLOY` is marked as `YES`
#### Ideal File Structure
> For openvino model serving to work the output path must contain 2 files with same name with extensions \*.xml and \*.bin
### Bentoml
Bentolml model serving serves a bentoml model served built using `bentoml build` command . It fetches the model with `bentoml list --output json` command and lists one which you want to serve .
```
zeblok modelserve --type bentoml
? Select your Model to Deploy:
? Select your DataLake Bucket to Deploy :
? Do you want to auto-deploy? :
? Select your namespace:
? Select your DataCenter:
? Select your Kiosk:
? Select your Plan:
? Name of your Deployment:
```
#### Components
1. `MODEL_NAME` - Select the model built using the bentoml build command `bentoml build`
2. `BUCKET_NAME` - Select the bucket where you want to store the model files
3. `AUTO_DEPLOY` - This is Boolean Input which defines if you want to deploy the model directly to AI-API
4. `NAMESPACE` - Namespace where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
5. `DATACENTER` - Datacenter where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
6. `KISOK` - Kisoks where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
7. `PLAN` - Plan in which the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
8. `DEPLOYMENT_NAME` - Name of deployment used to deploy AI-API and asked if `AUTO_DEPLOY` is marked as `YES`
> The model serving of bentoml needs bentoml installed on your workstation
>
> If there is no model to select in first question then try running `bentoml build` in the model creation .
### Mlflow
Mlflow model serving uses process to serve a model first by getting model into your workstation directory from the MLflow registiry and then serving it onto your workstation
{% hint style="warning" %}
Before utilizing the ML-Flow model serving please ensure the trained model is saved onto your Private ML-flow registory.
ML-Flow registory can be spawned as a Microservice i.e. the ML-flow Microservice within the Microcloud itself.
Please Make sure you already have a ML-Flow Microservice
{% endhint %}
```
zeblok modelserve --type mlflow
? Enter the mlflow url for model :
? Enter the runId for model :
? Enter the rootPath for model in directory :
? Enter the IR output name :
? Select your DataLake Bucket to Deploy :
? Do you want to auto-deploy? :
? Select your namespace:
? Select your DataCenter:
? Select your Kiosk:
? Select your Plan:
? Name of your Deployment:
```
**Components**
1. `MLFLOW_TRACKING_URL` - Mlflow tracking URL where your model is stored to .
2. `RUN_ID` - Run id of the model you are willing to serve
3. `DOWNLOADED_MODEL_PATH`- Enter folder name where your model is stored .
4. `IR_OUTPUT_NAME`- Enter any name that you want to put for model .
5. `AUTO_DEPLOY` - This is Boolean Input which defines if you want to deploy the model directly to AI-API
6. `NAMESPACE` - Namespace where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
7. `DATACENTER` - Datacenter where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
8. `KISOK` - Kisoks where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
9. `PLAN` - Plan in which the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
10. `DEPLOYMENT_NAME` - Name of deployment used to deploy AI-API and asked if `AUTO_DEPLOY` is marked as `YES`
{% hint style="info" %}
The model serving of mlflow needs mlflow installed on your workstation
{% endhint %}
### LLAMA
This command serves LLAMA model with CPP serving .
```
zeblok modelserve --type llamacpp
? Enter the path for generated IR model (without / at the end):
? Enter the IR output name with its extention :
? Select your DataLake Bucket to Deploy :
? Do you want to auto-deploy? :
? Select your namespace:
? Select your DataCenter:
? Select your Kiosk:
? Select your Plan:
? Name of your Deployment:
```
#### Components
1. `IR_OUTPUT_PATH`- Enter the path where your model and `requirements.txt` file is situated .
2. `IR_OUTPUT_NAME` - Enter the model name with its extension whether its .gguml or .gguf
3. `BUCKET_NAME` - Select the bucket where you want to store the model files
4. `AUTO_DEPLOY` - This is Boolean Input which defines if you want to deploy the model directly to AI-API
5. `NAMESPACE` - Namespace where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
6. `DATACENTER` - Datacenter where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
7. `KISOK` - Kisoks where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
8. `PLAN` - Plan in which the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
9. `DEPLOYMENT_NAME` - Name of deployment used to deploy AI-API and asked if `AUTO_DEPLOY` is marked as `YES`
> `IR_OUTPUT_PATH` must contain the model and `requirements.txt` file to work
>
> Make sure you enter `IR_OUTPUT_NAME` with its extension .
>
> Depending on the size of model it will take time around 2 to 3 hours
### VLLM
This command serves model with Vllm model serving .
```
zeblok modelserve --type vllm
? Enter the directory name containing your model :
? Select your DataLake Bucket to Deploy :
? Do you want to auto-deploy? :
? Select your namespace:
? Select your DataCenter:
? Select your Kiosk:
? Select your Plan:
? Name of your Deployment:
```
#### Components
1. `MODEL_DIRECTORY`- Enter the folder name where your model and files are situated inside the workstation .
2. `BUCKET_NAME` - Select the bucket where you want to store the model files
3. `AUTO_DEPLOY` - This is Boolean Input which defines if you want to deploy the model directly to AI-API
4. `NAMESPACE` - Namespace where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
5. `DATACENTER` - Datacenter where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
6. `KISOK` - Kisoks where the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
7. `PLAN` - Plan in which the ai api to be deployed and asked if `AUTO_DEPLOY` is marked as `YES`
8. `DEPLOYMENT_NAME` - Name of deployment used to deploy AI-API and asked if `AUTO_DEPLOY` is marked as `YES`
> Make sure `MODEL_DIRECTORY` you enter contains all the files related to model .
>
> Depending on the size of model it will take time around 1 to 2 hours
### Serve without Specified Method:
If you run the modelserve command without specifying the method, it will prompt you to choose a serving method interactively
```
zeblok modelserve
? Choose the serving method: (Use arrow Keys)
> openvino
mlflow
bentoml
llamacpp
vllm
```
* It will prompt: "Choose the serving method:"
* Use arrow keys to select the desired method.
* Press Enter to confirm your selection.
{% hint style="success" %}
* If the type option is provided, the program serves the model using the specified method.
* if no type is provided,the program prompts the user to choose a serving method
{% endhint %}