Utilizing ZBL object Store

This page provide details about modifying you docker images of micrservices which are to be onboarded onto ZBL AI-Microcloud in order to utilize and interact with ZBL object storage through them.

Ai-MicroCloud has a feature called Datasets. This feature uses respective object storage technology provided by the cloud providers in the backend. Users can upload their data and use it for development and production. Here is the link https://computationaldocs.zeblok.com/info/data-and-metrics/datalake/datasets of how to use the dataset feature using UI. Also, you can check the video at the end of the link on mounting the dataset with Ai-Workstation.

To mount the dataset with Ai-Microservices, follow the steps below:

User needs to use the following "shell script" inside the microservice to access the datasets within Ai-MicroCloud. Please copy the script to the machine where you will create your docker image.

Create a Dockerfile User needs to use the given environment variables, install required packages and include the above script inside your existing dockerfile.

Below is the example of Dockerfile:

FROM ubuntu:22.04 

WORKDIR /app 

#Environment variables  

ENV BUCKET_TYPE="objectstorage" 

ENV S3_BUCKET_NAME="my-bucketname" 

ENV NB_USER="myusername" 

ENV datalake_url="https://datalake.url" 

ENV datalake_secret="myecrets" 

ENV platform_url="https://myurl" 

ENV ATTACH_BUCKET="yes" 

ENV DEFAULT_BUCKET="mydefaultbucket" 

ENV ACCOUNT_NAME="myaccountname" 

ENV ACCOUNT_KEY="key" 

ENV AWS_SECRET_KEY="myaccesskey" 

ENV AWS_ACCESS_KEY_ID="mysubcribeid" 

RUN mkdir bucket 

RUN mkdir static 

#Copy the zeblok_ds_mount.sh script  

COPY zeblok_ds_mount.sh /app 

#Install the required packages 

RUN apt update && apt-get install -y s3fs build-essential libfuse-dev libcurl4-openssl-dev libxml2-dev pkg-config libssl-dev mime-support automake libtool lsb-release fuse3 

RUN wget https://github.com/Azure/azure-storage-fuse/releases/download/blobfuse2-2.2.0/blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb && dpkg -i blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb && apt-get -f install -y && rm -rf blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb 

#Run the script on starting the container 

ENTRYPOINT ["sh", "/app/zeblok_ds_mount.sh"]

All the env values will be picked up from backend. User doesn’t have to add env variables while spawning the microservice

Check whether dataset is mounted "Once you onboard the microservice and spawn it, you should see the dataset on path /app/bucket"

Before the last step once the image is created please follow the link to onboard your newly created microservice onto the ZBL AI-Microcloud

PreviousMicroServices NextResource Manager

FROM ubuntu:22.04 WORKDIR /app #Environment variables ENV BUCKET_TYPE="objectstorage" ENV S3_BUCKET_NAME="my-bucketname" ENV NB_USER="myusername" ENV datalake_url="https://datalake.url" ENV datalake_secret="myecrets" ENV platform_url="https://myurl" ENV ATTACH_BUCKET="yes" ENV DEFAULT_BUCKET="mydefaultbucket" ENV ACCOUNT_NAME="myaccountname" ENV ACCOUNT_KEY="key" ENV AWS_SECRET_KEY="myaccesskey" ENV AWS_ACCESS_KEY_ID="mysubcribeid" RUN mkdir bucket RUN mkdir static #Copy the zeblok_ds_mount.sh script COPY zeblok_ds_mount.sh /app #Install the required packages RUN apt update && apt-get install -y s3fs build-essential libfuse-dev libcurl4-openssl-dev libxml2-dev pkg-config libssl-dev mime-support automake libtool lsb-release fuse3 RUN wget https://github.com/Azure/azure-storage-fuse/releases/download/blobfuse2-2.2.0/blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb && dpkg -i blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb && apt-get -f install -y && rm -rf blobfuse2-2.2.0-Ubuntu-22.04.x86_64.deb #Run the script on starting the container ENTRYPOINT ["sh", "/app/zeblok_ds_mount.sh"]