Tutorials

Ai- DataLake Tutorials and Commands

Overview

Zeblok Datalake is a high-performance data-store, compatible with the S3 protocol, intended for secure high speed data transfer. As an object store, Zeblok Datalake allows users to read and write large amounts of unstructured data such as photos, videos, log files, spreadsheets, CSV files, etc. Datalake is cloud native and supports GPU-optimized queries through the addition of BlazingDB.

With a robust and simple online interface, API CRUD access, in-built Zeblok CLI magic commands, and Python integrations, users can instantly run complex AI related queries for a variety of datasets, visualize the results, and share/revoke reports with clients.

Zeblok CLI

Get the Latest Version Zeblok CLI using Below Command

Code

npm i -g  zbl-cli

Output

ERROR! Session/line number was not unique in database. History logging moved to new session 6
/opt/conda/bin/zeblok -> /opt/conda/lib/node_modules/zbl-cli/index.jserial remove 10yan/.npm/_lock
+ [email protected]
updated 1 package in 4.359s

Configure the Zeblok CLI

Figure: CLI Configure

Set DatalakeUrl

zeblok setUrl http://18.189.16.253:9000

Login using Zeblok CLI

To login using Zeblok CLI open terminal from Jupyterlab environment and type the below command.

Note:

SecretKey and Username are avaialable on Zeblok Computational Dashboard "Manage your Zeblok Accounts" Page

Code

zeblok login <username> <secret key>

Output

  _____         _       _           _        ____            _             _           _           
 |__  /   ___  | |__   | |   ___   | | __   |  _ \    __ _  | |_    __ _  | |   __ _  | | __   ___ 
   / /   / _ \ | '_ \  | |  / _ \  | |/ /   | | | |  / _` | | __|  / _` | | |  / _` | | |/ /  / _ \
  / /_  |  __/ | |_) | | | | (_) | |   <    | |_| | | (_| | | |_  | (_| | | | | (_| | |   <  |  __/
 /____|  \___| |_.__/  |_|  \___/  |_|\_\   |____/   \__,_|  \__|  \__,_| |_|  \__,_| |_|\_\  \___|
                                                                                                   


Added `zb` successfully.

List all the Files and Directory on Datalake

Code

zbl tree -f zb/datalake/<username>/

Output

zb/datalake/akash/
├─ FL_insurance_sample.csv
├─ ZeblokReadme.ipynb
├─ algebra.log
├─ main.go
├─ sales.csv
├─ sample_taxi.csv
└─ docs
   ├─ MAINTAINERS.md
   ├─ minio-admin-complete-guide.md
   ├─ minio-client-complete-guide.md
   ├─ minio-client-configuration-files.md
   └─ zh_CN
      ├─ CONTRIBUTING.md
      ├─ minio-client-complete-guide.md
      └─ minio-client-configuration-files.md

Copy file from DataLake to local Notebook Directory

zbl cp zb/datalake/<username>/<filepathOnDatalake> <filepathOnNotebookStorage>

Example is given below:

Code

zbl cp zb/datalake/akash/main.go main.go

Output

...h/main.go:  979 B / 979 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 29.49 KiB/s 0s

Copy file from Local Notebook Directory to Datalake

zbl cp <localfilePath> zb/datalake/<username>/<datalakeFilePath>

Example:

Code

zbl cp ZeblokReadme.ipynb zb/datalake/akash/

Output

...dme.ipynb:  54.33 KiB / 54.33 KiB ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 1.29 MiB/s 0s