CS425-MP3-SM: SDFS

Usage

Run Server

Similar to failure detector, we only need to run a single script to start both failure detecor and SDFS srever (in different ports).

$ python3 sdfs.py

Then it will continuously print membership list for the node with default status LEAVED. To join the group, use the following interface:

You are able to input directly to the terminal as command including

• join: join the node to group
• leave: leave the group
• ml: print current node's membership list

SDFS Commands

Except for commands from failure detector (join, leave and ml), the following commands are used to control SDFS:

• get [sdfs_file_name] [local_file_name]: get a (latest version) file from SDFS
• get-versions [sdfs_file_name] [num-versions] [local_file_name]: get versions file from SDFS
• put [local_file_name] [sdfs_file_name]: put a local to SDFS
• delete [sdfs_file_name]: delete a file in SDFS
• ls [sdfs_file_name]: list all replicas for a file stored in SDFS
• store: list all files store in SDFS and their replicas
• lives: check all current available replicas

Steps

• ssh into each vm and cd to the folder
• run the select image script to get 'amount' of images

python3 select_image.py --src cat_dog_testing_set --target_path animal_images --amount [amount] --job animal
python3 select_image.py --src mnist_testing --target_path number_images --amount [amount] --job number

• copy the number.txt and animal.txt files generated in the last step to the client machine (running on the client folder)

sshpass -p 'Yyyy19491001' scp 'fa22-cs425-3201.cs.illinois.edu':/public/mp4-other-version/animal.txt animal.txt
sshpass -p 'Yyyy19491001' scp 'fa22-cs425-3201.cs.illinois.edu':/public/mp4-other-version/number.txt number.txt

• run the 'send_images folder' script to send those images to 'images' folder on each vm

python3 send_images.py --image_folder_path animal_images --last_mid 10
python3 send_images.py --image_folder_path number_images --last_mid 10

• run the coordinator and the standby coordinator on VM1 and VM2

python3 coordinator.py -vm1
python3 coordinator_standby.py -vm2

• run the worker on each VM

python3 worker_local.py

• join the worker on each VM

join

• run 'putfolder amount images' on each VM

putfolder 1000 images

• run the 'job_type file_list.txt' command in the client side(start infer)

animal animal.txt
number number.txt