MP3 - Simple Distributed File System

Overview

In this MP, Our team implemented a unidirectional, ring-based, ping-ack style Simple Distributed File System with Golang, gRPC, and Protobuf. We use gRPC as our communication medium because it is extremely efficient in passing data and it has native support for Golang. Our system consists of two parts: a DNS server, and multiple host servers. The DNS server is responsible for maintaining the leader information, and each host server is the server storing files distributedly.

For more technical details and how we implement it, please refer to the report in the same repository.

How to Compile & Run

To compile the project, you need to have Golang installed on your machine. You can download Golang from here. Once you have Golang installed, you need to download dependency gRPC and Protobuf. You can do this by running the following command in your terminal:

go get -u google.golang.org/grpc
go get -u github.com/golang/protobuf/protoc-gen-go

After you have all the dependencies installed, you can compile the project by running the following command in your terminal:

make all

This will generate the executable files in the bin folder. You can run the executable file by running the following command in your terminal:

cd bin
./dns

and

cd bin
./sdfs

How to Use

You need to run only one dns executable file. You can run multiple sdfs executable files. Each sdfs executable file will be a host server. Here is a list of commands you can use in the sdfs executable file:

join            # Join the ring
leave           # Leave the ring
list_mem        # List all members in the ring
clear           # clear all content from log file
stat            # print statistics of Bps read/write, #pings, #failures, system elapsed time
low_droprate    # Set the drop rate to low (0.03)
mid_droprate    # Set the drop rate to medium (0.3)
high_droprate   # Set the drop rate to high (0.9)

Here are more commands for the sdfs executable file:

get <sdfsfilename> <local_filename>  # Get a file from the SDFS
put <local_filename> <sdfsfilename>  # Put a local file to the SDFS
delete <sdfsfilename>                # Delete a file from the SDFS
ls <sdfsfilename>                    # List the servers storing the file
store                                # List all files stored in the current server
get-versions <sdfsfilename> <num versions> <localfilename>        # Retrieve the last num versions of the file