| VIVADO_HLS_ROOT_DIR | <path> | Path to Vivado HLS directory, e.g. /opt/Xilinx/Vivado/2019.1 |
3. Build HLS IP cores and install them into IP repository
```
make installip
```
For an example project including the TCP/IP stack or the RoCEv2 stack with DMA to host memory checkout our Distributed Accelerator OS [DavOS](https://github.com/fpgasystems/davos).
All interfaces are using the AXI4-Stream protocol. For AXI4-Streams carrying network/data packets, we use the following definition in HLS:
```
template <int D>
struct net_axis
{
ap_uint<D> data;
ap_uint<D/8> keep;
ap_uint<1> last;
};
```
Make sure that Vivado and Vivado HLS are in your PATH. Use version 2018.1
<aname="tcp-interface"></a>
### TCP/IP
Navigate to the _hls_ directory:
#### Open Connection
To open a connection the destination IP address and TCP port have to provided through the `s_axis_open_conn_req` interface. The TCP stack provides an answer to this request through the `m_axis_open_conn_rsp` interface which provides the sessionID and a boolean indicating if the connection was openend successfully.
cd hls
Interface definition in HLS:
```
struct ipTuple
{
ap_uint<32> ip_address;
ap_uint<16> ip_port;
};
struct openStatus
{
ap_uint<16> sessionID;
bool success;
};
void toe(...
hls::stream<ipTuple>& openConnReq,
hls::stream<openStatus>& openConnRsp,
...);
```
Execute the script generate the HLS IP cores for your board:
./generate_hls vc709
For the VCU118 run
./generate_hls vcu118
#### Close Connection
To close a connection the sessionID has to be provided to the `s_axis_close_conn_req` interface. The TCP/IP stack does not provide a notification upon completion of this request, however it is guranteeed that the connection is closed eventually.
Interface definition in HLS:
```
hls::stream<ap_uint<16> >& closeConnReq,
Navigate to the _projects_ directory:
```
cd ../projects
#### Open a TCP port to listen on
To open a port to listen on (e.g. as a server), the port number has to be provided to `s_axis_listen_port_req`. The port number has to be in range of active ports: 0 - 32767. The TCP stack will respond through the `m_axis_listen_port_rsp` interface indicating if the port was set to the listen state succesfully.
Create the example project for your board.
Interface definition in HLS:
```
hls::stream<ap_uint<16> >& listenPortReq,
hls::stream<bool>& listenPortRsp,
```
For the Xilinx VC709:
#### Receiving notifications from the TCP stack
The application using the TCP stack can receive notifications through the `m_axis_notification` interface. The notifications either indicate that new data is available or that a connection was closed.
vivado -mode batch -source create_vc709_proj.tcl
Interface definition in HLS:
```
struct appNotification
{
ap_uint<16> sessionID;
ap_uint<16> length;
ap_uint<32> ipAddress;
ap_uint<16> dstPort;
bool closed;
};
hls::stream<appNotification>& notification,
```
For the Alpha DATA ADM-PCIE-7V3:
#### Receiving data
If data is available on a TCP/IP session, i.e. a notification was received. Then this data can be requested through the `s_axis_rx_data_req` interface. The data as well as the sessionID are then received through the `m_axis_rx_data_rsp_metadata` and `m_axis_rx_data_rsp` interface.
vivado -mode batch -source reate_adm7v3_proj.tcl
Interface definition in HLS:
```
struct appReadRequest
{
ap_uint<16> sessionID;
ap_uint<16> length;
};
hls::stream<appReadRequest>& rxDataReq,
hls::stream<ap_uint<16> >& rxDataRspMeta,
hls::stream<net_axis<WIDTH> >& rxDataRsp,
```
For the Xilinx VCU118:
Waveform of receiving a (data) notification, requesting data, and receiving the data:
After the previous command executed, a Vivado project will be created and the Vivado GUI is started.
#### Transmitting data
When an application wants to transmit data on a TCP connection, it first has to check if enough buffer space is available. This check/request is done through the `s_axis_tx_data_req_metadata` interface. If the response through the `m_axis_tx_data_rsp` interface from the TCP stack is positive. The application can send the data through the `s_axis_tx_data_req` interface. If the response from the TCP stack is negative the application can retry by sending another request on the `s_axis_tx_data_req_metadata` interface.
Click "Generate Bitstream" to generate a bitstream for the FPGA.
Interface definition in HLS:
```
struct appTxMeta
{
ap_uint<16> sessionID;
ap_uint<16> length;
};
struct appTxRsp
{
ap_uint<16> sessionID;
ap_uint<16> length;
ap_uint<30> remaining_space;
ap_uint<2> error;
};
hls::stream<appTxMeta>& txDataReqMeta,
hls::stream<appTxRsp>& txDataRsp,
hls::stream<net_axis<WIDTH> >& txDataReq,
```
## Testing the example project
Waveform of requesting a data transmit and transmitting the data.
The default configuration deploys a TCP echo server and a UDP iperf client. The default IP address the board is 10.1.212.209. Make sure the testing machine conencted to the FPGA board is in the same subnet 10.1.212.*
As an intial connectivity test ping the FPGA board by running
<aname="roce-interface"></a>
### RoCE (RDMA over Converged Ethernet)
ping 10.1.212.209
#### Load Queue Pair (QP)
Before any RDMA operations can be executed the Queue Pairs have to established out-of-band (e.g. over TCP/IP) by the hosts. The host can the load the QP into the RoCE stack through the `s_axis_qp_interface` and `s_axis_qp_conn_interface` interface.
After reprogramming the FPGA the first ping message is lost due to a missing ARP entry in the ARP table. However, the FPGA should reply to all following ping messages.
RDMA commands can be issued to RoCE stack through the `s_axis_tx_meta` interface. In case the commands transmits data. This data can be either originate from the host memory as specified by the `local_vaddr` or can originate from the application on the FPGA. In the latter case the `local_vaddr` is set to 0 and the data is provided through the `s_axis_tx_data` interface.
For the TCP and UDP iperf test, see [here](http://github.com/dsidler/fpga-network-stack/wiki/iPerf-Benchmark).
## Configuration
<aname="interfaces"></a>
## Benchmarks
(Coming soon)
Coming soon
<aname="publications"></a>
## Publications
## Publications
- D. Sidler, G. Alonso, M. Blott, K. Karras et al., *Scalable 10Gbps
- D. Sidler, G. Alonso, M. Blott, K. Karras et al., *Scalable 10Gbps
TCP/IP Stack Architecture for Reconfigurable Hardware,* in FCCM’15, [Paper](http://davidsidler.ch/files/fccm2015-tcpip.pdf), [Slides](http://fccm.org/2015/pdfs/M2_P1.pdf)
TCP/IP Stack Architecture for Reconfigurable Hardware,* in FCCM’15, [Paper](http://davidsidler.ch/files/fccm2015-tcpip.pdf), [Slides](http://fccm.org/2015/pdfs/M2_P1.pdf)
- D. Sidler, Z. Istvan, G. Alonso, *Low-Latency TCP/IP Stack for Data Center Applications,* in FPL'16, [Paper](http://davidsidler.ch/files/fpl16-lowlatencytcpip.pdf)
- D. Sidler, Z. Istvan, G. Alonso, *Low-Latency TCP/IP Stack for Data Center Applications,* in FPL'16, [Paper](http://davidsidler.ch/files/fpl16-lowlatencytcpip.pdf)
<aname="citation"></a>
## Citation
## Citation
If you use the TCP/IP stack in your project please cite one of the following papers and/or link to the github project:
If you use the TCP/IP stack in your project please cite one of the following papers and/or link to the github project:
```
```
...
@@ -95,6 +310,19 @@ If you use the TCP/IP stack in your project please cite one of the following pap
...
@@ -95,6 +310,19 @@ If you use the TCP/IP stack in your project please cite one of the following pap
booktitle={FPL'16},
booktitle={FPL'16},
title={{Low-Latency TCP/IP Stack for Data Center Applications}},
title={{Low-Latency TCP/IP Stack for Data Center Applications}},
}
}
@PHDTHESIS{sidler2019innetworkdataprocessing,
author = {Sidler, David},
publisher = {ETH Zurich},
year = {2019-09},
copyright = {In Copyright - Non-Commercial Use Permitted},
title = {In-Network Data Processing using FPGAs},
}
```
```
For more information please visit the [wiki](http://github.com/dsidler/fpga-network-stack/wiki)
<aname="contributors"></a>
## Contributors
-[David Sidler](http://github.com/dsidler), [Systems Group](http://systems.ethz.ch), ETH Zurich
-[Monica Chiosa](http://github.com/chipet), [Systems Group](http://systems.ethz.ch), ETH Zurich
-[Mario Ruiz](https://github.com/mariodruiz), HPCN Group of UAM, Spain
-[Kimon Karras](http://github.com/kimonk), former Researcher at Xilinx Research, Dublin
