IP-Over-SCSI information page

About

The purpose of IP Over SCSI is to connect two or more computers with SCSI interfaces using the IP protocol. This was first suggested by Ben Elliston on his RFC and (maybe first) implemented by Randy Scott in his senior design project.

The advantage of using SCSI interfaces to encapsulate IP is the large bandwidth possible using SCSI interfaces, that can be up to 80 MByte/s with SCSI-2 and 160 MByte/s with SCSI-3 (Note: with fast ethernet only 10.7 MByte/s can be achieved). It can be used in local networks or Beowulf clusters where a high throughput is necessary for large messages (and a small latency for small messages in e.g. reduction loops...) in MPI applications (see last part of this page).

Status

Today the project isn't very well coordinated and there is not much information on the Web. Also there isn't a central site about the project and its development.

There is a patch to the 2.2.14 Linux kernel that was developed by Marc Schaefer.

By Marc Schaefer:

Basically, the status is:

I have tested the network -> SCSI code (it's different from the original code, i.e. it should work now :-)
I have not yet tested the SCSI -> network code
I have ported everything to 2.2.x

Mailing List

To subscribe to the mailing list send an email to linux-ip-scsi-reques@alphanet.ch with "subscribe" in the body.

Tested and untested SCSI interfaces (by Marc Schaefer):

qlogic 10xx, 21xx boards should be compatible (although not yet implemented), since they support the target and initiator mode with the feral SCSI drivers on Linux (http://www.feral.com/).

aha154x and also an AMD chip are supposedly supported in the standard patch. However, I don't really see the point, since those are quite slow SCSI cards, and even 100 Mb/s Ethernet is going to be faster in most cases.

Cables (by Marc Schaefer)

You don't need special cables. A normal host-to-disk cable will suffice. The only problem is to ensure that termination is enabled at both ends. This is usually done in the BIOS configuration.

Links

RFC2143 - original proposal by Ben Elliston
Implements IP over SCSI - page of the unofficial patch
Senior Design Project - page of Randy Scott's project
SCSI-2 Spec - Introduction
SCSI FAQ - Editor Gary Field
X3 Standards

Interest of UALG Vision Lab: MPI-over-SCSI

The Ping-Pong Beowulf (two dual-PIII nodes at 450 MHz with fast ethernet between them) outperforms the SGI Origin 200, hence in the future we will work more with Beowulfs. The throughput between the two nodes is 10.7 Mbyte/s (86 Mbit/s of the 100 max); between two CPUs in one node it is only a factor of 2.66 more: 28 Mbyte/s. These throughputs have been measured for messages from 10,000 REALs to 1,000,000 REALs (one REAL is 4 bytes). The latencies for small messages are 0.35 and 0.15 ms.

The measured factor of 2.66 implies that if the ethernet wire is replaced with a SCSI cable, and the TCP/IP protocol is still used, the throughput between nodes will increase from 10.7 to 28 Mbyte/s, but NOT to the SCSI throughput...

Apart from buying gigabit ethernet (Myrinet), which doubles the price of a cluster, we want to develop a cheap but fast alternative: MPI-over-SCSI in which the MPI and SCSI protocols are directly linked and in which MPI uses a routing table to get the best communication performance. More about this follows later.
Hans du Buf

Back to the Vision Laboratory

Luis Sismeiro, 27 March 2000