Notes on file transfer from Mark5 disk pack to external Raid array. ----------------------------------------- December 29, 2004. --------------------------------- Wolfgang has installed a gigabit ethernet card in the Mark5, and we have configured for high-speed data transfers. (See notes below for configuration) Instructions for doing a data transfer: Mark5 --> Raid ------------------------------------------------------- - in this case we are going from "mark5gb" to raid array on "fourier" 1. [on fourier] cd /export/raid/scratch/fghigo Net2file -r 8388608 -f testfile.data & 2. [on mark5gb] tstMark5A > net_protocol=tcp:8388608:2097152 > disk2net=connect:fourier > disk2net=on:: > disk2net? [get status ] 3. [on mark5gb after data transfer finishes] > disk2net=disconnect [this stops "Net2file"] --------------------- Instructions for Raid --> Mark5 ---------------------------------------- - Going from raid array on "fourier" to the Mark5 1. [on mark5gb:] tstMark5A > net_protocol=tcp:8388608:2097152 > net2disk=open:newscanname 2. [on fourier:] cd /export/raid/scratch/fghigo File2net -c mark5gb -f test2.data -s 8388608 & -- the whole file "test2.data" is transferred to the mark5, apparently starting at the current write pointer location. -- File2net terminates itself when the transfer is done. 3. [on mark5gb:] > net2disk=close =========================================================== Configuration notes: ------------------------------------------------------ you need to add to /etc/services on the machine on which Net2file runs the lines: m5drive 2620/tcp # Mark-5 Drives m5drive 2620/udp # Mark-5 Drives m5data 2630/tcp # Mark-5 Data m5data 2630/udp # Mark-5 Data ----------------------------- Hi Frank Probably the roundtrip time is negligible between your systems, but I found that it does not hurt to increase the congestion window. You must make sure that your default and maximum values are set to sensible values for high bandwidth transfers. If you set these values echo 1 > /proc/sys/net/ipv4/tcp_timestamps echo 1 > /proc/sys/net/ipv4/tcp_window_scaling echo 0 > /proc/sys/net/ipv4/tcp_sack echo 16777216 > /proc/sys/net/core/wmem_max echo 16777216 > /proc/sys/net/core/rmem_max echo "4096 4194304 16777216" > /proc/sys/net/ipv4/tcp_rmem echo "4096 4194304 16777216" > /proc/sys/net/ipv4/tcp_wmem you should be ok (on both machines). In this way the default socket buffer will be 4 MB, the maximum 16 MB. Then there is another buffer size to be set in the Mark5 program. Use the net_protocol command (in tstMark5A) net_protocol=tcp:8388608:2097152 (socket buffer is now 8MB, mark5 workbuffer 2 MB) before starting disk2net. Net2file has a socket buffer option Net2file -r 8388608 8MB is a bit large, but as I said, it can't hurt. Try this and see if things get any better. --Arpad