Experimental Results: --------------------- The tables below show summarize the throughput measurements as previously. There are two additional columns, #sIO and #pIO which is the total count of the number of I/Os for the server and proxy node respectively. The number of I/Os for the client node was not taken since it is merely the transfer size (48 Megabytes) divided by the buffer size. The data was gathered by recording the return value of the read or write function call for every I/O performed by the node. This data not only provides the count of I/Os but also shows size grouping. However for brevity, only the I/O count is tabulated. For some table entries these columns are blank, notably for the cases for small buffer sizes. This is due to the fact that the limitation of loop counters was encountered when attempting to process the large amount of data from these experiments. Client Server Elapsed_Time Thruput #sIO ------------------------------------------- 4096 4096 1.626255 30376.90 12288 2048 4096 2.296014 21504.70 16350 1024 4096 3.362815 14649.99 19535 512 4096 4.724369 10415.61 21530 256 4096 7.737024 6353.21 22995 128 4096 13.432970 3660.81 23651 64 4096 23.333790 2107.54 24045 48 4096 29.907310 1644.69 32 4096 48.474370 1014.79 Client Server Elapsed_Time Thruput #sIO ------------------------------------------- 4096 4096 1.598318 30766.36 12288 4096 2048 1.656203 29731.28 24576 4096 1024 1.746479 29052.99 49152 4096 512 2.388863 20971.26 98304 4096 256 4.210448 11737.68 196608 4096 128 6.976955 7074.61 393216 4096 64 11.504700 4280.05 786432 4096 48 16.336830 3026.62 4096 32 21.170410 2324.10 Client Server Elapsed_Time Thruput #sIO ------------------------------------------- 4096 4096 1.609279 30551.55 12288 2048 2048 2.318683 21264.76 24576 1024 1024 3.071749 16087.61 49152 512 512 4.713824 10554.56 98304 256 256 8.343012 5967.27 196608 128 128 14.182600 3542.73 393216 64 64 28.471890 1772.20 786432 48 48 49.094820 1015.50 32 32 128.432900 383.94 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 2.999346 16404.17 0.257 2.124 12288 12288 2048 4096 4096 3.940273 12483.37 0.210 2.370 13104 13104 1024 4096 4096 4.980775 9882.98 0.282 3.182 14936 14876 512 4096 4096 5.724129 8752.63 0.436 3.666 16726 16421 256 4096 4096 8.380045 5875.21 0.602 5.439 22161 22146 128 4096 4096 13.684760 3592.98 0.654 6.818 22161 22146 64 4096 4096 24.657960 1994.04 0.677 9.246 24019 24019 48 4096 4096 31.449610 1563.70 0.643 10.581 32 4096 4096 46.385580 1059.90 0.757 13.969 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 2.951419 16656.16 0.269 2.228 12288 12288 4096 4096 2048 3.251854 15174.45 0.199 2.467 12288 24576 4096 4096 1024 3.257454 15189.09 0.213 2.150 12288 49152 4096 4096 512 3.927568 12764.86 0.289 2.012 12288 98304 4096 4096 256 4.966228 10138.71 0.274 1.774 12288 196608 4096 4096 128 7.164429 6950.23 0.222 1.729 12288 393216 4096 4096 64 12.171580 4068.01 0.225 1.630 12288 4096 4096 48 15.720110 3132.97 0.266 1.812 12288 4096 4096 32 24.087370 2049.11 0.233 1.797 12288 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 3.016768 16320.94 0.257 2.342 12288 12288 4096 2048 4096 3.923654 12556.31 0.481 3.222 24576 15345 4096 1024 4096 5.679717 8668.82 0.917 4.639 49152 18121 4096 512 4096 9.399901 5240.76 1.841 7.559 98304 20859 4096 256 4096 16.367710 3006.58 3.444 13.221 196608 22789 4096 128 4096 31.675000 1560.38 9.018 23.719 4096 64 4096 53.342560 922.10 13.588 42.333 4096 48 4096 69.584420 706.95 17.881 54.894 4096 32 4096 106.527200 462.39 30.318 82.404 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 2.974195 16565.27 0.211 2.350 12288 12288 2048 2048 4096 4.133799 11902.04 0.450 3.281 24576 15092 1024 1024 4096 6.164829 7992.48 0.957 4.894 49152 19122 512 512 4096 10.451161 4715.97 2.031 8.263 98304 21104 256 256 4096 18.807820 2620.29 3.982 15.090 196608 23066 128 128 4096 37.130580 1324.47 8.130 30.029 393216 23823 64 64 4096 84.137660 584.34 14.867 71.294 786432 24189 48 48 4096 126.425700 388.84 20.348 109.344 32 32 4096 221.760400 221.77 33.135 193.900 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 3.110395 15825.27 0.217 2.264 12288 12288 2048 4096 2048 3.966562 12421.74 0.225 2.392 13492 24576 1024 4096 1024 5.395088 9158.17 0.356 3.178 16712 49152 512 4096 512 7.603905 6494.39 0.391 3.804 17592 98304 256 4096 256 11.372370 4354.66 0.384 4.935 18457 196608 128 4096 128 15.966410 3131.70 0.492 6.511 22925 393216 64 4096 64 27.984730 1780.97 0.540 9.214 23916 786432 48 4096 48 33.752020 1481.94 0.549 10.855 32 4096 32 48.274100 1035.31 0.485 13.882 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 3.012788 16354.68 0.242 2.275 12288 12288 4096 2048 2048 3.914812 12579.65 0.430 3.217 24576 24576 4096 1024 1024 5.717559 8637.98 0.909 4.572 49152 49152 4096 512 512 9.654981 5149.07 2.057 7.438 98304 98304 4096 256 256 16.794560 2951.30 3.440 13.374 4096 128 128 30.341820 1637.45 7.353 23.028 4096 64 64 56.350890 884.14 14.076 42.337 4096 48 48 74.685280 670.05 18.952 55.484 4096 32 32 136.450100 364.38 30.751 85.172 Client Proxy Server Elapsed_Time Thruput User_Time Sys_Time #pIO #sIO --------------------------------------------------------------------------- 4096 4096 4096 3.019448 16286.94 0.220 2.286 12288 12288 2048 2048 2048 4.137213 11908.48 0.504 3.282 24576 24576 1024 1024 1024 6.010369 8217.45 0.905 4.826 49152 49152 512 512 512 10.200835 4863.92 1.920 8.059 98304 98304 256 256 256 19.624430 2529.08 4.136 15.274 196608 196608 128 128 128 36.115320 1372.77 7.292 28.746 393216 393216 64 64 64 94.335220 523.52 17.324 73.963 786432 786432 48 48 48 141.873300 347.95 20.906 117.773 32 32 32 249.935556 196.67 27.327 215.981 Conclusions and Observations: ----------------------------- From the client/server tables, we can see that reducing the client buffer size appears to have a greater effect on throughput than lowwering the server size. This is not simply due to the number I/Os since we can see higher throughput for cases of higher number of I/Os, as when the client buffer size is reduced. It is interesting to take some example cases and look at how the I/O sizes are grouped. The following tables show how the I/Os are broken up for the server node for various buffer sizes. client = 2048 server = 4096 --------------------------- 8124 2048 byte transfers 8226 4096 byte transfers client = 4096 server = 2048 --------------------------- 24576 2048 byte transfers client = 2048 server = 2048 --------------------------- 24576 2048 byte transfers **************************** client = 1024 server = 4096 --------------------------- 2 3072 byte transfers 3 2048 byte transfers 9870 4096 byte transfers 9660 1024 byte transfers client = 4096 server = 1024 --------------------------- 49152 1024 byte transfers client = 1024 server = 1024 --------------------------- 49152 1024 byte transfers **************************** client = 512 server = 4096 --------------------------- 10558 512 byte transfers 1 3072 byte transfers 1 2560 byte transfers 2 2048 byte transfers 2 1536 byte transfers 10964 4096 byte transfers 1 3584 byte transfers 1 1024 byte transfers client = 4096 server = 512 --------------------------- 98304 512 byte transfers client = 512 server = 512 --------------------------- 98304 512 byte transfers **************************** client = 256 server = 4096 --------------------------- 1 512 byte transfers 1 3072 byte transfers 1 2048 byte transfers 11418 256 byte transfers 1 3328 byte transfers 11572 4096 byte transfers 1 768 byte transfers client = 4096 server = 256 -------------------------- 196608 256 byte transfers client = 256 server = 256 ------------------------ 196608 256 byte transfers **************************** It looks like when the server read size is greater than the client write size, the client writes become grouped together. Even though the number of I/Os are less, the throughput is less also. It is likely that collating the smaller writes into larger reads extracts a time penalty.