try crossover between 2 machines first
Sir Runcible Spoon,
Try linking your ethernet cable directly between two machines and see how fast it goes (you will problably have to set up ip manually though as no DHCP server and you will have to connect to each other by typing the ip address. This takes out all equipment just to see what your best speed is.
Then also change on both cards your MTU to 9000 (providing your cards support jumbo frames). Doing this on between 2 macs usually get's around 80 to 90 MB/s. Do note sometimes when altering MTU connection goes a bit flakey though usually unplugging the cable and replugging it gets them to sync up a bit quicker - note setup has to be done on both machines.
If the switch also supports jumbo frames then afterwards can be left like that. If they are transmitting to each other over IP then those speeds can be obtainable.
350Mbps though is around what I'd expect a gig network to do without changing to jumbo frames etc... it has to do with what buffer the card has and how many packets per second it is able to process. When MTU is at 1500 which is the normal default for 100Mbps a packet has 1500 bytes in total. Of this 53 bytes is overhead so actual packet is 1447 bytes.
If the card/pc combo can process say 40000 packets per second then that becomes 55 Megabytes/second. This is raw ethernet though and the packets will additionally be wasted by the protocol - ftp usually being the leanest hence the fastest. Packets per second on Gb max is 1.4 million per second though it's not just the card but the OS, cpu speed and bus speed the card is plugged into...
if MTU is upped to 9000 though then the packet should be around 8947 bytes x 40000 in theory would get 341 Megabytes/Second but the ceiling is 100 Megabytes/second.
Of note most home routers cannot come anywhere near such packet switching speeds (usually they max out at around 4000 pps as the processor clock is the limiting factor though this would only effect routing to the internet or between ethernet and wireless).
ps I'm using rough figures the numbers are ballpark not spec accurate... It's mostly figures I see by observing ping flood responses when I put in a network to see if there are any bottlenecks anywhere. I'm not a routing engineer...