Hardware for 20x USB 3.0 drives connected to one machine
-
I have had various issues with having 20 USB 3.0 drives hooked up to one machine. 10x 8TB drives, 10x 5TB drives. Sometimes they are stable for weeks, sometime a few drives drop out regularly and I need to reset the drives or the PC, or sometimes several drives are painfully slow (instead of taking 10-15secs to read, they take 7-8minutes). I have tried multiple machines with various results.
I have 3x 7-port USB hubs. I have also used a couple PCI-E to 4-port USB cards on machines that didn't inherently have USB3 ports but were otherwise pretty juiced up machines. I have tried combinations of plugging some in to the PCIE card, some in to the onboard USB3.0 ports, balancing on the 7-port hubs, etc.
I am having a hard time determining truly what the bottlenecks are. I know that this is quite demanding on a PC, but I really believe it should be possible.
Is anyone else doing something similar? Any recommendations for hardware? How about any theory regarding PCIE USB cards, USB3 hubs, vs onboard ports, etc?
Thanks!
-
Are you using USB 3 cards with 4 controller and 4 ports, or does it have 1 controller divided by 4 ports? I use Sonnet allegro pro with 4 controllers (must be the pro version). When mining, the miner will only read about 0.024 % of your plots, but you will eventually get a bottleneck on your USB 3 controller. I haven't got the exact figures here, but I think you can have about 2-3 HDDs connected to one controller without slowing down read speed, but with 4 or more you will likely get a bottleneck. This may not be true if your HDDs are small capacity, but it will be true with your 5 and 10 TB HDDs. Therefore, your 7-port USB hub will lead to a bottleneck, as it will force all 7 connected HDDs to be read by the same controller. If you're not quite sure what I mean, you should try to learn more about the difference between a USB 3 port and a USB 3 controller.
-
I use 2, 7 port and 1, 4 port usb hubs. I found that especially for the 7 port hubs the power supply was small. I actually got 2 PS from a different 4 port hub. That helped. Also if a drive would drop out I will move it to a different cable or port or both until the system gets stable.
-
@Propagandalf What means bottleneck if I may ask?
-
@pr0cesor The thing that slows the whole process down. You can pour water out of a jug rapidly, but pouring it out of a bottle it gets slowed down at the bottleneck.
-
@pr0cesor bottleneck = slowest part, the part limiting others to work faster .. if a bottle had no neck, you could drink beer faster :-)
@DrTrouble If a pcie card has 4-ports or 7-ports does not matter in speed if all ports are using one controller ... to improve speed ensure you get usb controller cards with dedicated controllers for every port ... no shared. Like this ones: http://geizhals.eu/?cat=ioexpcard&xf=1331_4~621_PCIe+x4
A system seams to run in trouble on too many usb3 controllers ... @Propagandalf had a lot of issues as he tried to run 5 pcie cards with 4 controllers each.
-
Dang it!!! That is my problem- I was wondering why I'm getting only a combined 100MB/s writing on 3 USB 3.0 drives all plugged into "separate" ports on my USB 3.0 card- duh!! Piece of crap $40 card
-
there are usb 3.0 hubs on ebay. Internal ones up to 7 ports each for PCIE and external ones up to 10 ports each . The internal pc ones are preferable but external ones are ok but best if they are mains powered (for gain, speed and non mains powered external) better speed can be obtained with an internal PCIE SATA card because USB 3.0 is best as a "no more room" option.
-
@ZapbuzZ said in Hardware for 20x USB 3.0 drives connected to one machine:
there are usb 3.0 hubs on ebay. Internal ones up to 7 ports each for PCIE and external ones up to 10 ports each . The internal pc ones are preferable but external ones are ok but best if they are mains powered (for gain, speed and non mains powered external) better speed can be obtained with an internal PCIE SATA card because USB 3.0 is best as a "no more room" option.
How many USB 3 controllers you have is a determining factor for your maximum read/write capacity over the USB bus. I believe the internal 7-port card and external 10-port hub you mention only lead back to the same USB 3 controller, and hence the maximum bandwidth is divided over each port.
The pro version of Sonnet Allegro pro, as a comparison, has 4 unique controllers and 4 ports (one controller for each port), whereas the non-pro version only has 1 controller but 4 ports. If you have many HDDs, having many controllers is the way to go, but it all depends on how good read/write times you need since pro controller cards are expensive.
-
if the miner does intelligent caching then that would improve bandwidth simply working through plots faster

