Network drives (windows)


  • Mod

    Problem: Network drives.

    For example, we have network drive - 10Tb
    we can't change position of HDD's heads, for reading this file we must read (and download) all bytes from start.
    time for reading ~22hrs (1Gb/s)

    Solution:

    Plotter makes 4096 files (split 1 plot by 4096), which contain 64*nonces bytes.
    Each plot = one of 4096 scoops
    something like
    1234567891011121314_0_1000000_1000000#3095

    miner reads one of that files at new block (scoop defined)
    time for reading ~10 sec (1Gb/s)

    any ideas?

    @luxe @haitch @catbref @daWaIlet @Creepsky @IceBurst @Lexicon @crowetic



  • @Blago Yes I had a similar thought when doing Amazon and Google Cloud Mining, would of course need a modified plotter and Miner.

    How much work is it?

    Rich



  • @Blago said in Network drives (windows):

    Problem: Network drives.

    For example, we have network drive - 10Tb
    we can't change position of HDD's heads, for reading this file we must read (and download) all bytes from start.
    time for reading ~22hrs (1Gb/s)

    Solution:

    Plotter makes 4096 files (split 1 plot by 4096), which contain 64*nonces bytes.
    Each plot = one of 4096 scoops
    something like
    1234567891011121314_0_1000000_1000000#3095

    miner reads one of that files at new block (scoop defined)
    time for reading ~10 sec (1Gb/s)

    any ideas?

    @luxe @haitch @catbref @daWaIlet @Creepsky @IceBurst @Lexicon @crowetic

    For drives over a lan having large files is not problem over the wan this might work, but amazon will lock your account for usual behaviour


  • admin

    @Blago I have run a network miner and it wasn't that slow. How are you repositioning? It shouldnt matter if it's local or network if you do a fileseek to offset X. If you're using a driver level reposition that could be a problem - but I doubt you're doing that.



  • a bunch of smallest plots in the network drive that can be called numerically from name buffer to target correct nonces without reading 1 huge plot once known?
    I mean the miner reads all these little plots firstly in the directory discovers the nonce numbers indexes them in accordance to the plot names in a buffer and calls to read from little plots thereby reducing reading time because they are in smaller many plots but then the buffer would be quite big so it'd have to have a lot of virtual memory access in case the computer has not enough SO many little plots can skip being read for the nonces.



  • so therefore an alternative way to control hard drive heads in the network @Blago :D



  • i think that @blago needs to clarify a little of what he want to achive. If you use network drives based on protocols like SMB or NFS or similar you do not need to change the format since you can do seek a possition over those kind of networks. However over protocols like https or ftp it is alot harder.

    But on the contrary to keeping the format i would like to sugest that moving to 4096files per plot is actually a good idea anyway and would bring many benefits.

    +The plot would always be optmized.
    +You can easily split a plot over many drivers
    +you can also increese / decreese the plot size without problems.
    +new format would not contain a stagger at all. (same as optmized)
    +repairing of plots would probably be faster.

    -Alot of files to keep track of.



  • @Quibus I understand that is it takes about 3ms to close a plot file and open the next one during mining. If you split your plot into 4096 files, wouldn't that cause some performance loss?

    Thank,
    ITguy



  • No. since they are sorted per scoops you would only read 1 of them.


Log in to reply
 

Looks like your connection to Burst - Efficient HDD Mining was lost, please wait while we try to reconnect.