Help understand stagger size and RAM usage



  • Hi,

    I did a lot of reading trying to understand how mining works and I have learned a lot so far and found the Burt way of mining very interesting.

    There are few things that are still not clear to me so I got few questions hopping that you guys can help me understand this better.

    I know that the miner reads the drives in parallel and the files inside the same plot in serial, if I understand this correctly, that mean that the RAM usage will grow by the number of drives not files.

    Given that 24576 is ~6GB RAM;

    1 - Let say I have one drive as follow, am I right to say that this will consume 6GB of RAM not 12GB?
    drive-1
    |_accountID_sn_n_24576
    |_accountID_sn_n_24576

    2 - Let say I have two drives as follow, if 1- is right then this will consume 12GB RAM right?
    drive-1
    |_accountID_sn_n_24576
    |_accountID_sn_n_24576

    drive-2
    |_accountID_sn_n_24576
    |_accountID_sn_n_24576

    I am trying to maximize my RAM usage while mining on my new 6 x 6TB drives and 40GB RAM machine, and since the miner reads the drives in parallel, I decided to allocate 6GB RAM per drive so I decided to go with a stagger size of 24576 (~6GB). I wanted to plot in ~500gb files and since I was trying to keep the number of nonces a multiple of the stagger, I settled for 2064384 which gives me files of 504gb and a ratio of 84 between stagger and nonces.

    I used gpuplotgenerator and everything seems to be working as expected, the plotter is generating files in parallel at the rate of ~50k nonces/minute while using ~38Gb of RAM.

    Now that the plotting has finished, the plotter replaced the stagger size to be equal to the number of nonces used instead of the one I specified;

    accountID_0_2064384_24576 (input) ->
    gpuplotgenerator ->
    accountID_0_2064384_2064384 (output)

    I have read that this is normal since I am using the plotter in direct mode which generates the plots already optimized, now my question is;

    3 - How will this affect my RAM usage when mining since that's dictated by the stagger size you use? I know that my original file (accountID_0_2064384_24576) will consume 6GB of RAM, how much RAM will the optomized plot accountID_0_2064384_2064384 consume? I am planning to use jMiner which uses chunkPartNonces=320000 by default, even that seems way too high for RAM usage, my guess is that the miner will just use as much RAM as it can but I am not 100% sure about that.

    4 - I see that many people scan TBs of drives in a matter of seconds, how is that possible? How can the miner scans that much space in such a short time? I can't be possible that the miner is reading all the nonces, is there like a set of nonces that needs to be verified each round that the miner needs to read?

    thanks guys in advance, I am really excited by this new way of mining and I am really interested in understanding how it works, not just get it working. I spent too much time trying to look for a "technical" document that explains how the mining works but could not find such a thing, if you know something like please point me to it.

    Thanks.



  • @Mountassir18 when the plots are being read by the miner, they do not consume the memory you think they do, that was for plotting, not mining... while mining my system is using 4GB of 24GB total, and the actual miner is using it appears 4MB of that in standby mode and only jumps to 14MB while searching the plots, thats it, MB not GB or TB, such a small amount the system doesn't even register a bump in the graph...

    when a block comes up, you don't scan through 100% of the file, you're looking at a fraction of each plot file, not the whole file... if I recall off the top of my head you're reading 1/4096th of all that information, so divide whatever amount of space you have, by 4096 and thats how much data will actually be read when the miner looks...



  • @Darkbane Thanks for clarifying, It makes a lot more sense now knowing that the miner only has to read a fraction of the plot file.

    Regarding the RAM usage, I think how the stagger size has been defined in various places makes you think that a lot of RAM will be required (or at least recommended) if you want decent mining speeds. This is just an example in Burst Tutorial
    "The place that stagger size really comes into play, is during mining. During mining, the larger the stagger size the better on your drives, the less seek time, and faster your computer will read through the plots, thus faster deadline submission and more of a chance of hitting a block."

    But I guess from what you are saying, this is only true up to a certain point, after that more RAM won't necessarily help your mining speed.

    Thanks


  • admin

    @Mountassir18 RAM is important to make your stagger as large as possible during plotting, which make mining more efficient.

    However, with xplotter and gpuplotgenerator in direct mode creating optimized plot, RAM is no longer so important.



  • @Mountassir18 said in Help understand stagger size and RAM usage:

    @Darkbane Thanks for clarifying, It makes a lot more sense now knowing that the miner only has to read a fraction of the plot file.

    Regarding the RAM usage, I think how the stagger size has been defined in various places makes you think that a lot of RAM will be required (or at least recommended) if you want decent mining speeds. This is just an example in Burst Tutorial
    "The place that stagger size really comes into play, is during mining. During mining, the larger the stagger size the better on your drives, the less seek time, and faster your computer will read through the plots, thus faster deadline submission and more of a chance of hitting a block."

    But I guess from what you are saying, this is only true up to a certain point, after that more RAM won't necessarily help your mining speed.

    Thanks

    the information is not really presented in a way someone new will understand, its the same way math class felt for me every year, it all sounded like greek but it was called math, and every year I had to make heads or tails of how things related to one another that others with more knowledge seemed to suggest but they forgot I was completely new to this and would likely misunderstand what they mean...

    I think in this case, they could have explained whats impacted to help people understand it more... stagger is essentially referring to the order of the data within the file created, the lower the stagger number, the less organized that data is, which will cause a hard drive to jump back and forth from one sector to another, in order to piece together the data, a higher stagger has all that data organized in a row...

    so its correct, but I can agree misleading to someone who doesn't know whats going on... thats why I get upset at people giving advice to folks to buy hard drives and jump into this, without them understanding very basic things they need to make good decisions... same reason I am very critical of website calculators showing people potential profits if all the data used to calculate that profit remained the same today, but seconds after you used that calculator, you could get a different result, little alone 30 days later when you will get a dramatically different result...

    this is why my best advice to people is don't buy hardware for mining, use what you have, and you won't have much disappointment or loss... learn it before you invest in it, because if you invest before learning about something, you'll likely regret the loss far more than if you had waited to learn about it... I mean I see people buying tons of RAM because they think they need to for plotting, even if it does help some folks, it won't make ANY difference for others who already have enough RAM for what they are doing... most folks don't need to buy ANY... yet they still buy it...


Log in to reply
 

Looks like your connection to Burst - Efficient HDD Mining was lost, please wait while we try to reconnect.