Got Big amounts of storage and just about to start mining



  • Hi Guys,

    I have stumbled across burstcoin mining by accident which looks like a extremely nice posibility for me to do some proper mining.
    I currently have the below equipment available for mining
    4 QNAP 459 and 419 with 4x 3TB = 48 TB
    1x Dell NX3200 with 12x 4TB = 48TB
    1x Dell Equallogic with 24x 3TB = 72 TB
    2x Dell Equallogic with 16x 2TB = 64TB
    I also have some stationary computers with smaller drives and some USB3 passport drives.
    I also have some sas drives with 300GB and 10k / 15K rpm
    and some micelanous 2.5 drives SSD's and HDD's varying from 128GB to 750GB, and some other 3.5" drives that I don't know the status of.

    I am still at work and have not been able to get home to dig down into the mining business, I have some questions thou on a few different topics.

    SSD's / Ram Drives:
    I have briefly read that SSD's is not good due to the lack of size. Does that mean that the increased speed of a SSD does not improve the mining ?

    in example a normal sata 7200rpm drive has between 60-100 MB read / write while a SSD normally has around 500-550 MB read/write which is an improvement towards 8-10 times in read / write speeds.
    Not to mention my NVMe 950 PRO drives where I got 512GB spare at them moment with 2500/1500 MB read write
    I also have a ramdrive on my computer with 40GB (DDR4 3200Mhz) which has 10GB/s read write
    Does the speed of the drives not improve the mining eventhou the size of the drive is small?

    NAS/SAN's:
    Also, how is it with network attached Storage (Nas) and SAN's ? I have read that it will take ages to plot a big volume and that it would be smarter to have several "smaler" volumes.
    On the Qnaps I was thinking of having the drives as normal without a raid setup and split them out as seperate volumes for the computer to connect to.
    ON the Equallogics I actually have no idea what I should do. 2 of the Equallogics has Fiber connection so I should be able to get 10Gbit bandwidth (1250MB) but compared to how many drives it has I will not get the full potential of each drive. I potentially can configure LAG and get 2 ports link agregated (have only used failover previously) and get 20Gbit bandwidth (2500MB/s)
    The other has 4x1Gbit which I should be able to put LAG on which tops my throughput at 4gbit (500MB/s) total. From my expectations I will not get full benefit of the equallogics. Does anyone have any inputs or thoughts ?

    Dell NX3200:
    This is a server with 12 internal drives, I believe the throughput here to be very good, but the CPU is a cheap Xeon which will potentially spend a long time on plotting the drives.

    Plotting:
    I have one main computer with a i7,6770k GTX980ti graphics card and one PC with a athlon phenom 6 overclocked to 4Ghz with R9 290x Graphics card I was thinking about using these to plot the drives with and then put the drives in the respective casings / servers.
    I also have some Dual 4core Xeon servers and some dual 6core/xeon servers that potentially can do some plotting if necesarily. Does anyone have any input on what will be the best for plotting ?

    USB3 Drives:
    will cheap USB3 drives be better for the performance as these will get the full bandwidth of the drive ?

    Mining in pool or by yourself:
    What is the benefits in each case ? will it be too hard to mine on your own even with the level of storage I have on my own ? Or will it be worth the fee of mining in a pool which at least will secure some burst's eventhou you don't get the full benefit of your mining yourself ?

    What is the risk of a drive getting corrupted ?
    What will be lost if a drive gets corrupted ? will you loose anything at all, or does the content get lost ? (I don't fully understand the algorithm yet and how the drive is used in the mining)

    As you might see I have loads of questions and my setup is far from the normal miner :D
    Iguess I will need to learn by trial and error, but I was hoping some of you had had similar experience and inputs on some of the points above.


  • admin

    @Thedude I'll cover as many of your questions as I can ....

    SSD's. Speed is important, but capacity is more important. You want to get your DL's submitted as fast as possible, which favors SSDs, but the capacity of your plot is far more important. Think of Burst mining as a lottery. your plots are tickets into that lottery - the more tickets you have, the more likely you are to win.

    SAN/NAS - I have a 50TB miner that mines that capacity over 1Gb link, and it earns well. Direct Attached Storage will normally be faster, but SAN/NAS is fast enough, and again capacity trumps speed.

    The Dell NX3200 - if it's a low end Xeon, and good sized drives, then yes - it's going to take a while to plot. Potential options are a) A better CPU, b) a GPU.

    Plotting: For some of the systems, you can plot om your primary PC and then move the drives, but generally for the servers, it won't work. The servers will use array controllers that add additional metadata and change the layout - the Array controller wont recognize a PC plotted file, the PC won't recognize an array configured disk.

    For the server plotting, the more threads and RAM the faster you'll plot.

    USB 3 - maybe, the common USB drive used is the Seagate archive drives. These drives are SMR (Shingle Magnetic Recording) - they mine just fine, but are incredibly painful to plot. Think two weeks to directly plot an optimized 8TB plot.

    Solo/Pool - depends on capacity. If you've less than 50TB, pool mine. If you've got more, then it's personal choice. Pools will give you more frequent payouts, solo will give you bigger payouts for blocks you forge.

    Corruption. If a drive gets corrupted, reformat and re-plot with the same parameters of the original file.

    Feel free to ask more questions or for clarifications.



  • I'm in a similar situation, realised burst mining was a thing and had this sitting around collecting dust...

    2 x Bladecentre Chasis (M8677-2XX & M8677-3XM)

    • (2) Fibre card for SAN connectivity
    • (4) Cisco switch cards
    • (3) Management interface cards
    • Acoustic Attenuation Module AKA The Muffler (to make quiet like)
      12 x HS21 Blades
    • SAS Drives
    • upgraded ram; differes per machine, also have some spare
    • duel xeons
    • fibre cards in every blade! (cost me a fortune :P)

    STORAGE / SAN

    1 x HP Storageworks SAN Fibre Switch
    2 x HP Storageworks HSV210-A EVA Controller
    4 x HP Storageworks drive Shelves
    28 x 300GB 15K Fibre Channel drives
    28 x 146.8GB 15K Fibre Channel drives

    Other/Extras

    2x PCIe Intel 39Y6138 (Quad Gigabit) Network cards
    1x PCIe QLogic fibre card
    1x PCI QLogic fibre card
    2x SFP Optical Transceivers

    Is this gear too old to use / incompatible?



  • @thebaron bah nothing is to old to use. PLotting might be slower if just using the CPUs on the server compared to newer versions or a GPU but mining will be fine



  • @manfromafar i would like to see you mine on a ZX81 or C64? lol



  • Thanks @Haitch you sir are a legend :)

    You answered my question very simple and straight forward. Thanks for taking the time to assist me in my journey of burstmining.

    I will start going through the basics and setting up my NAS'es after work now and take the NX3200 and Equallogics a bit later this week as they are offsite at another location.

    Is it normal to plot the drives as one plot of 4TB/3TB or do you normally plot in chunks of 500GB so that the plotting goes quicker and if you loose a drive you can replace the damaged plots with different sized drives ?

    @thebaron hi mate, sounds like we are in a very similar situation, I bought out a bankrupt company and have been sitting on the equipment for 2 years now. Lets keep in touch on the setup to share experiences.
    From my point of view I believe you can do a nice setup with the equipment you have available, your 15K drives should perform pretty well. the fibreconnectivity increases the bandwidth from normal ethernet and as Hiatch mentions above, storage size is more important than the connectivity so I believe you should be good with your equipment as well.
    I will do some experiments with different quick storage solutions in addition to see how that affect the results over time.


  • admin

    @Thedude I normally do one large file - if you use xplotter it can be resumed if interrupted. Other prefer to do smaller chunks, but the most efficient plot file is a single optimized plot.



  • @thedude, yeah let's. It will be an interesting little recycling project!



  • I got some more questions now as my build has started.

    Wallets:
    Is it best to set up different wallets per machine / or is it possible to use the same wallet across all machines ?
    If I can use the same wallet, do I copy out some wallet files on the first computer and then paste it to the second, third ++ computer ?
    is there any pro's / con's on using one or more wallets ?
    I have read that you can encounter issues with same drive names if you use the same wallet on several machines, is this a big issue ? or can it be worked around ? I will in the end of the build have more drives than the alphabet got characters :P

    Raid:
    In terms of raid setup, I have heard some videos saying that Raid is not good in burst mining I can see that for raid 1,10,5,6,50,60, but I don't see how Raid0 will not increase performance as that will keep all the storage and increase the read speed while having one big plot instead of several smaller.
    of course the back side of this is the plotting time and if one drive goes down you need to rebuild the whole plot for all drives as Raid 0 don't have redundancy.

    Drives:
    I started with my NVMe and RAMdisk yesterday to investigate the speed over storage size thing. I only had 530GB in total between the RAMdisk and NVMe drive. Their speed was both at about 250MB and search speed was about 0.2-0-5 seconds. wich just to confirm the stated above, Quick drives as SSD, NVMe and Ramdrives is not cost efficient for this "business".
    But, I just set these up quickly for testing and to see how things worked for my own learning to be able to build the rest as good as possible.

    Mining:
    While mining my drives quickly found some green lines, but afterwards waited several minutes before searching again, is this normal ? Where can I find literature on the fundamentals of the mining and how it works, what the different lines mean and how the algorithm works ?

    Plotting:
    is there any best practice on how to plot drives ?
    I started with plotting one drive, then plotted the next starting on the last nounce of the previous plot.
    I would like to calculate the plots so that I can plot on several computers at once with either GPU or CPU plotters.
    I have seen some say start with 1000000000 on the first drive then 20000000000 on the next, is this a good way to do it ? is there any back side of not having a continuous plotting sequence ?

    Pools:
    Any recomandations of pools while I build plots, I am now only at 1.3TB as I plotted desktop remaining space to see the basics. I estimate my pool size in the end will be at least 100TB possibly going towards 150TB depending on what I have space for and what works best of the setups I build :) I got equipment for possibly 200TB+, but I am not sure if the enterprise Equallogic SAN's will be cost efficient to run.



  • Just to do an update on my progress. (it is safe to say I got a bit hooked on this :P)

    I have now set up my 4x Qnap 12TB (48TB)
    I put 2 and 2 drives in Raid0 to get bigger volumes with increased speed.
    I have at the moment 3 seperate computers plotting the different volumes, I will set up a fourth today so I can plot all 4 Qnaps in paralell.

    I have bought
    3 Seagate 4TB External HDD's
    2 Seagate 5TB External HDD's
    I have ordered
    5 Seagate 5TB External HDD's

    I have 2 Precision M4700 Laptops which plot 2 External drives each (ETA 3 days plottin a 5TB)

    my plotting strategy is:
    disk 1 start 1000000000
    disk 2 start 2000000000
    disk 3 start 3000000000
    etc...

    Currently I will be at 144TB plotted storage within the next week / week and a half.

    I am not sure if I will connect the 2 equallogic 16 Bay 2TB SAN's as I think it will be a nightmare in terms of power consumption and plotting speed. I think it will be better to try to sell them and go for 4x 8TB external drives as replacements per Equallogic. The Equallogics are not easily sold second hand thou.

    I have some miscelanous 3.5" and 2.5" lying around in different sizes which i will fit into the different workers afterwards. these are at the lowest priority as they are not big in size.

    I am on the edge of scratching my Synology 4Bay (4x4TB) media center for mining as well >.<

    My additional investment at the moment is at $2262 for the additional external drives. The other hardware I had lying around. This gives me a ROI of 3,3 months at the current prices when everything is up and running properly.



  • @Thedude said in Got Big amounts of storage and just about to start mining:

    I am not sure if I will connect the 2 equallogic 16 Bay 2TB SAN's as I think it will be a nightmare in terms of power consumption and plotting speed. I think it will be better to try to sell them and go for 4x 8TB external drives as replacements per Equallogic. The Equallogics are not easily sold second hand thou.

    hehe, I know what you are talking about. have literally a dozen Infortrends (12x16x2TB, FC-SATA) lying around. I switched them off a year ago - ebay price is 200 USD max. per unit. And getting bad feedback because 3-4 year old HDDs start to fail ..

    Then I had the idea to put all those 2 TBs into a SAS JBOD, started building, then disks started to fail.. which in itself is no problem (only Seagates fail, Hitachis are robust as ever) as I have them in zfs raidz1, but the rebuilds took away io cycles that mining was affected.. facing 0.30$/kWh I turned off anything under 4 TB/drive.

    I think of scrapping the rest.



  • @haitch said in Got Big amounts of storage and just about to start mining:

    @Thedude I normally do one large file - if you use xplotter it can be resumed if interrupted. Other prefer to do smaller chunks, but the most efficient plot file is a single optimized plot.

    I am just about to put this theory to the test, erasing a drive with 1TB plots to make a single giant plot that will take a day to complete... what tests have you done and what results did you get, wondering if its just a few milliseconds or what, but since curiosity killed the cat, I will have to do it myself since there is no cat left anymore!

    when I was goofing around yesterday I did notice the expected difference in 512 legacy allocation versus 4096 allocation in terms of sequential read/write speeds, which made sense since almost every drive I'm aware of today is made with 4096 sectors and windows is emulating 512 for legacy support, when I goofed around and tried 64k allocations there was almost no difference from 4096 and in one test I actually got slightly less than a 1% worse result... all of this is like one giant excuse to goof around with hardware and software... yay!


  • admin

    @Darkbane I don't have empirical numbers, but after the first drives I plotted not knowing what I was doing that had multiple smaller files, vs the replotted drives with individual, mining time was faster. From memory in the 10-15% range



  • @haitch said in Got Big amounts of storage and just about to start mining:

    @Darkbane I don't have empirical numbers, but after the first drives I plotted not knowing what I was doing that had multiple smaller files, vs the replotted drives with individual, mining time was faster. From memory in the 10-15% range

    you're in luck, its been 24 hours, my gigantasaurus plot has finished, and I'm comparing it to the other drives... so far, not much difference, albeit a slight difference that I am trying to take with a grain of salt since each one is finding a different amount of deadlines which causes a different pause on each drive... however my comparison is 9x 1TB plots vs 1x 9TB plot so I may not be expecting to see much difference in all honesty, at least not like you mentioned with having many many files... mostly I was hammering away at the 512 vs 4096 allocation sizes to see how the miners compare...

    to be honest, I think my tests are going to be bottlenecked by the miners, when using crystaldiskmark I was able to show a measurable difference between the 512 and 4096 allocations, I had a max sequential read around 240MB/s but so far I am limited to 80MB/s with CPU AVX (not AVX2 as my cpu is older) and 160MB/s with the GPU miner... so I'm beginning to wonder if I can even logically test the maximum performance if I am not able to sustain higher speeds with either program... its not that I expect to hit 240MB/s in reality, but I was certainly hoping for more than 160MB/s with the GPU, as I can transfer drive to drive and sustain 220MB/s quite easy, I was hoping to get closer to that on a single large file...

    I guess its time to delete plots on that other drive so I can try making a thousand smaller plots! maybe this is why curiosity killed the cat, he kept poking at things that worked just fine, because he wanted to see what happened if he stuck his head a little bit closer to that swirling fan... I better put a helmet on...



  • So, 64K allocations is not the optimum way to go, huh? Sounds from your experiment that you had the best luck with a 4096 allocation.


Log in to reply
 

Looks like your connection to Burst - Efficient HDD Mining was lost, please wait while we try to reconnect.