Shingled magnetic recording (SMR)



  • I'll throw some more numbers at you guys, since someone will probably find this useful at some point:

    With GPU plotter in direct mode, I can write a 500GB plot file to two 8TB Seagate NAS drives in 3.5 hours. That's somewhere around 20,000 nonces/minute. The initial plot file structure writes at nearly 200MB/s, and then that drops down to about 120MB/s while filling in the plots with nonces. So to plot all 16TB, it takes about 2.5 days. Given a generous estimate that the SMR drives can push 10MB/s writing plots in direct mode, it would take them over NINE days to write the same plots. No, I have not fully tested that, but I did start the tests and saw the write speeds in win10's resource monitor first-hand. So even if it takes 3 days to plot to the NAS drives and 3 days to write those plots back to the archive drives, that's still 3 days faster and only half of the 6 days it takes is actually spent doing intensive plotting.



  • @sevencardz @nixxda @luxe and others who I discussed this with:

    My scenario
    I have been plotting 4x 8TB Seagate backup plus SMR drives simultaneously, using GPU plotter in buffer mode. Initially, I was getting about 180 MB/s write speed with ETA of 1d 10h to complete. While I was away and got back the next day it had reached 80 % completion, but write speed had dropped to 10 MB/s and took forever. If relevant, the drives had NTFS file system, 64k cluster size and 3 of 4 drives had completed.

    Solution
    Abort the plotting process, use plotschecker to repair and trunkate the file(s) that were incomplete and recalculate how many nonces are missing in order to fill the drive(s). Update your gpuplotgenerator batch file with your new values and restart your computer.

    When your system is back online, run gpuplotgenerator.exe through your .bat and you should find that your remaining plots will write at full speed again. If you choose not to restart, your write speed will be 10 MB/s.

    Theory about solution
    I think that when you restart your computer, you force your HDDs to flush their internal buffer or write cache. I believe that the SMR drive cannot handle a continuous 'hammering' of write requests, and will therefore write slower and slower until it reaches a very low speed.

    Maybe the same positive effect will be produced if we plot, say, 4TB first, wait for it to complete, and then plot another 4TB to fill the drive. I am currently testing this. My hypothesis is to not let it hit rock bottom speed before the plot finishes.

    Perhaps it is possible to use some kind of memory buffer cleaner together with GPU plot generator to circumvent this issue?

    Reference material and more theories for others to test
    http://goughlui.com/2015/06/07/slow-hard-drive-things-to-check/ please read this article, as it contains a perspective on SMR drives that I have not seen explained elsewhere yet.

    I checked my 8TB drives, and they actually had write cache disabled by default! According to the article, activating or disabling write cache has a huge impact on performance, as seen in these graphs:

    0_1485469926148_upload-8e2db24c-1435-4630-83da-73e1f8aefc40

    SMR write cache on
    0_1485469958192_upload-415e2048-830a-4ec6-b99c-c388a98da9ba

    SMR write cache off
    0_1485469972186_upload-eba63029-4c58-46b9-bc3a-f093faf02569

    I did not change my drive settings, so mine have had their write cache off all the time. I truly believe that performing the steps in the article will help with drive performance in general, when you are filling up your drives with non-Burst related items. Maybe they will also improve write speed when plotting and perhaps even mining, someone will have to try it out.

    Like I wrote in the 'Solution' section, I was able to fix the slow write speed problem by other means. I'm not sure, but I think GPU plot generator in buffer mode actually does many of the same things as what activating each drive's write cache will do.



  • OMG @propagandalf gets all the cookies. My problems were entirely tied to write caching being turned off!

    So, by default the expansion drives were set with write caching turned OFF on the disks. In my case, I was simply copying 500GB plot files from one drive to another and getting about 30MB/s write speeds at best. I turned write caching on, rebooted, and now the same plot files are copying at 130MB/s. Praise the sun!

    I also noticed that the 8TB drive I removed from the enclosure and installed internally via SATA had write caching ON by default. Seems like the OS tries to make some intelligent decisions by turning caching off on external drives, but keeps it on for internal drives. I assume this is the reason @AngryChicken had no problem with write speeds.

    TLDR: Make sure disk write caching is turned ON for every drive you plot or you will have a bad time!



  • Great information with real world examples. Hey found a 6 min block And blago read 21mb/ s. Can't wait to test it tomorrow on my new build



  • @Propagandalf @sevencardz Yup this is the case on my end. Never had a problem, also when others pull the drives out of the case and install directly they have better experiences. Maybe it is a system default, but good to know itès just a setting.



  • My test plots have now completed. By plotting 4TB files onto 10x SMR 8TB drives (BUFFER mode), with 1.25GB stagger on each and default cluster size, they completed without even slowing down. I still have not activated write cache on each drive, so no idea if that will improve plotting even further, but I don't think so in my case as 180 MB/s has been stated as a typical top end write speed according to various sources. I guess it should be tested whether it can increase, but I'm pretty satisfied with this, as I have used 24 hours to plot 40TB. I could probably plot even more simultaneously until I reached a bottleneck somewhere. I used 3x MSI rx470 8GB GPUs (1 for quad HDDs, separate command window instances), and 4x USB 3.0 controllers. My AMD FX 8350 CPU had about 35-65 % load during plotting, and I had 32 GB ram total capacity, whereas 12.5 was reserved for stagger.

    However, @sevencardz activated write cache and was able to copy files much quicker. Underneath the option of activating write cache there is another option too: "Turn off Windows write-cache buffer flushing on the device". Apparently this is supposed to make drive write speed even more efficient, but with increased risk of something happening that I cannot remember right now.

    My theory, based on everything I have read and been told, is that default cluster size works best with SMR drives, and that creating single plot files that are over 4TB will eventually overload the SMR drive somehow by hammering it with write requests. By having a low stagger such as 1GB, you avoid long, continuous write sequences that may be the cause of the overload, as the drive only writes for a few seconds, gets time to rest and pick up strength (maybe clearing its buffer too), while GPU pumps out new nonces, and the process repeats itself. Many people will think this sounds daft, but it could be the reason. It would also be interesting to find out to which degree GPU plot generator imitates the function of the drive write cache with its buffer plot mode.

    In a few days I will start to plot the remaining 4TB on these same drives and see if I get the same result. If you don't hear anything from me, you can assume it worked. :)



  • @Propagandalf Holy maroni!-) 40TB/day!!

    "Write cache off" is standard for USB-Disks. So not to loose Data when someone just pulls the USB-Cable out! (power loss on non ex. powered disks)

    I never bothered with the second setting. Someone now what it means?!

    ps. could you add that your using "buffer mode"! (I got really jealous there for a moment until I remembered !-) (but I still am!!-)



  • @nixxda said in Shingled magnetic recording (SMR):

    ps. could you add that your using "buffer mode"! (I got really jealous there for a moment until I remembered !-)

    lol, I did type buffer mode actually, but maybe it got lost inside the parenthesis. =) 40TB per day is still nice, though!



  • @propagandalf Plotting to ten 8TB drives in parallel... that's some real next level stuff right there.

    And heck yeah - we are proud members of the FX-8350 Burst miner club. AMD4LYFE.

    To be clear, I kept the "Turn off Windows write-cache buffer flushing on the device" unchecked for my copies.

    I also think you're right about the stagger size and/or giving the drives time to catch up, especially if you're writing to ten of them because that'll spread out the 'write load' more efficiently. I also noticed that these SMR drives reportedly have a feature where they can boost their writing speed temporarily. I actually saw this happen while copying to the 8TB external drives because during each mining round, the write speeds would drop back down to 30MB/s (due to the fact that the internal 8TB drives were being monopolized by the miner for the round I assume), but right after the round was over, they would boost up to almost 200MB/s for about 20 seconds before they settled back down to 130MB/s again. So just 20 seconds of semi-down time gave each drive a significant temporary boost.



  • @sevencardz Thanks! AMD rox ;)

    I did not know about this temporary boost. Is it a setting, or is it just a built-in feature that is always on?



  • It's strange I guess I got lucky lol. 4 of my drives are the Seagate 8tb and they work perfectly fine. Never had an issue with them up to date :D and even sometimes a better read time than the non Seagate drives I have



  • @propagandalf The storage review here talks about "burst write activity". Of particular interest are these lines:

    "In our test we saw, as expected, the SMR drives took much longer for a traditional full backup, averaging 30MB/s. However we saw sustained read speeds during a 400GB VM recovery in excess of 180MB/s, which is really the core metric. Given the low cost/TB, the drives do very well here if the backup admin can get a little creative. Design your backup window to work with the lower sustained write performance (or design it to fit inside the burst write window completely) but still have your data ready at your fingertips without compromising restore speeds."
    http://www.storagereview.com/seagate_archive_hdd_review_8tb

    I'm wondering if they had turned write caching ON during their full backup (assuming it was off) if the speeds would've gone from 30MB/s to 130MB/s like mine did. Or perhaps it only helps if you are writing massive 500GB files. That basically means these drives are almost as good as normal PMR drives if you can limit the workload to huge sequential writes - which I do by writing all the plots to NAS drives first and then copying them over.

    Also, it seems that if you stagger your write operations so that you're not hammering the queue of write operations on the drive, then you take advantage of the burst speeds more effectively. Of course, this seems to work best if you have other drives you can write to while others are catching up based on propagandalf's results. They don't mention what burst speeds they got, but they do say "burst write speeds are in-line or surpassing traditional HDDs".

    @Zeus If you plugged your drives in to the mobo via SATA or if you were lucky enough for your OS to automatically turn write caching on for the disk, then I'm not surprised you've never had an issue. I'm pretty sure the OS intentionally turned off write caching on the disk for me as a safety feature to avoid loss of data in situations where the user unplugs the external drive without properly disconnecting it in Windows. Of course, as long as you don't plan to unplug your drive in the middle of a write operation, then you don't need or want such a safety net.



  • I had one of those 8TB Seagate Expansion drives and it failed after just a few days. I guess those drives are just terrible quality. I've had no problems with the 4TB / 5TB Seagate's.



  • This post is deleted!


  • @ZapbuzZ Why not just use blagos version 1.0 cpu plotter its way faster now as long as you have a decentish cpu and it optimizes your plots straight away and works your way would take way too long. I have used fastcopy too it's a good program for copying things when you've got to copy lots of data uninterrupted and fast.



  • @weaveR said in Shingled magnetic recording (SMR):

    Why not just use blagos version 1.0 cpu plotter its way faster now as long as you have a decentish cpu and it optimizes your plots straight away and works your way would take way too long.

    That program is really great, but where GPU plotter excels is when you can plot multiple HDDs simultaneously. For instance, I plotted 40TB in 24 hours (unoptimized, buffer mode) by connecting three GPUs to my mobo. I would not be able to plot that much in such a short time using CPU plotter unless I had multiple CPUs to work with. On average I had 41k nonces per minute on each GPU.

    I'm not sure how long it will take for me to optimize these 40TB, but I think I am able to set each drive to optimize simultaneously as well. Could someone please post an example output for running plotoptimizer.exe on 2 or more plots if that is possible? Or do we have to run several instances of the program in order to achieve that?

    @sevencardz Did you try to plot in direct mode with GPU plotter after turning on the write cache for the SMR drives? Or is that still a lost cause?



  • @weaveR have used blagos cpu plotter but over 1 tb it slows down drastically I haven't the high end cpu nor gpu tech so I use the Cer Jenror plotter. No plot needs to be optimised until used for several months. However the plotter that I use requires the plot to be made contiguous (defragmented). I know I can make smaller plots with Blagos ploter v1.0 fastly but I prefer to make larger ones as to be afk. lol. @Propagandalf I don't use the GPU Plotter as it is slower than the CPU plotter that I use simply because I have a low profile Nvidia PCI-E v.20 graphics card. Besides, I prefer a system lag free plotting experience so i can surf the net whilst plotting (or mine for that matter)



  • @RiskyFire @Burstde I think the higher failure rate may be due to the combination that these drives have no active cooling solution, huge 8TB capacity, and they use intensive read, modify, write operations due to the SMR architecture. I sent one back a few weeks ago after I wrote to it for a day because it was uncomfortably hot to the touch and the plastic had actually started to warp. Since then, I put these drives right on top of my computer's rear exhaust fan at the top of the case, which pushes air right into the bottom vents of the enclosures and I've had no issues with heat since.

    @Propagandalf I haven't tried writing in direct mode to the 8TB Expansion drives with disk write caching on, but I fear that I would still see the terrible single digit MB/s speeds during the secondary 'fill in nonces' stage at some point just due to the SMR architecture. The plots have already copied over from my 8TB NAS drives, so I'm not in a position to test, but if I get some more of these SMR drives in the future, I'll certainly give it a shot.

    I only used plotoptimizer once and I used the GUI, which is pretty simple, just point and shoot. However, it took so long because it has to rewrite the plots, and you need a place to write them to, so I just ended up just replotting the rest of what I had done up to that point in direct mode with GPU plotter.

    What kind of read speeds are you getting now? Given that your plots are not optimized and have a small stagger size, I'd be concerned about slower read times.



  • @Propagandalf Oh I do the same thing plotting multiple hard drives at once but instead of using multiple graphics cards I use multiple computers blagos cpu plotter doesn't lag the hell out of my main computer either which is nice unlike the GPU plotter but it does lag my mining PC but it still manages to mine while I'm doing that usually the plot scanning is just a bit slower.

    Plotting 40TB is pretty impressive in 24 hours though it took me 35 hours to plot one hard drive that was 6TB and I didn't record how long it took on the mining PC I think it took a few more hours with blagos plotter but they were optimized plots. My computers aren't really bleeding edge technology though.

    When I tried using the optimizer I think I remember it lagging my computers optimizing just one plot and the progress output was crazy but it's really annoying when the computers unusable I have to find something else to do lol. I do have a laptop though but it's got a bad keyboard that only works when it feels like to. I guess I could plug a keyboard in but I'd rather use a desktop.

    @ZapbuzZ Never heard of that plotter before. Yeah I probably wouldn't make too many plot files if I was you I think I've heard that slows the plot scanning down when your mining but I do have quite a few plot files myself on some of the drives from when I was learning. Unfortunately I can't replot larger files easily being the nounces would overlap.



  • @sevencardz said in Shingled magnetic recording (SMR):

    What kind of read speeds are you getting now? Given that your plots are not optimized and have a small stagger size, I'd be concerned about slower read times.

    I just watched task manager while the last block round was started, and clicked my way through the ten drives. It looked like I was averaging 23 MB/s on each one. Jminer says 238 MB/s average, which is probably the total amount that has been read in the program during a mining round, which seems consistent with 23 MB/s x10. My total plot size of 40 TB was read in 40.5 seconds, with 4x USB 3.0 controllers on duty. It is likely that my read speeds will increase dramatically when I get these plots optimized!


Log in to reply
 

Looks like your connection to Burst - Efficient HDD Mining was lost, please wait while we try to reconnect.