Shingled magnetic recording (SMR)



  • @sevencardz Well legally they would have to state that the drives are refurbished. But would you mind giving me some more info on how your rig is setup?

    I can say that if you are using the usb 3.0 that comes with the external drives, this may be why your transfer/write speeds are so slow. I personally pulled the HDD's out of the external casings that converts sata into usb 3.0, and plugged them into my motherboard directly. (This improves speed a bit)

    You should also check your motherboard manual, because the bandwidth could be shared among other ports ether usb or direct sata.

    Lastly I would check CPU usage as if your cpu is maxed out from other tasks it may, queue data until it can process it.

    My Drives straight out of the box write @ 220MB/s (Peak) and 98MB/s when finishing writing the final plots. The read speed of the drives differs as I don't bother optimizing my plots. (35 sec for 30TB is good enough)



  • @AngryChicken Good point, I guess these drives just suck in general when it comes to write speed and I'll just have to live with it.

    CPU/RAM: AMD FX-8350, 16GB RAM
    MOBO: ASUS SABERTOOTH 990FX R2.0
    GPU: EVGA GTX 950

    USB3.0 can theoretically handle 640MB/s and that's twenty times faster than what these drives are currently writing at in my system. Shucking them and hooking them up internally via SATA is a good idea, but it sort of defeats the purpose for which I bought them, plus I've already filled up my internal SATA ports and drive bays. I have two of these 8TB Seagate Expansion drives and each is connected to a dedicated USB3.0 port on the mobo, which is an ASUS SABERTOOTH 990FX R2.0. I also have four 4TB Seagate Expansion drives hooked up to a Rosewill RC-508 4 Port Single Bus, so that card is actually sharing its USB3.0 bandwidth between all four drives and read speeds are well over 100MB/s on each of those, so the USB3.0 bus being the issue seems very unlikely.

    Granted, the system is actively mining 50TB with jminer and I do see the write speeds drop to about 20MB/s during a mining round, which makes sense because I am copying plots from drives that are actively being mined. The write speed picks back up to 30MB/s on each drive after the round is over, but it never goes higher. Between each round, the only other software running is the local wallet, so I can't think of any reason for the slowness besides SMR.



  • You are correct with seagate using refurb drives. The two 8tb internal drives I bought new have been replaced each 4x with refurbs under warranty. this last time everything has been solid. Every replacement was refurbished with 3 going back doa or dead after format.
    I bought a seagate ext 8tb plus and that has been rock solid too. That being said they are slower than a wd 2tb internal drive which I've a found a few blocks with yet most are with seagates. All together 26tb @ 30mb/s usb connection to 4core laptop probably look at improving read times next.

    I believe the plots start to degrade on the 8tb drives after 2 months



  • @sevencardz if you get card like this you can add more direct sata ports to your computer https://www.amazon.ca/IOCrest-Controller-Profile-Bracket-Components/dp/B00ESFEI2E/ref=sr_1_5?ie=UTF8&qid=1485375953&sr=8-5&keywords=sata+controller

    Also it's not how much each usb 3.0 port can handle it's how much the bus can. For example if you have two usb ports that use the same bus then the bandwidth will be that of a single usb 3.0 instead of two, this can worse with more and more usb 3 on the same bus.

    EDIT: so i checked you motherboard model and couldn't find details on the usb bus, but if you look right next to your ram slot there is a usb 3.0 header/input slot that you can try to see if this improves your speed.
    A General rule that I follow is if the usb ports are stacked they use the same bus.



  • would someone please try Xplotter on SMR's now already!-) Or do I really have to replott one of mine? I think it could be working great! please just tell me how it behaves! ;-)



  • @AngryChicken Yes that's true, I could buy all sorts of expensive hardware but none of that would solve the problem that these drives just have terrible write speeds. Maybe the first time you write to them they're fine, or maybe some are actually refurbs, or maybe Seagate puts desktop drives in some and archive drives in others. I don't know, but what I do know is the USB3.0 ports on the back of the mobo should not be sharing a single bus on a $180 premium board and even if they were, two drives running at even 200MB/s would never saturate that 640MB/s bus. That Rosewill card I posted runs four drives from a single USB3.0 bus and none of those drives ever had these ridiculous slow downs with reads or writes. I already connected the USB3.0 header to my front panel I/O and connecting the drives there makes no difference for write speeds.

    @nixxda I've never used XPlotter. :) I tried the GPU plotter on these 8TB SMR drives and write speeds were even worse than just doing a straight copy. I'm talking single digit MB/s... no joke.



  • I'll throw some more numbers at you guys, since someone will probably find this useful at some point:

    With GPU plotter in direct mode, I can write a 500GB plot file to two 8TB Seagate NAS drives in 3.5 hours. That's somewhere around 20,000 nonces/minute. The initial plot file structure writes at nearly 200MB/s, and then that drops down to about 120MB/s while filling in the plots with nonces. So to plot all 16TB, it takes about 2.5 days. Given a generous estimate that the SMR drives can push 10MB/s writing plots in direct mode, it would take them over NINE days to write the same plots. No, I have not fully tested that, but I did start the tests and saw the write speeds in win10's resource monitor first-hand. So even if it takes 3 days to plot to the NAS drives and 3 days to write those plots back to the archive drives, that's still 3 days faster and only half of the 6 days it takes is actually spent doing intensive plotting.



  • @sevencardz @nixxda @luxe and others who I discussed this with:

    My scenario
    I have been plotting 4x 8TB Seagate backup plus SMR drives simultaneously, using GPU plotter in buffer mode. Initially, I was getting about 180 MB/s write speed with ETA of 1d 10h to complete. While I was away and got back the next day it had reached 80 % completion, but write speed had dropped to 10 MB/s and took forever. If relevant, the drives had NTFS file system, 64k cluster size and 3 of 4 drives had completed.

    Solution
    Abort the plotting process, use plotschecker to repair and trunkate the file(s) that were incomplete and recalculate how many nonces are missing in order to fill the drive(s). Update your gpuplotgenerator batch file with your new values and restart your computer.

    When your system is back online, run gpuplotgenerator.exe through your .bat and you should find that your remaining plots will write at full speed again. If you choose not to restart, your write speed will be 10 MB/s.

    Theory about solution
    I think that when you restart your computer, you force your HDDs to flush their internal buffer or write cache. I believe that the SMR drive cannot handle a continuous 'hammering' of write requests, and will therefore write slower and slower until it reaches a very low speed.

    Maybe the same positive effect will be produced if we plot, say, 4TB first, wait for it to complete, and then plot another 4TB to fill the drive. I am currently testing this. My hypothesis is to not let it hit rock bottom speed before the plot finishes.

    Perhaps it is possible to use some kind of memory buffer cleaner together with GPU plot generator to circumvent this issue?

    Reference material and more theories for others to test
    http://goughlui.com/2015/06/07/slow-hard-drive-things-to-check/ please read this article, as it contains a perspective on SMR drives that I have not seen explained elsewhere yet.

    I checked my 8TB drives, and they actually had write cache disabled by default! According to the article, activating or disabling write cache has a huge impact on performance, as seen in these graphs:

    0_1485469926148_upload-8e2db24c-1435-4630-83da-73e1f8aefc40

    SMR write cache on
    0_1485469958192_upload-415e2048-830a-4ec6-b99c-c388a98da9ba

    SMR write cache off
    0_1485469972186_upload-eba63029-4c58-46b9-bc3a-f093faf02569

    I did not change my drive settings, so mine have had their write cache off all the time. I truly believe that performing the steps in the article will help with drive performance in general, when you are filling up your drives with non-Burst related items. Maybe they will also improve write speed when plotting and perhaps even mining, someone will have to try it out.

    Like I wrote in the 'Solution' section, I was able to fix the slow write speed problem by other means. I'm not sure, but I think GPU plot generator in buffer mode actually does many of the same things as what activating each drive's write cache will do.



  • OMG @propagandalf gets all the cookies. My problems were entirely tied to write caching being turned off!

    So, by default the expansion drives were set with write caching turned OFF on the disks. In my case, I was simply copying 500GB plot files from one drive to another and getting about 30MB/s write speeds at best. I turned write caching on, rebooted, and now the same plot files are copying at 130MB/s. Praise the sun!

    I also noticed that the 8TB drive I removed from the enclosure and installed internally via SATA had write caching ON by default. Seems like the OS tries to make some intelligent decisions by turning caching off on external drives, but keeps it on for internal drives. I assume this is the reason @AngryChicken had no problem with write speeds.

    TLDR: Make sure disk write caching is turned ON for every drive you plot or you will have a bad time!



  • Great information with real world examples. Hey found a 6 min block And blago read 21mb/ s. Can't wait to test it tomorrow on my new build



  • @Propagandalf @sevencardz Yup this is the case on my end. Never had a problem, also when others pull the drives out of the case and install directly they have better experiences. Maybe it is a system default, but good to know itès just a setting.



  • My test plots have now completed. By plotting 4TB files onto 10x SMR 8TB drives (BUFFER mode), with 1.25GB stagger on each and default cluster size, they completed without even slowing down. I still have not activated write cache on each drive, so no idea if that will improve plotting even further, but I don't think so in my case as 180 MB/s has been stated as a typical top end write speed according to various sources. I guess it should be tested whether it can increase, but I'm pretty satisfied with this, as I have used 24 hours to plot 40TB. I could probably plot even more simultaneously until I reached a bottleneck somewhere. I used 3x MSI rx470 8GB GPUs (1 for quad HDDs, separate command window instances), and 4x USB 3.0 controllers. My AMD FX 8350 CPU had about 35-65 % load during plotting, and I had 32 GB ram total capacity, whereas 12.5 was reserved for stagger.

    However, @sevencardz activated write cache and was able to copy files much quicker. Underneath the option of activating write cache there is another option too: "Turn off Windows write-cache buffer flushing on the device". Apparently this is supposed to make drive write speed even more efficient, but with increased risk of something happening that I cannot remember right now.

    My theory, based on everything I have read and been told, is that default cluster size works best with SMR drives, and that creating single plot files that are over 4TB will eventually overload the SMR drive somehow by hammering it with write requests. By having a low stagger such as 1GB, you avoid long, continuous write sequences that may be the cause of the overload, as the drive only writes for a few seconds, gets time to rest and pick up strength (maybe clearing its buffer too), while GPU pumps out new nonces, and the process repeats itself. Many people will think this sounds daft, but it could be the reason. It would also be interesting to find out to which degree GPU plot generator imitates the function of the drive write cache with its buffer plot mode.

    In a few days I will start to plot the remaining 4TB on these same drives and see if I get the same result. If you don't hear anything from me, you can assume it worked. :)



  • @Propagandalf Holy maroni!-) 40TB/day!!

    "Write cache off" is standard for USB-Disks. So not to loose Data when someone just pulls the USB-Cable out! (power loss on non ex. powered disks)

    I never bothered with the second setting. Someone now what it means?!

    ps. could you add that your using "buffer mode"! (I got really jealous there for a moment until I remembered !-) (but I still am!!-)



  • @nixxda said in Shingled magnetic recording (SMR):

    ps. could you add that your using "buffer mode"! (I got really jealous there for a moment until I remembered !-)

    lol, I did type buffer mode actually, but maybe it got lost inside the parenthesis. =) 40TB per day is still nice, though!



  • @propagandalf Plotting to ten 8TB drives in parallel... that's some real next level stuff right there.

    And heck yeah - we are proud members of the FX-8350 Burst miner club. AMD4LYFE.

    To be clear, I kept the "Turn off Windows write-cache buffer flushing on the device" unchecked for my copies.

    I also think you're right about the stagger size and/or giving the drives time to catch up, especially if you're writing to ten of them because that'll spread out the 'write load' more efficiently. I also noticed that these SMR drives reportedly have a feature where they can boost their writing speed temporarily. I actually saw this happen while copying to the 8TB external drives because during each mining round, the write speeds would drop back down to 30MB/s (due to the fact that the internal 8TB drives were being monopolized by the miner for the round I assume), but right after the round was over, they would boost up to almost 200MB/s for about 20 seconds before they settled back down to 130MB/s again. So just 20 seconds of semi-down time gave each drive a significant temporary boost.



  • @sevencardz Thanks! AMD rox ;)

    I did not know about this temporary boost. Is it a setting, or is it just a built-in feature that is always on?



  • It's strange I guess I got lucky lol. 4 of my drives are the Seagate 8tb and they work perfectly fine. Never had an issue with them up to date :D and even sometimes a better read time than the non Seagate drives I have



  • @propagandalf The storage review here talks about "burst write activity". Of particular interest are these lines:

    "In our test we saw, as expected, the SMR drives took much longer for a traditional full backup, averaging 30MB/s. However we saw sustained read speeds during a 400GB VM recovery in excess of 180MB/s, which is really the core metric. Given the low cost/TB, the drives do very well here if the backup admin can get a little creative. Design your backup window to work with the lower sustained write performance (or design it to fit inside the burst write window completely) but still have your data ready at your fingertips without compromising restore speeds."
    http://www.storagereview.com/seagate_archive_hdd_review_8tb

    I'm wondering if they had turned write caching ON during their full backup (assuming it was off) if the speeds would've gone from 30MB/s to 130MB/s like mine did. Or perhaps it only helps if you are writing massive 500GB files. That basically means these drives are almost as good as normal PMR drives if you can limit the workload to huge sequential writes - which I do by writing all the plots to NAS drives first and then copying them over.

    Also, it seems that if you stagger your write operations so that you're not hammering the queue of write operations on the drive, then you take advantage of the burst speeds more effectively. Of course, this seems to work best if you have other drives you can write to while others are catching up based on propagandalf's results. They don't mention what burst speeds they got, but they do say "burst write speeds are in-line or surpassing traditional HDDs".

    @Zeus If you plugged your drives in to the mobo via SATA or if you were lucky enough for your OS to automatically turn write caching on for the disk, then I'm not surprised you've never had an issue. I'm pretty sure the OS intentionally turned off write caching on the disk for me as a safety feature to avoid loss of data in situations where the user unplugs the external drive without properly disconnecting it in Windows. Of course, as long as you don't plan to unplug your drive in the middle of a write operation, then you don't need or want such a safety net.



  • I had one of those 8TB Seagate Expansion drives and it failed after just a few days. I guess those drives are just terrible quality. I've had no problems with the 4TB / 5TB Seagate's.



  • This post is deleted!

Log in to reply
 

Looks like your connection to Burst - Efficient HDD Mining was lost, please wait while we try to reconnect.