Plotting and Mining on Amazon Cloud Drive
-
@Glen yep
-
@iKnow0
Thanks for the math. Looks like it is spot on!
I have a 12MB/s connection and a 2.9TB plot and it takes about 62 secs to read so not far off the figure you quoted if extrapolating out to 30TB.
I think that unless you have a Gbps connection (Who has one of those?) then the plots are probably going to have to be no more than around 10TB max as you probably wont get to finish reading them when the next block appears.
-
Any new experiences with the mining on ACD?
-
Hey guys - do you have any experience with actual minig process on ACD?
I have massive ACD account with ~13TB of plots. I'm mining with my EC2 instance, but unfortunatelly I'm observing very low performance (0.5-2 MB/s) that is unacceptable.
What environment / programs for ACD mounting have you been using during mining?
I've tried with acd_cli on Linux, and NetDrive2 on Windows (on Microsoft Azure instances), but performance on all of them is similar.
-
@Axadiw I am mining under windows on an ec2 t2.micro free account.i am using net drive and am getting anything from 2 to 10MB/shut on a 4TB set of plots.
The problem is that the network performance on this free instances "Low to moderate" according to AWS. This in practise means around 70Mbps but fluctuates depending on how much you are downloading. So as your plot size increases so does the amount your downloading which then proportionally slows the network performance.
I did try the same thing on a Premium VPS service (without a network performance penalty)but found that I got similar performance so not sure what is going on there and can only suspect that the speed may be throttled from Amazon Cloud?
-
Hmm, is it performance is decreased because of that I've been plotting with very small plot files (3-10GB files, together it will be ~3000 separate files) ?
Is it possible to increase parallel reading in miner somehow?
Or maybe is it possible to merge smaller plotfiles somehow?
-
@Axadiw
Having smaller plot files will affect things somewhat. My plots are 24GB in size as larger files fail to upload consistently.
If there was some way to merge them that would be great but I don't think any tool exists? Also if it did, I am assuming that the data would need to be downloaded,joined and then uploaded as a larger file which wouldn't work
-
hmmm, you're right.
BTW. in which region are you hosting your ec2 instance?
-
@Axadiw
eu-west-1c
I wonder if you have had your account locked yet?
That amount of data over that connection speed would mean that you are pretty much downloading constantly.
Have Amazon locked your account as a result of this?
I have had it locked several times now
Easiest way to unlock it is by using their CS chat.
-
I have really enjoyed reading through this project! I have a question though @RichBC in your batch file you list Optimiser where can I find that?
-
@gersey https://forums.burst-team.us/topic/26/plot-optimizer-v1-6-with-gui-1-0-3-j6jq-win
If you go up a couple of levels from this link you will find all the Burst related Software.
Rich
-
@RichBC Thanks for the help. Have you started to mine yet or are you still plotting?
-
@gersey I will be plotting until the end of next week. Then the hire of the VPS comes to an end and I am going to start mining experiments in earnest, without the risk that Uploading and downloading at the same time causes. Currently at about 30TB
Still big risks on can you read fast enough, will the read error problems stop the miner, will Amazon shut it down etc. :-)
Rich
-
@RichBC
Whats your connection speed like Rich? Upload/Download speeds?
Will you be mining from your home PC or using an Amazon AWS instance or other?
-
@machasm Home (2) is 80Mb/s Down 20Mb/s up. Not fast enough I know, but that is where I will do my first tests. Next test will be from an EC2 Free instance which about 3 x that speed, still not fast enough but there are actually other problems, as listed, to get out of the way before speed is the only issue. After that I am unsure I am uncertain if there is a cost effective solution to 1000Mb/s
Rich
-
I've been following this thread for about a week and started doing some tests myself with the free Amazon AWS (t2.micro).
This thing, however insanely fast on the upload side of things does not have enough time-credits to plot the files in a sustained manner. There has to be some inactivity to bank up CPU time...which is not what we're looking for to upload the plots obviously. Might work just fine to mine afterwards though. In case that goes south, a good candidate for the mining phase might be (depending on the necessary bandwidth) a small Linode.com VPS.So I got a VPS at NeuPrime.com ... the 4€/month promo for: 1 core 2.3Ghz, 1Gb of Ram, 30GB SSD which is about the same as the AWS.
One problem I have, is with NetDrive, more specifically the cache it is uing:
After a file is uploaded it stays in the cache for a very long time (as described here) which off course ends up saturating the disk at some point depending on the size of the plots that are being uploaded.
I suppose you got around this in one way or another ?
-
@Kahana82
I am not using Netdrive to upload. I am using rclone.
Netdrive is only being used when mining. As you have stated the t2.micro instance isn't much good for uploading but is sufficient for mining.
I haven't observed any problems with Netdrive when mining due to cache?
-
@Kahana82 I have set the Cache size in options down to 1GB, but like machasm only use it for Mining / Download.
Rich
-
@RichBC I know this is a stupid question but in the Batch file for
set account= I am guess that is your burstcoin Numeric Account # ?
-
@gersey Yes correct you will of course need to replace it with your numeric ID :-)
Rich
