[RndTbl] RAID5 rebuild performance

Adam Thompson athompso at athompso.net
Thu May 19 21:27:38 CDT 2011

Well, I did say there was more than one variable involved!  25MB/sec is a bit slow, but I don’t know how efficiently that LSI chip is at managing bus contention, or how efficient the kernel driver for that chip is.  I do know from experience that 8-disk arrays are well into the land of diminishing returns from a speed perspective: RAID-1 on two disks or RAID-10 on four seems to be the sweet spot for speed.


Once your rebuild has finished, I would recommend doing some throughput tests (both reading and writing) on the array; something perhaps like:

# sync; time sh –c ‘dd if=/dev/zero of=/mnt/raidarray/BIGFILE.ZERO bs=1M count=1024;sync’

followed by

# time dd if=/dev/md0 of=/dev/null bs=1M count=1024

Those are both very naïve approaches, but should give you a feel for the maximum read and write speeds of your array.  I strongly suspect those numbers will be much higher than the raid re-sync rate, again, mostly due to write-flush barriers in the md code.


I’m interested in knowing how this ends, personally… please let us know.




(P.S. does anyone know how to avoid top-posting in Outlook 2010?)



From: roundtable-bounces at muug.mb.ca [mailto:roundtable-bounces at muug.mb.ca] On Behalf Of Kevin McGregor
Sent: Thursday, May 19, 2011 14:52
To: Continuation of Round Table discussion
Subject: Re: [RndTbl] RAID5 rebuild performance


raid6: using algorithm sse2x2 (2883 MB/s). So, 1% of that is reasonable? :-) Oh well, I guess I can wait until tomorrow for the rebuild to finish.

On Thu, May 19, 2011 at 2:12 PM, Adam Thompson <athompso at athompso.net> wrote:

Look back in dmesg output for the RAID module speed tests, notice which one was selected, and divide that number by 2.  There's your theoretical bottleneck on the CPU.
Take the minimum sustained disk/channel/controller throughput, factor in interrupt latency, device driver efficiency, etc. and make a rough guess as to the overall throughput.
Consider that md code seems to have a lot of write barriers for safety -  so even a rebuild will aspens much of its time waiting for the disk to sync().
All in all, I think your numbers are probably reasonable.

Kevin McGregor <kevin.a.mcgregor at gmail.com> wrote:

>I installed Ubuntu Server 10.04.2 LTS AMD64 on a HP ProLiant ML370 G3 (4 x
>dual-core/hyperthreaded Xeon 2.66 GHz, 8 GB RAM) and I used the on-board
>SCSI controller to manage 8 x 300 GB 15K RPM SCSI drives in a software RAID
>5 set up as a 7-drive array with 1 hot-spare drive. All drives are the exact
>same model with the same firmware version.
>It's currently rebuilding the array (because I just created the array) and
>/proc/mdstat is reporting "finish=165.7min speed=25856K/sec". Does that
>sound "right" in the sense that it's the right order of magnitude? I though
>it should be higher, but I haven't set up such an array before, so I don't
>have anything to compare it to.
>If it's slow, does anyone have a suggestion for speeding it up?
>Roundtable mailing list
>Roundtable at muug.mb.ca

Roundtable mailing list
Roundtable at muug.mb.ca


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20110519/76d8e015/attachment.html 

More information about the Roundtable mailing list