[BBLISA] Fileserver opinion

[Toby Burress]

On Fri, Aug 13, 2010 at 03:11:59PM -0400, Jeff Wasilko wrote:
> On Fri, Aug 13, 2010 at 02:56:49PM -0400, Toby Burress wrote:
> > On Fri, Aug 13, 2010 at 02:42:12PM -0400, David Miller wrote:
> > > What does your zpool look like?  Ideally if you're using RAIDz or RAIDz2
> > > then you should be using multiple RAIDz sets in the pool.  This way IO is
> > > stripped across the RAIDz sets and any degradation, and recovery, should
> > > only involve the smaller RAIDz set.  Which should be relatively quick
> > > depending on the size and type of drives involved.
> > 
> > The server that it taking a billion years to resilver does in fact have
> > 15 disks in one big raidz2 pool.  The other server has a single pool of
> > three raidz2 arrays of 8 disks each, so hopefully that will yield better
> > recoveries.  Although if the bottleneck is reads, then wouldn't it be
> > faster to read from 14 disks than 7?  And if the bottleneck is just
> > writes, then wow, I need to buy some different disks next time.
> > 
> > Since the load on the machine is 3, and it's doing nothing but
> > resilvering, I suspect the bottleneck is actually the CPU.  I don't
> > know a ton about the implementation of ZFS, but I do know it checksums
> > every block.  It would be insane for it not to verify those checksums
> > while resilvering, and perhaps it even recomputes them while writes them
> > to the new disk.
> 
> Did you lose 1 or 2 disks in the raidz2 pool? 

One disk.  I actually have replaced two disks at a time and it was about
the same.