[BBLISA] Ethernet Mixed Fram...or NFS server sagas

[bblisa at richfox.org]

Hi,

...

On Thu, 17 Nov 2005, Doug Mildram wrote:

> doug.mildram> far off the intended topic of enet link layer stuff,

I think these questions are important and I appreciate them since I'm 
still in  the planning/implementation stage.

> do you know anyone who uses a redhatlike kernel for high performance
> NFS server?

I don't really know anyone. That is, I don't really know anyone in 'the 
business' who are doing things like this--except for maybe the bblisa 
folks (hopefully), which is why I asked.

> To augment my "critical" NetApp data, I brought up (briefly)
> some redhat7 systems as NFS servers and......man....they were terrible.
> (symptom: even with enuf memory (1-2gb), the OS crawled to a near-death
> slowness, with the load average running up to 20  ("client" jobs
> never got my rh7 boxes load avg over 2).

> I admit I didnt try hard enough with a 2.6 kernel. (at the time,
> I had to try building 2.5 myself, and it just didnt seem to help.)
>
> I'd be inspired to hear that Fedora has a better NFS server,
>  (and I kinda thought fedora's focus was homeuser/multigizmo...)

Yes, I have always considered Redhat in that regard too. But I doubt that 
RedHat Enterprise Linux is intended as a home user type system, and as I 
understand it, Fedora Core are the release candidates for RHEL.

On that note, Yellow Dog Linux (for the PPC architecture) is built on 
Fedora and is intended as a high performance system (and in fact, that's 
what I'm replacing, a dual g4 Xserve running YDL 3 running kernel 2.4.20. 
It's done a decent job but it's old and becoming unstable).
At times we've had over 50 machines reading/writing intensively to an 
exported  1.2TB filesystem from this box. (2 clusters and a number of 
other standalone servers). We never considered it too slow, but we had 
no relative experience to compare it too. Now we will.

> To solve my need, I converted (24) ancient P500 desktops to freeBSD4.8
>  nfs servers (each with 1 disk and 1 100mb card).  And they rock.
> FreeBSD5.4 came along later/recently, and it is equally good.
>
> My situation is unique where
> a) this particular data is somewhat disposable, and
> b) I spread/allocate 1-2 users (engrs running HW simulations, many GB/day)
>     average per NFS server..so it's a pain in the * approach to
>       distributing the load.

I prefer FreeBSD too. My choice of Linux was really for two reasons:

All of our systems are linux (except for the special 'sysadmin' boxes 
which are FreeBSD). The staff here who know their way around 
unix compatible systems know linux because so much of their research 
software is written for it. If I can get good performance out of linux 
then if I get hit by a bus these folks won't be left high and dry with a 
system that *looks* familiar but is different enough to cause 
much confusion.

Although we are only going to be exporting two separate 1.3 TB filesystems 
I can easily see us hitting grant jackpot and buying one of those fancy 7 
TB Xserve RAIDs. I was under the impression that there are still 
outstanding issues with filesystems over 2TB on FreeBSD. 
http://www.freebsd.org/projects/bigdisk/index.html
Have you ever worked with large filesystems > 2TB on FreeBSD?

Ah the heck with it. I'm going to use FreeBSD. I think on point #1 if 
these folks find  themselves in the position where they are looking at 
low level stuff on  this box, then the learning curve is going to be the same 
whether it's linux or FreeBSD. That is, they can get around a UNIX 
compatible system just fine, but don't know the low-level stuff. Besides, 
the FreeBSD community provides unparalleled  support and if you're gonna 
have a problem with a community supported OS, it might as well be that 
the community is really supportive of. On point #2 I think I'm
being premature on the worrying, but at the same time, I have to be
accountable for the decision if we run into problems with expanding.

Conveniently, I happen to have the FreeBSD 5.4 disks in my briefcase/bag 
thingy. It's like a sign...

> Hope this helps, will enjoy your gathered feedback...plz do post it!

I will.

If anyone thinks I'm making a terrible mistake, I'd appreciate the 
criticism...

Thanks,
Rich.