Another factor is the number of files in a folder. As the number of files per folder increase the time required to create a new file in that directory can slow down significantly.<div><br></div><div>Also what protocol are you using to send the files from one server to the other?</div>
<div>--</div><div>David<br><br><div class="gmail_quote">On Thu, Jul 22, 2010 at 2:43 PM, Ian Stokes-Rees <span dir="ltr"><<a href="mailto:ijstokes@crystal.harvard.edu">ijstokes@crystal.harvard.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im"><br>
<br>
On 7/22/10 2:34 PM, Rudie, Tony wrote:<br>
> As a couple of people have said, it's not the gear that's the<br>
> problem, it's all those little files. But following in your<br>
> footsteps, doing the same calculation based on 1.3 million files, we<br>
> get:<br>
><br>
> 50% done = 650K files, in 20 * 3600 seconds = 9 files per second.<br>
> That seems low as well. I just unpacked a tar file with 1000 files<br>
> in it in 3 seconds.<br>
<br>
</div>I should have included those numbers in my first email. Yes, I've<br>
looked at those, and they seem stupidly bad, like the system is running<br>
orders of magnitude slower than it should.<br>
<br>
What keeps frustrating me is that no one can describe to me or point me<br>
at a web page that describes the tools and techniques required to figure<br>
out what is going on.<br>
<font color="#888888"><br>
Ian<br>
</font><br>_______________________________________________<br>
bblisa mailing list<br>
<a href="mailto:bblisa@bblisa.org">bblisa@bblisa.org</a><br>
<a href="http://www.bblisa.org/mailman/listinfo/bblisa" target="_blank">http://www.bblisa.org/mailman/listinfo/bblisa</a><br></blockquote></div><br></div>