CACHING *Bangs head on desk*

[Ryan]

*FLASHING WARNING* - Rant ahead!

I can't tell you how frustrated I am with this whole thing...

72 hours, 3 FULL DAYS...

Rebuilding... Search Index (post 2207425)

I've still got well over another million "posts" to go...

Listen I get it... this is "beta" but if it's beta and you make me pay for it at least let me try to provide some feedback on this cache system.

</rant>

I guess I kinda feel better... btw Floris thanks for backing me up on this whole thing.

 

[Floris]

You got a server there with a ssd drive btw to test on? I wonder how the read/writes affect this cache rebuild. The memory usage and cpu usage appears to remain quite normal .. so must be the hard drives doing overtime with silly queries.

 

[Ryan]

You got a server there with a ssd drive btw to test on? I wonder how the read/writes affect this cache rebuild. The memory usage and cpu usage appears to remain quite normal .. so must be the hard drives doing overtime with silly queries.

Actually I do...

I'll test it... but I'm testing right now on a 4x146 gig 15k RPM SAS setup in RAID 10...?

This should be plenty... if not XF has a big issue

 

[Hescominsoon]

I have a whittle board so i don't think i'd be able to reproduce....but 3 days? I'm curious how many are you having done per batch and how long of a pause do you have set? If this is a Linux box bring up top and see if you are stuck in i/o wait..that'll tell you if the hdd's are getting destroyed or not.

 

[Hescominsoon]

actually iv'e seen VB take more than a day for larger boards like yours..but 3 days does seems to be excessive..would need to see a few thing on the box to day for sure..

 

[Floris]

vBulletin does not have this cache rebuild. This is about full text indexes in innodb tables - and xf is doing search rather smart. However, the first time you need to rebulid such caches for big boards it takes a while, and server load goes through the roof.

 

[Hescominsoon]

right..so if we tweak the number of items it does AND the delay between runs you should be able to manage the load..of course the more time you give it that longer it takes.

 

[Floris]

i did 10 items per page, and 30 seconds delay so the load could go down
after 6 or 12 hours it go to the point where it continued the indexing around post 250,000 ..
the load went up to 20, then 30, then 40 .. and server became unresponsive so i took the web page down.

 

[melbo]

Strange, I did 135,ooo posts in about an hour for full upgrade and rebuild.

 

[Floris]

Yeah, smaller boards go through just fine. I've done many - but around 75 to 150k it is very wonky. And results seem to differ per server setup.

 

[Floris]

Beta5 seems to have made things worse. I can't even complete it now: It runs, and within 2 minutes:

It's a bug with Zend Framework, Mike wrote a work around for it. I guess we have to wait for beta 6 or rc1.