Trying to diagnose and fix Sitefinity stability issues

Hi,

The main problem we experience with Sitefinity, and which prevents us from launching our sites publicly, is related to stability. Our sites are not stable, and tend to get slow or crash. We couldn't find anything useful in the logs, and are not sure what to do at this point. It seems to me that our sites get more and more unstable as more content is added on them. In the beginning, when the sites were nearly empty, we didn't have any notable perf issues.

For the info, we currently have data cache disabled, as our sites are deployed on multiple servers (although we experience issues whether the data cache is enabled or not). By the way, we're on Sitefinity 4.3, and from what I saw in the change log, it doesn't sound like Sitefinity 4.4 makes things better on this particular point. Correct me if I'm wrong. I also wonder whether Sitefinity 5.0 will help or not.

Anyway, as I have no idea what to do at this point, I thought we could tweak some IIS settings to try and make things better, on the web site or application pool level.
Could we increase the number of worker processes? Recycle worker processes more often? Disable or increase the time of the rapid-fail protection? Anything else?

Any other ideas to debug or improve the situation?

Please advise.
Thanks.

Dear Thomas

I was wondering the following
a) how many pages
b) how many users on site same time
c) how many users in backend on same time
d) multilingual
e) custom modules
f) RAM available on server

When you say get slow is it only on initial access of the site or every page load (I warm up the sites when I have not constantly users on the site)

When you say crash what would that be?

Can the site be accessed from outside so we can test it? Sometimes the bottleneck can be a provider, location.

Markus

Hi Markus,

a) How do I find out? In SF 3.7, it was written on the Pages index, but not in 4.x... Do I have to query the database manually? If so, what's the query?
b, c) Not sure if you mean on a single server (our sites are spread across 6 servers in production), or in total. Anyway, it's really few. Maybe 5 at most per server, and 10 at most in total. Half of them being on the backend too.
d) We have 2 sites, one is multilingual, the other isn't. Let's focus on the site that isn't multilingual, as it's the most critical one right now.
e) A few simple custom widgets (like to embed Flash, or to have a special sub navigation).
f) 4 GB. But they're mutualized servers, several applications run on them side by side.

"When you say get slow is it only on initial access of the site or every page load (I warm up the sites when I have not constantly users on the site)"
Not just initial access. And it seems somewhat random, sometimes pages are served fast, sometimes it gets slow. We can't figure out why it slows down at times.

"When you say crash what would that be?"
"The system is restarting" screen, or the server just doesn't answer and serve pages. We also had cases when the backend seemed to be dead (it just kept loading forever), but the front still worked properly.

"Can the site be accessed from outside so we can test it? Sometimes the bottleneck can be a provider, location."
No, this is an intranet site.

Thanks for your help.

Dear Thomas

a) An estimate would have been enough 20 pages, 100 pages or 700 pages (shared content)
b) so low traffic. Do you think that you have periods of 3 and more minutes where nobody is on the site?
d) fair enough to concentrate on the simple ones
e) ok
f) can you monitor how much RAM the simple not multilingual site is using.
I have a seperate app_pool for all my sites on the server and therefore can monitor it.

When the app_pool recycles then it the site needs to fire up again which can take time. Also when maybe you have not much traffic the pages will not get cached in RAM (but then again you said you turned caching off - so thats for sure no speed gain)

Just out of curiosity - are you expecting more traffic or why is it spread across servers. Maybe one Server with chaching would improve stuff. As mentioned before I have a task running on my server which checks the site every 2 min if it response and therefore seems to keep the site warm.

Have you used fillder2.com to see what is going on.
Here is a link which should help you find some more optimization possibilities.
http://www.sitefinity.com/devnet/forums/sitefinity-4-x/general-discussions/memory-use-and-recycling.aspx

Post is over 1/2 year old, so some stuff might not be true anymore

Other if you have done all this and the site runns still slow then I assume opening a ticket might do the trick.

Markus

Hi Markus,

I've been busy lately, and somehow missed the notification mail telling me you replied.
Thanks you for your reply.

a) Let's say around 200 pages. A few dozens of page templates, though.
b) Possibly, since there are several servers. But the slowness issues are definitely not due to the application starting up or cache issues, in case that's what you had in mind. Because it can happen as you simply refresh the page after having loaded it.
f) I can only do that on the test environment (otherwise, I have to contact another team). Right now, it fluctuates between 260 and 300 MB. The max we've seen was around 500-600 MB.

We also have a separate app pool for each of our Sitefinity sites.
Also, I turned the data caching off, but there's still another cache for the HTML code in Sitefinity, from what I understand.

"Just out of curiosity - are you expecting more traffic or why is it spread across servers."
Yes, we expect more traffic. Also, it's a mutualized farm.

"Maybe one Server with chaching would improve stuff."
Data caching must be disabled in a multi-server environment, though, according to the Sitefinity team. It's actually disabled automatically if you enable load balancing.

By the way, we found out we had a certain page that caused issues. It just crashed the app when viewed from the front. From the back, we had a "Cyclic child/parent relationship. A control/layout cannot be the parent of its own parent." exception. After remaking that page, this helped the stability.

We still have stability issues, though. Looking at the Error log, we have exceptions such as "BackendError : Telerik.OpenAccess.RT.sql.SQLException: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding."

One of our contributors also had this error while using the back office. I brought this up here.

Thanks for the link, I'll check it out.