Lieven Cardoen

Two weeks ago I had a lot of problems in a project of mine. Often some clients couldn’t connect to weborb.aspx (actually 0.01% of the times). After a lot of searching we weren’t really able to find the problem. It seemed to me that maybe it was a network issue. The customer had just changed a lot of servers and improved their network. What previously worked didn’t seem to work anymore now.

We changed some things after which everything started to work smoothly again:

- When an error is thrown from the server or there’s a network problem we invoke the server call again with Credentials. We use authentication, so if at a certain moment in time IIS decides to recycle, the next calls to the server will fail. So we implemented a fallback system. If a fault occurs we authenticate again and try the call a second time. If the second time would also fail, then we give the user an Alert with some information what to do.

- When profiling SQLServer2008 we saw that there were millions of logs being written to the database by WebORB. We had configured WebORB logging with Log4Net and set the Filter in Log4Net on INFO. Basically everything that was logged was written to the database. This meant a lot of overhead. After having put the Filter on Warning connections to database reduced by a factor 100.

Configuring WebORB Logging with Log4Net (in global.asax):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

     void Application_Start(object sender, EventArgs e) 
    {
        log4net.Config.XmlConfigurator.Configure();
        log4net.ILog log = log4net.LogManager.GetLogger("Global.asax");
        log.Info("Application_Start(" + sender.ToString() + ", " + e.ToString() + ")");
 
        //Configuring WebORB Logging with Log4Net
	try
	{
		Weborb.Util.Logging.Log.addLogger("log4NetLogger", new Edu3.Util.Logger());	
                Weborb.Util.Logging.Log.removeLogger("default");
	}
	catch (Exception exception)
	{
		log.Error("Application_Start", exception);
	}
    }

- We profiled IIS7 with some tools that were proposed on StackOverflow. These tools were very interesting, but still I thought to get much more information. Strange that there’s not a professional profiling tool for IIS7. We used Administration Pack for IIS 7.0, the stackoverflow question is at link. The strange thing was that in 0.01% of the requests a 404 was returned. Even after having enabled Failed Request Tracing in IIS7, I couldn’t find the reason for the 404.

- Recycling happens automatically @ 23 hours. (this was allready done earlier when I had problems in November).

After these tweaks everything is working fine again. Those 404′s remain a mystery…

Ciao, Lieven Cardoen