I’ve recently been working on one of my ASP.NET Sample Applications that I use for demo purposes. When doing demo’s I usually run scheduled synthetic web transactions in the background to simulate some user load. One of demo’s last week took a bit longer than expected. So I ended up execting more load over a longer period of time than I’ve done in the past. While working with my app I discovered a “new behavior” – IT STOPPED RESPONDING.It turned out that I ran out of connections in my database connection pool which caused every additional request to run into a timeout. The timeout manifested itself as an exception that was thrown by the ADO.NET Driver after waiting 90 seconds for a free connection from the connection pool. The following PurePath shows a request that ran into the timeout:
Tracking down the .NET resource problem
In order to analyze this problem I created an extended memory dump that allowed me to see all objects on the heap including the references to other objects. With that information I started looking for my connection objects which were obviously leaking as I couldn’t get any additional connections from the connection pool.
From this view I can either analyze the reference tree of a single instance of my Connection object or I can get an aggregated overview of the reference tree of all instances.
The reference tree showed me that 55 instances of my DBConnection class held all the references to the Connection – which was not suprising. It was however surprising that some of my DBConnection objects were referenced by the ASP.NET Session State (so somebody put in the object in the Session State object) and others (the majority) were referenced by my business class implementation.
The business class implements Web Services that requires an ASP.NET Session. So – for every session that was opened the .NET Framework created a business class instance. Each business class instance opened a connection at the beginning and didn’t close it till the object was disposed – which was when the session expired.
My implementation basically required one open database connection per active session – which is of course a stupid thing to do – but – things often seem more logical after you identified a problem than when you actually create the problem.
Using Dynatrace’s capabilities to analyze objects on the heap – including the complete reference tree allowed me to track down the resource problem that I had.