RSS

Monthly Archives: November 2012

Why tools that monitor and profile your website are so important

Over the last 12 months, we have been able to grow the unique visitor volume for BlogTalkRadio.com by 10X AND increase speed from an average of 10 seconds to 5 seconds per page load.

The architecture on how we scaled the site are a topic of another blog post. This blog post pertains to some critical tools that we used to drop the average page load in half even though volume increased dramatically.

If you are not measuring your site speed and the experience for your users, you are throwing darts against a wall in the dark. There are countless studies showing how page performance directly leads to increased visitor engagement and revenue.

The image below shows some of the famous stories where page performance increased revenue.

Our story is very similar. We have definitely seen an increase in revenue and visitor activity with faster page performance.

So what tools do we use to understand how our site is performing?

They key ones that we use are NewRelic and Dynatrace. We have always used Firebug and webpagetest.org but these only show you one off loads which often is not representative of your whole user base.

NewRelic shows us how each page performs, what part of the page life cycle is doing and even drills down to the back end and db calls.

Dynatrace has two parts to it.

  • There is a Free client piece that can be installed on your machine that gives extremely detailed information about how your page loads and about JS execution. It will show where your JS is running poorly.
  • There is an expensive server side that can show you down to the line of code how your pages are performing.

Using these tools, we worked release by release making changes to the hotspots that we found to increase performance.

These tools come in very handy when you release code to ensure that your changes did not cause an unanticipated performance issue. We had such an issue where we released something that was supposed to improve performance but under real conditions created a bug which greatly slowed down the site. We knew within an hour that we had an issue and released a hotfix. Without NewRelic, we would not have known that we had such an issue.

A HUGE win for us which showed the value of the Dynatrace Server tool happened the first day we used the  tool. We noticed that we kept seeing a helper class method call when reading the config variables taking longer than expected. Was taking 10-50ms each time. Some pages it would do this a couple of times.

We make a lot of use of config settings on almost every page of the site. The helper class that we were using was written years ago and used everywhere. Nobody has looked at that class since it was written.

Since Dynatrace was showing this class in the traces, I decided to look into it.

public static class myAppSettings
{
        public static string GetKeyAsString(string keyName)
        {
             try
             {
                 var config = WebConfigurationManager.OpenWebConfiguration("~");                 
                 var appSection = config.GetSection("appSettings") as AppSettingsSection;

                 try
                 {
                      return appSection.Settings[keyName].Value;
                 }
                 catch (NullReferenceException)
                 {
                     return keyName + " is missing";
                 }
             }
            catch
            {
                return WebConfigurationManager.AppSettings[keyName];
            }
       }
 }

Do you see the issue in the above code? This method is called every time we read a config value. This happens more than once per page load.

Issue was “var config = WebConfigurationManager.OpenWebConfiguration(“~”);”
This opens the config file. This code, every time a config variable was read, rather than reading from an in memory dictionary as expected, was opening the config file. Not good.

Since this is a static class, we just changed to open the file once when the class is instatiated as such:

public static class myAppSettings
{
        static Configuration config = WebConfigurationManager.OpenWebConfiguration("~");
        public static string GetKeyAsString(string keyName)
        {
             try
             {            
                 var appSection = config.GetSection("appSettings") as AppSettingsSection;

                 try
                 {
                      return appSection.Settings[keyName].Value;
                 }
                 catch (NullReferenceException)
                 {
                     return keyName + " is missing";
                 }
             }
            catch
            {
                return WebConfigurationManager.AppSettings[keyName];
            }
       }
 }

This seemingly small change yielded the following change in the NewRelic stats that shows the average time it takes on the server to generate the html.

You can see the dramatic drop in time on the day that we released the change. Finding silver bullets like this are very exciting.

This also explained a couple of odd behaviors that we noticed over the last couple of years that we were not able to understand until we started using these tools.
1) I always wondered why, on our site, when we pushed out a new config file, it immediately picked up any changes. I expected that it should only pick it up when the app pool was recycled. Now we know. It was opening it constantly.

2) A change we put in the header caused the web servers to jump in CPU. This change had included a couple of lookups in the config. This led us to think that reading the config was slow even though all of the articles that we read said it should be just reading from a dictionary in memory. Now we know why it was slow.

Some other hot spots that we found using these tools that helped to speed them up were around how we were loading third party widgets.
1) Facebook Comments needed to be made asynchronous and moved to after dom ready. We learned that you cannot move it till after the load event because it wont render.
2) Addthis share bar loads, twitter, google plus, facebook like, etc. Even though the JS that they provide says it is non blocking. It was to a point and loaded oddly on the page. By changing how it loads and moving it later in the page cycle, it enabled us to ensure the page was rendered before loading these third party tools.
3) On some of our pages we pull in json via an ajax request on dom ready and merge it with a template on the page to display information. By merely putting JS at the bottom of the page, it still executes before dom ready. therefore, some third party js was blocking what we wanted to do on dom ready.

Key learnings is not to just load third party code where they recommend. If we did that our pages would still be 10 seconds+. Experiment with moving them around and controlling when they load via $(window).load and $(document).ready.

NOTE: Be very careful of 3rd party code that has document.write in it. If you move that after the dom has been created, it will break most browsers. If Chrome sees it, it creates a new dom and blanks the page.

So without these tools (as well as firebug, Chrome Developer tools, webpagetest.org) we were flying blind as to how our site was performing out in the wild. Very often, it is different than in your lab setting and definately different than just looking at it as a one off page load via firebug or webpagetest.org. Don’t get me wrong, firebug and webpagetest.org are important tools. However, when you look at actual performance en-masse, in the wild, you see a ton of things that you normally wouldnt. Understanding what your site is doing, why it is doing it, and how your user’s experience is affected is critical to the success of your web property.

Advertisements
 

Tags: , , ,