The Road To Cost Effective Bandwidth Management

You have your site, full of content, visitors come and go every minute and your Google analytics account shows that your site is very popular. You host this site on a great hosting provider that has a good price/quality value and you are sure that 40Gbs of monthly bandwidth is enough to serve your visitors. Or not?

Lets have a closer look to an example site’s pages:

1. Main page – Size: ~10kb
2. Contact us page – Size: ~ 7kb
3. Portfolio page – Size: ~18kb
4. Blog page – Size: ~14kb
5. Forums – Size: ~42kb

Now if we make a median size of the 5 pages we get : (10+7+18+14+42)/5 = 91/5 = 18.2kb. This means that if our site gets 1000 unique visitors each day with an average of 2 pageviews per user, our site’s monthly bandwidth would be 30*2000*18.2 = 1GB. This might seem great since we have 39GBs of spare bandwidth to use but lets add some images, CSS and JS files in our example. We add jQuery (minified), one basic CSS file sized 7kb and some images with a total size of 50kb on each page:

1. Main page – Size: ~10kb + 55.9kb + 7kb + 50kb = 123kb
2. Contact us page – Size: ~ 7kb + 55.9kb + 7kb + 50kb = 120kb
3. Portfolio page – Size: ~18kb + 55.9kb + 7kb + 50kb = 131kb
4. Blog page – Size: ~14kb + 55.9kb + 7kb + 50kb = 127kb
5. Forums – Size: ~42kb + 55.9kb + 7kb + 50kb = 155kb

So, we re-calculate our median: 123+120+131+127+155 = 656/5 = 131.2kb. Now let’s calculate our bandwidth again: 30*2000*131 = 7.8GB. This means that our site is able to serve ~5 times our visitors or 7 pageviews per user. For an example site with 5 pages this would not be a problem at all. For a site with more than just 5 pages, this is not enough.

Trying to minimize our site’s size, the basics:

There are some basic tips and tricks that we can follow, to reduce our page’s size and gain our bandwidth back.

Choosing the right image format

Most of the time your images are not optimized as they should. For example using .PNG format images might be a good choice for small images but it is not the case for larger images. For example a screenshot of our desktop (1440*900) when saved with the .PNG format is sized: 2.2MB and only 800kb with the .JPG format on top quality. So, try to use the best format that suites your needs with quality and size in mind.

Use CSS Sprites instead of image replacement

Having a beautiful design is always appreciated by your visitors. Not only your site looks better, but it is friendlier to the visitor’s eye. A very common design pattern is the use of rollover images that give the visitor a clue about a specific action available. In the past, most of the time this effect was implemented with a simple image replacement like this:

#rollover{
background-image:url(images/roll1.png);
}
#rollover:hover{
background-image:url(images/roll2.png);
}

Even though this technique works OK (some flicker might occur), each and every time a user activates this effect, a request to the server will be pushed to the server to get the roll2.png image.Even though the size of the page will be less than it would be with both the images loaded, it is likely that the user will hover at least once over the element that triggers this effect. The image is not cached until the effect is activated so you should count at least once the size of this extra image in your overall page size.

The Sprites technique is here to solve this issue. The principle is that you have one image file with all available states of the effect, you load this file once and you create the effect just by moving the image around. This technique not only saves bandwidth, it also makes the effect solid without flickering.  You can read more about using CSS sprites here.

Less nesting means less code.. smaller page size

This has been covered again and again but most of the time flames rise about TABLEs vs DIVs vs CSS layouts. You can have good and lightweight code with both tables and divs as long as it is not deep nested. For example a typical table layout would look like this:

<table>
<tr>
<td></td>
<td>Header</td>
<td></td>
</tr>
<tr>
<td>Left Menu</td>
<td>Content</td>
<td>Right Sidebar</td>
</tr>
</table>

and a typical tableless layout would look like:

<div id="wrap">
<div>Header</div>
<div id="main">
<div id="left">
Left Menu
</div>
<div id="right">Right Sidebar</div>
<div id="content">Content</div>
</div>
</div>

This is nearly the same size. The difference in the size will be visible in the number of nested elements that you will add to make the layout look as you want it to look. Keep this example in mind the next time you keep nesting elements on your pages.

Link to CSS files, don’t include their code

Sometimes, you might notice that many pages include their CSS styles in the page code itself. This might reduce the requests to the server for each CSS file but it adds all the style code to page’s code resulting in a larger page size. CSS files are cached from the browser as soon as they are loaded, keeping all your styles in one file will keep the browser with the cached version of the CSS until this file has changed, resulting in less page size.

Activate Apache mod_deflate

If you use Apache, you can activate the mod_deflate module which will compress files before sent to the browser. This results in smaller file size and guess what.. less bandwidth.

Read this article to learn how to activate the module.

Make your site’s size even smaller

Applying the above tricks will gain some precious bandwidth for us to use. The next tips will try to go deeper into the bandwidth hunt.

Keep Bandwidth thieves away

Bandwidth stealing is through hot-linking is something that can boost your bandwidth to the limits. What is hot-linking? Hot-linking is when someone places an image to his site directly from your server. If a hot-linker has a popular site, then your bandwidth will disappear at once. Keeping bandwidth thieves away is as easy as adding this snippet to your .htaccess file and keep your images, js and css files away from the thief’s eyes (Replace example.com with your domain):

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example.com/.*$ [NC]
RewriteRule \.(gif|jpg|js|css)$ – [F]

Use a Content Delivery Network (CDN)

In the example we showcased above, we saw that jQuery weights ~56kb and this is the minified version. jQuery comes with a lot of plugins for nearly any effect and functionality we can imagine and it is a must have tool for our sites. So using jQuery is something we can’t live without. At least we can load this library and many other tools and libraries from content delivery networks that exist just for that. This way, we will still use their functionality and we won’t have to host them.This article showcases a list of CDNs and we also suggest Google’s CDN for JS libraries.

Compress your files scripts and styles

You can also compress your JavaScript and CSS files in order to reduce the total size loaded from each. Compressing js and css files means that you create a new version of your files with comments and white spaces removed. This makes your files smaller and workable and even though they are not readable, this way is suggested for production sites. You can always keep your original files, make changes and compress them again.

Some free JavaScript and CSS compressors are:

• javascriptcompressor.com
• JS packer
• YUI Compressor
• CSS Drive Compressor
• csscompressor.com

Cache your pages

Each browser keeps a cached version of your pages and files for faster loading of pages. You can use this feature to lighten your site’s bandwidth needs. For example setting the expire value of a page to 1/1/2000 is not what you really need. Keeping your site cache friendly is something you should really reconsider and this article will explain how to cache in depth.

Reduce your page’s code size

OK. This might sound strange but a simple :

ob_start();
//...
//HTML here
//...
$htmlOutput = ob_get_clean();
$htmlOutput = preg_replace('!\s+!', ' ', $htmlOutput);
echo $htmlOutput;

can really reduce your page’s size. The above snippet strips any extra white space from your page’s HTML code (assuming you don’t have in-line javascript and styles with comments in your pages) in the case where mod_deflate is not activated and you can’t activate it on your server. Don’t fall for the “code is not readable” cliche. We need our code to be readable only on developing environments, not on production servers. Check Google’s code for example.

Keep Bots away or limit them

Keeping bots away is a resource saver. OK, we all love Googlebot,MSNbot and Yahoo slurp but these are not the only bots that come into our site consuming our precious bandwidth. The first thing we need to do is keep bots away from our server. This can be done with 2 ways:

First by using the SetEnvIfNoCase rule in our .htaccess file:

#First add as many bad bots as you can:
SetEnvIfNoCase User-Agent "^BotNameHere" bad_bot
SetEnvIfNoCase User-Agent "^AnotherBot" bad_bot
#and then add this rule:
<Limit GET POST>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>

Second way is by using the Apache’s modSecurity module:

#Keep bots away:
SecFilterSelective "HTTP_USER_AGENT" "^BotNameHere.*"
SecFilterSelective "HTTP_USER_AGENT" "^AnotherBot.*"

You should keep in mind that creating a large list of bots using the second method might result in a heavy load to your server even for small pages though. Here is a good list of bad bots to keep away.

We can also set a crawl delay for most of the bots right from our robots.txt file (add more bots as needed):

User-agent: MSNBot
Crawl-delay: 10
Disallow:

User-agent: Slurp
Crawl-delay: 10
Disallow:

Googlebot’s crawl delay can be set from the webmasters tools panel.

Be Paranoid : Always use a favicon

This is the last but not something you shouldn’t take into consideration. A favicon is a little image file (.ico or .png format) that most modern web browsers request by default. You can go away with not using a favicon but keep in mind that each time a browser asks for a favicon, a 404 error page is served for this file. In the case of a default Apache 404 page you get 234bytes (0.23kb) from your bandwidth which sometimes, might look smaller than a 1kb favicon file, Many sites use customized 404 error pages though. So, if a custom 404 error page weights 25kb, then you should surely reconsider the favicon issue.

Popularity: 3%

Related posts:

1. 10 Things to Consider Before Creating a Content Management System Having created more than 5 different content management systems, I...
2. CAPTCHA Problems, A Suggestion and Alternatives You have your blog ,contact form, application setup and your...
3. Use Google’s Power To Create Powerfull Search Engines (Part I) Did you know that you can use Google’s power to...
4. Use Google’s Power To Create Powerfull Search Engines (Part II) In our previous article, we learned how to create a...
5. Creating an Unobtrusive Javascript Image Callery Using YUI The Yahoo User Interface (YUI) is a set of utilities...