|
Introduction
In today's world of Web services and RSS feeds, it seems like every application
you build needs to pull in data from somewhere. Luckily, ASP.NET makes
retrieving data from a remote server via HTTP extremely simple... maybe too simple.
Many applications are brought to their knees because for every request
that comes in, multiple requests go out.
The first time I found the need to display data from a remote server on a web page was
when a client I was building a web site for wanted to put a stock quote on their home page.
Not wanting to disappoint them (they were a big client after all -- they had stock!), I
set out to figure how exactly I was going to do it. When I finally got the script working
(which was no easy task -- anyone else remember the Internet Transfer Control and WinInet?),
I was disappointed at how slow it was. Between the slowness, the fact that the web server
had a tendency to crash when it made requests, and the fact that I was still doing a lot of
my work via dial-up, I made sure that those stock quotes were cached for a good long while!
Retrieving Data from a Remote Server via HTTP
I'd been using the same script to retrieve HTTP data in classic ASP for years
without it ever giving me any trouble. So, when I needed to write a .NET version,
I figured I'd just convert my existing script and keep using it.
For those who are interested in seeing the classic ASP version,
I wrote up it up in a short article a few years back:
A Simple Method for Caching HTTP Requests
Well it turned out that there wasn't much in the script that didn't need to change.
Since we're now using the .NET Framework instead of COM Components, the whole HTTP section
needed to change, and since .NET now has built-in caching facilities, I no longer needed
to resort to using an Application variable in order to get in memory caching.
So much for converting the old script... it was time to start fresh.
The first step is obviously to just retrieve the data from a remote server via HTTP.
The process really isn't that complex, but it can look that way because of the
number of objects involved.
http.aspx
<%@ Page Language="VB" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script runat="server">
Sub Page_Load(Src as object, E as EventArgs)
Dim strRssFeedText As String
' Call the GetWebPageAsString function to retrieve
' the 15 Seconds RSS feed as text.
strRssFeedText = GetWebPageAsString( _
"http://www.15seconds.com/dyna/15secs_rss.asp")
' HTMLEncode the RSS feed and display it in the browser.
litTextRetrieved.Text = Server.HTMLEncode(strRssFeedText)
End Sub
Function GetWebPageAsString(strURI As String) As String
' Declare our variables.
Dim objWebRequest As WebRequest
Dim objWebResponse As WebResponse
Dim objStream As Stream
Dim objStreamReader As StreamReader
Dim strResponseText As String
' Create a new request.
objWebRequest = WebRequest.Create(strURI)
' Get the response to our request as a stream object.
objWebResponse = objWebRequest.GetResponse()
objStream = objWebResponse.GetResponseStream()
' Create a stream reader to read the data from the stream.
objStreamReader = New StreamReader(objStream)
' Copy the text retrieved from the stream to a variable.
strResponseText = objStreamReader.ReadToEnd()
' Close our objects.
objStreamReader.Close
objStream.Close
objWebResponse.Close
' Set return value.
GetWebPageAsString = strResponseText
End Function
</script>
<html>
<head>
<title>HTTP Request Sample</title>
</head>
<body>
<pre>
<asp:Literal id="litTextRetrieved" runat="server" />
</pre>
</body>
</html>
When run, the script above makes a request to "http://www.15seconds.com/dyna/15secs_rss.asp", which
just happens to be the address of 15 Seconds' RSS feed. At the bottom of that feed is a comment indicating
the current time at the server... and yes... it's usually a few minutes off.
Caching Data from a Remote Server via HTTP
I first learned how big a deal caching is when I was trying to upgrade
an old computer. I had just gotten a new hard drive and had installed it
along with my old one in order to copy over all my data. I booted off of
a plain old boot floppy and started using XCopy to transfer the files from
one drive to the other. It was painstakingly slow. So slow that I thought
there must be a problem with the new drive. Well after several hours of
troubleshooting and a few phone calls later, I discovered just how big a
difference SMARTDrive
can make. I've been a huge fan of caching ever since.
So why is caching of HTTP requests so important. Because they're one of
the slowest things you can ask a web server to do. HTTP requests are
even slower then most database requests. At least your database is usually
located on a local network. HTTP requests can be sent anywhere on the
Internet and there are plenty of things that can go wrong before you get
a response. First you need to resolve the IP address of the server you're
trying to contact via DNS. Then your request needs to get to the server.
The server then needs to respond, and finally you need to receive the
response. We take for granted a lot of the steps that go into making
a successful HTTP request because of how reliable the Internet has become,
but there are a number of things that need to happen in order for you
to get a response and they don't always all happen quickly.
http_cached.aspx
<%@ Page Language="VB" %>
<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>
<script runat="server">
Sub Page_Load(Src as object, E as EventArgs)
Dim strRssFeedText As String
' Retrieve the entry from the cache
strRssFeedText = Cache("15SecsRssFeedText")
' If the data retrieved is null then we
' get it from the remote server and insert
' it into the cache for future calls.
If strRssFeedText = Nothing Then
' Call the GetWebPageAsString function to retrieve
' the 15 Seconds RSS feed as text.
strRssFeedText = GetWebPageAsString( _
"http://www.15seconds.com/dyna/15secs_rss.asp")
' Insert the text into the cache. I'm setting it
' to expire 15 minutes from when we add it.
Cache.Insert("15SecsRssFeedText", strRssFeedText, _
Nothing, DateTime.Now.AddMinutes(15), _
System.Web.Caching.Cache.NoSlidingExpiration)
End If
' HTMLEncode the RSS feed and display it in the browser.
litTextRetrieved.Text = Server.HTMLEncode(strRssFeedText)
End Sub
Function GetWebPageAsString(strURI As String) As String
' Declare our variables.
Dim objWebRequest As WebRequest
Dim objWebResponse As WebResponse
Dim objStream As Stream
Dim objStreamReader As StreamReader
Dim strResponseText As String
' Create a new request.
objWebRequest = WebRequest.Create(strURI)
' Get the response to our request as a stream object.
objWebResponse = objWebRequest.GetResponse()
objStream = objWebResponse.GetResponseStream()
' Create a stream reader to read the data from the stream.
objStreamReader = New StreamReader(objStream)
' Copy the text retrieved from the stream to a variable.
strResponseText = objStreamReader.ReadToEnd()
' Close our objects.
objStreamReader.Close
objStream.Close
objWebResponse.Close
' Set return value.
GetWebPageAsString = strResponseText
End Function
</script>
<html>
<head>
<title>Cached HTTP Request Sample</title>
</head>
<body>
<pre>
<asp:Literal id="litTextRetrieved" runat="server" />
</pre>
</body>
</html>
Notice the lines in red in the listing above. They are the key lines that handle the
caching. The first one retrieves the cached data. We then check to see if any data
was returned on the second red line. If it was, then we use that data to build the
page and never make the HTTP request to the remote server. If nothing was returned
from the cache then we request the data from the remote server. The third red line then
stores a copy of the data in the cache for future calls to the page.
The ASP.NET caching engine is quite robust and allows you to specify when a cached item should
expire in a number of ways. That's why the Cache.Insert method looks so confusing.
The version I'm using has the following syntax:
Insert(key As String,
value As Object,
dependencies As CacheDependency,
absoluteExpiration As DateTime,
slidingExpiration As TimeSpan)
Key and value are easy enough. The key is how you identify the data you're caching. And the value is the actual data you
want to cache. The dependencies are used if you want to make the cache dependent on other objects (usually a file).
I'm not using it so I pass in Nothing.
In the example above I'm using the absolute expiration parameter to make the cached data expire 15 minutes from the
time it was cached. Sliding expiration allows you to say don't remove this from the cache unless it
isn't used for a certain amount of time. It's like a countdown timer that keeps getting reset each time the
cache is used. If it doesn't get reset before it reaches zero then the cache is flushed.
For more information on the Insert method of the Cache object see
Cache.Insert Method
in the documentation on MSDN. The method is overloaded, but I'm using the
3rd version
which allows you to set expiration policies.
When you run the cached version of the script you'll once again see the 15 Seconds RSS feed retrieved.
This time however, the timestamp at the bottom doesn't change with every refresh. That's because you're
seeing the version from the cache instead of retrieving a new copy every time. However, if
you wait fifteen and then refresh the page you'll get a new timestamp which will then be returned for the next 15 minutes.
Conclusion
Now that you've seen how easy it is to cache objects in ASP.NET I hope you'll use the powerful
built-in caching for lots of things, but at the very least you really should use it to cache
HTTP requests. It not only makes your site run faster, but it also cuts down on network traffic
and causes less strain on the remote server as well. The only question left is how long should
something be cached? Well that depends on what it is and the answer will be different in each
case, but caching something for even just a minute can make a huge difference on a busy site.
|
