"The earth is flat..." "The sun revolves around the earth..." "Always use components for database access..."
These three statements all have two things in common. First, they were all
at some point in time accepted by the masses as being the truth. Second,
they are all false. Everyone has read countless articles dealing with
using components in Internet applications both for encapsulating business
logic and providing database access. However, rarely is anything published
regarding pure performance numbers on this approach.
With the release of Windows 2000 came significant enhancements across the
board in IIS and especially ASP performance. Early-binding to the
intrinsic objects along with improvements in template caching and other
enhancements have made ASP the top performer in rendering and database
access via ADO.
Based on the statistics presented in the following sections, you will see
that ASP outperforms components in ADO database access and rendering. In
some cases, the difference will be staggering. As with most Internet
applications, performance should be the top priority at all times.
Therefore, it is important that all solutions be fully tested with a stress
tool before making assumptions about the best architecture based on what is
read or hearsay.
All three code-sets (ASP,VB, and C++) were optimized before these tests
were conducted. They were each run through a battery of tests in order to
be sure that the way they were coded for this test was optimum and would
produce the best results for their particular code. Certain optimizations,
noted later, were left out on purpose in order to more accurately represent
the greatest cross-section of code in this type of architecture.
(note: Appendix A and Appendix B for this article can be downloaded here.)
System Architecture
As prefaced in this article, this test is exclusive for Windows 2000
performance. These numbers should not be considered for NT platforms as
the numbers will vary greatly. Below is a diagram of the system
architecture used in the test along with an explanation of the hardware
involved:
Server
Type
Processor
RAM
OS
IIS/COM+ Server
HP LH4R
Dual Xeon 500MHz
1GB
2000 Advanced Server
SQLServer 7.0
HP LXR8000
Quad Xeon 500MHz
2.3GB
2000 Advanced Server
Test Client
Toshiba Tecra 8100
PIII 500MHz
256MB
2000 Server
The test client is connected to the web server via three Cisco 2924
switches. This is due in part to the geographic difference in the location
of the test client to the location of the web and database servers. While
all machines are physically located in the same building, the main servers
are located in the data center. The test client is located on a different
floor that is connected to the main data center switch via a 400Mb Fast
EtherChannel.
In this configuration, there is very little overhead based on network
latency of the test scenario. The traffic between the switches normally
runs at less than 5% capacity during normal operation.
Test Scenario
As stated earlier, this is a test of database access using ADO from each of
the areas, ASP, VB Component, and C++ Component. Therefore, the code has
been limited to the creation of a single table from the resulting
recordset. The code for each of these methods can be seen in Appendix B.
However, they all have the same commonality in flow.
Create/Open a Connection using an ODBC DSN.
Create a Command object (set the type to adCmdStoredProc)
Append the parameter for the number of rows to return.
Execute the command, returning a recordset.
Close the recordset and connection and release memory for all objects.
This method proves to be the fastest for database access due to the following:
Stored procedure access is faster than dynamic SQL, even with the
dynamic SQL plan caching in SQLServer 7.0
Using the Command object and explicitly defining the parameters is much
faster than passing a query string since the OLEDB Provider does not have
to resolve the query type or the data types for all parameters passed to
the procedure.
ODBC Connection pooling prevents physical connections from being
created on each open statement. Closing the connection each time will
release the physically opened connection back to the pool.
The HTML returned to the client is again limited to only the
<table> structure created from the two-column recordset. In all
cases, a while loop is used to iterate the recordset rather than the faster
GetString method to dump the recordset. This is done to more accurately
represent "real-life" processing when iterating recordsets.
The stored procedure used for testing queries rows from one table. The
number of rows to be returned is passed in as a parameter to the procedure.
The tests were run with varying sizes of recordsets and varying threads.
The size of the recordsets ranged from 10 rows to 100 rows. Tests were not
run with more than 100 rows since the majority of well-designed web
applications will not incur this type of cost for rendering large
recordsets. Other methods, such as paging, would be employed if the result
sets were large (e.g. search results).
In addition, the thread counts were incremented ranging from 25 threads up
to 2000 threads. In all cases, the processor utilization for both
processors on the IIS/COM+ server was >99%. However, at the lower
thread counts (25, 50), the ASP queue is small or zero.
Tests were also run with low thread counts of 1,5,and 10. However, no
significant differences other than processor utilization were found at
these levels to merit their inclusion.
The tool utilized for testing was Microsoft's Web Application Stress Tool.
The primary settings for the scripts were as follows:
All scripts were run during minimum network utilization times. In
addition, there was no other activity on either the IIS/COM+ server or the
SQLServer during the testing period.
IIS was set to run in "Low" Application Protection. This will allow for
the fastest performance, especially for the tests with the COM+ application
running as a library application. This will also allow for all tasks to be
performed within the inetinfo process.
The five areas tested were ASP, VB component (library application in COM+),
C++ component (library application in COM+), VB component (server
application in COM+), C++ component (server application in COM+)
In all test cases, the load of the test client never exceeded 35% processor
utilization and used very little RAM.
Certain optimizations were left out of the code on purpose so it could
more accurately represent the majority of applications today. First, the
connection is established using an ODBC system DSN. Changing this to use
the OLEDB provider for SQLServer would provide an across the board
performance increase of ~5-10%. Second, the VB Component could have
declared its output string as absolute "String * 1000". While this would
have helped improve the performance of this component, the limited amount
of data would not have improved the performance enough to skew the
results.
Test Results
Due to the title of this article, you may have already guessed that the
winner of this head-to-head competition was ASP, hands-down. However,
let's dive into the statistics a little further and explain the results and
their ramifications.
The primary statistics looked at for this test were as follows:
Statistic
Description
Requests per Second
The true measure of the server’s
performance. Delineates how many executions can be serviced during the
test run time.
TTFB
Time to First Back. This result, expressed in
milliseconds, outlines the time that it takes for the first bytes of the
Response stream to return to the client. Buffering is on during this test,
so the entire HTML content will be generated on the server before being
returned to the client (faster performance)
Hits
Number of hits aggregated over the test time
period
ASP Requests Queued
The average number of requests waiting to be
serviced. As the queue length increases so will the TTFB. Based on the
response time, there is a limit as to how large you would want the average
queue length to be before scaling out with additional servers
The first series of tests were run on a recordset of 10 rows with the
number of threads varying between 25 and 2000. The results of the primary
metric, Requests per Second, are shown below.
As you can see, the ASP beats the performance of its next closest
competitor, VB (In-Proc - COM+ library application) by an average of 30%.
It has 2+ times better performance than the other methods. It is
interesting to note that the VB(In-Proc) performance goes up slightly as
the number of threads increases. However, after about 250 threads the TTFB
and ASP Requests Queued are too high to merit normal operation of a single
server. In fact, 250 would be considered by many to be too high. For this
reason, tests of higher row result sets are limited to a max of 250
threads.
Keep in mind that there is no delay in the stress script so requests are
constantly being forwarded as soon as a response to the thread is received.
Before looking at the larger row result sets, lets see what happened with
two of our other counters TTFB and Hits.
As expected, the ASP produces the fastest TTFB on all counts. Following is
a comparison on how the increased load affects the ASP Request Queue and
the corresponding TTFB. All cells are noted as Requests Queued - TTFB.
Once again, TTFB is expressed in milliseconds.
Threads
ASP
VB (In-Proc)
C++(In-Proc)
VB
C++
25
0
93.38
.5
144.8
0
174.78
.17
204.47
0
243.12
50
0
189.21
10.17
264.53
1
366.56
7.4
402.71
2.4
539.49
100
40.33
379.39
47.17
511
42
746.58
48.33
805.04
40.4
1100.48
150
81.33
568.33
87.17
746.24
80.6
1124.59
91.67
1212.79
83.33
1628.15
200
123.5
756.69
129.33
979.73
125.17
1485.82
133.33
1615.4
120
2168.3
250
165.33
947.19
171.17
1209.02
166.83
1837.94
172.33
2019.05
169.17
2712.71
As you can see, you would want to add more servers to your web farm as
these numbers get higher (~ 50-100 thread range). However, tests were
performed at these higher levels as well to see if any variance might occur.
The final metric, hits, shows the same common trend as the rest with ASP
leading the way.
The next stage would be to see what happened as the number of records in
the recordset increased. Varying sizes 20, 30, 50, and 100 rows were used.
If you are an avid reader of technical articles, then you would expect the
components to perform better as the number of rows increase due to the
additional processing load added. However, this is simply not the case.
ASP maintains the same variance relative to the other methods as the number
of rows increase. Below are the primary results based on the averages over
the different thread counts.
Even with increasing the sizes of the recordsets, changes are not seen in
the margin that ASP holds over the other methods. Complete lists of all of
the results are listed in Appendix A. Feel free to perform any additional
calculations to determine any additional points of interests. In addition,
the code for each of the methods is listed in Appendix B. You may use this
to perform your own tests, or modify it as necessary.
Conclusions
To summarize, none of the other component methods came close to offering
the pure ADO performance of ASP on a Windows 2000 platform. While the
components running in a COM+ library application compared more favorably to
ASP (as they should have), they still fall short of the raw performance
provided by ASP. In fact, even the In-Proc VB component would need 30%
more performance to compare favorably with ASP. However, a lot of
environments will still run their COM+ applications in a dedicated serverprocess (dllhost.exe) in order to protect the inetinfo process from any problems
stemming from the components in that application. In this common scenario,
ASP would provide a 2 to 1 performance gain!
Hopefully, this article has shown that the benefits of utilizing COM
components in an Internet architecture for database access is not
necessarily a good solution. It is highly important that you always test
any architecture before going too far down a particular path as the results
you get may differ greatly from either what you expect or what you may have
read, heard, or seen.
When dealing with Internet applications of moderate to high projected
volumes, performance should always be the utmost priority, well ahead of
code reuse. In the majority of cases, the most elegant solution is rarely,
if ever, the best solution in terms of performance.
The use of components is not without merit. It is often the best choice
for certain types of business logic encapsulation, especially for
cross-system integration. However, in some cases this seems to have gone
to extreme measures. It is prudent to take a step back and look deeper
into a given technology platform and truly understand how data is being
processed, marshaled, and transmitted based on applicable business
scenarios that are being dealt with. More often than not you will find the
precept of Ocum's Razor to hold true, in that when dealing with complex
systems, the simplest solution is most likely to be the best (and quite
possibly the easiest to scale!)
While this article deals exclusively with limited database access via ADO,
similar tests have also been performed on more complex scripts and
components with similar results. If you are running on a Windows 2000
platform, you should definitely consider extensively testing your
performance along these same lines. If your ultimate goal is performance,
you might be surprised to find yourself replacing some of your components
with well-constructed ASP.
About the Author
With more than ten years of experience in software development for a variety of vertical industries, Drew Seale is responsible
for the overall architecture and development of VertaPort applications and systems, including database design and
development.
Prior to joining VertaPort, Drew was the director of application architecture for Sea-Land Service, Inc., where he was solely
responsible for the architectural guidance and framework of various development teams located throughout the world.
During his career, Drew has gained expertise in the design, development and deployment of mission-critical Internet systems
and architectures, using Microsoft technologies. Drew is a Microsoft Certified Systems Engineer + Internet (MCSE+I) and
Microsoft Certified Database Administrator (MCDBA).
Drew earned a bachelor’s in advertising from the University of Texas in Austin and a master’s degree in business
administration, with a concentration in information systems, from Texas Tech University in Lubbock, Texas.
XCache combines dynamic content caching technology with content delivery network (CDN) support options, file compression and a whole lot of manageability features to help e-businesses deliver superior web site performance and reliability. You'll appreciate the administrative ease, your
visitors will appreciate increased page delivery speed.
XCompress works by compressing outgoing text between the Web server and the client. Page response times may improve by a factor of three or more while overall bandwidth use can drop by two thirds or more.
XCompress runs on Windows 2000 and Windows NT 4.0 and is tightly integrated with Microsoft Internet Information Server (IIS) with MMC and COM interfaces.
XTune 2.0 is the most powerful tuning application for IIS 4 or IIS 5 ever
conceived. Indispensable to the enterprise and straightforward, this web
tuning tool allows you to configure hidden operating system, network, Active
Server Pages and Internet Information Server settings for better
performance, without any additional hardware or software.
This version scans your system more deeply, offering more
performance-enhancing recommendations and greater insight into your web architecture. The Performance Wizard guides and teaches you throughout the complete tuning process, so you can learn while making your box run better than ever.
Performance monitoring helps organizations identify performance bottlenecks. The problem is that with so many performance numbers available, how do you know which ones to watch? This article helps you identify which are the critical performance counters in a SharePoint Portal Server environment and explains how to monitor them. By monitoring performance regularly, organizations can recognize performance trends as they develop and prevent problems before they get out of hand. [Read This Article][Top]
There is broad-reaching debate about remoting, Web services, Enterprise Services, and DCOM. In short, it is a debate about the best technology to use when implementing client/server communication in .NET. Rocky Lhotka shares his thoughts on the issue while offering clear explanations of basic application architecture terminology. [Read This Article][Top]
This article examines some of the new and exciting caching features in ASP.NET 2.0 and shows how to implement them in Web applications. [Read This Article][Top]
When it came time to find a technology for its massive upgrade, Match.com chose .NET. Has the online dating service's partnership with Microsoft been as successful as the relationships it has established for many of its millions of members? Read on ... [Read This Article][Top]
Longtime 15Seconds discussion list member Tore Bostrup offers valuable advice on designing databases and applications for efficient querying. [Read This Article][Top]
Narayan Veeramani shows how ASP.NET developers can improve application
performance by caching data stored in an Oracle database and keeping
the cached data in sync with the data in the Oracle database. [Read This Article][Top]
Ever developed a Web application that requires extensive processing? Ever had long running Web pages that often time out in the browser? Greg Huber reveals a simple technique that uses Microsoft Message Queuing (MSMQ) and the System.Messaging framework to handle long running Web processes. [Read This Article][Top]
As IT professionals try to reduce the cost of operating their Web sites, they should consider reducing the amount of bandwidth usage. Learn how to successfully compress your HTML output and save money on your monthly bandwidth. [Read This Article][Top]
Maintaining a large Web farm is both costly and unnecessary. Learn how to reduce your Web farm to just two servers in this controversial article by Wayne Berry. [Read This Article][Top]
