A reader wanting to fully understand this article about advanced Web-enabled application architecture needs to know several technologies. However, someone willing to learn, and with access to documentation, can also understand much, if not all, of it. I am not going to explain every single thing in this article, but will cover the major points of concern.
Required skills are listed a format that assigns a skill number. The associated skill numbers range from 1 to 5, with 5 indicating someone knows it very well. I am hoping that the reader is familiar with XML (4), XSL/XSLT (4), HTML form post (1), Visual Basic (4), ASP and VBScript (2), ANSI SQL (3), and RDBMS stored procedures (3).
While writing this article, I am working on a Windows 2000 Advanced Server, dual P3 800MHz CPUs, 30+ GB of Ultra Wide SCSI 2, 512 MB RAM. However, I am testing it on an NT 4 Server, dual P2 200MHz CPUs, 30+ gigabytes of Ultra Wide SCSI 2, with 196 MB RAM (also running with IIS4, ADO 2.6, and XML 3.0 SP1). All samples work on both machines. Much of the XML and XSL/XSLT is limited to the Microsoft Parser, but our Web servers all run this parser so this is not an issue.
Introduction
In today's rapidly growing Internet environment, developers involved with providing their customers with data that suits their needs are often faced with the immediate requirement to change their data's structure or add new data structures to those available to the customer. While the customer might think such a modification or addition is a simple task, the developer might have to add and/or modify code in several tiers of their existing environment. Also, unfortunately, for many it requires the recompiling of a new version of their middleware. For most enterprise solutions, constant change of middleware is unacceptable and not allowed. Further, even if such changes are allowed, it poses serious versioning and management problems. In this article, I shall try to provide a few ways to modularize your method and data structure definitions. This modularization will involve how definitions are constructed and interpreted by both the requesting party (i.e., the resource consumer) and providing service/component, and thereby make your management of changes more flexible and robust.
Before I continue, I want to qualify exactly how I define "Web-enabled" application, service, or component.
A Web-enabled application is one that requests, and is given access to, services and/or resources via an HTTP request. That's it; nothing more. Whether these requested services and/or resources perform an operation or return data to the requesting client matters not. It doesn't matter what the clients request gets in return. The HTTP request doesn't have to be one made over the Internet and can be one on a private network. These requests might be made by a Web browser, a Visual Basic (VB) application, or a C++ application. Hence, "Web-enabled" means that the glue is HTTP, but more specifically TCP/IP. I am going to leave this qualification as is because I am not going to detail TCP/IP.
Understanding Problems to Identify Goals
(This section addresses typical corporate architectures.Hopefully some of you won't feel so bad about your own company after reading this.)
eXtensible Markup Language (XML) is an ideal format for returning data to a data requestor. It has won this honorary title because it is raw text and can thus be understood and manipulated by any processing system. I say "processing system" because even cell phones and pagers can understand this data format, not just computer systems. However, companies have to be able to justify the investment of conversion to XML as a data-exchange format. Even most Electronic Data Interchanges (EDI), typically used by big corporations with unlimited financial resource, have not converted to XML, where one envisions XML as an ideal replacement. I mean, EDI is an area where XML would be a great replacement technology. And usually big corporations are the ones using EDI and they have tons of money. But, still most haven't converted to XML.
As Information Technology (the big "IT" [some people still think of this as a pronoun]) is, usually only IT developers and architects understand XML well enough to provide management with justifications for such a conversion. Many reasons exist that favor using XML. However, depending on your needs, many reasons exist for not using it, as well. My advice to you is: If you don't need it (and don't foresee the need for it), don't use it. I say this about many technologies. I won't elaborate in depth on common reasons why you should or should not use XML in this article; that deserves an article of its own. However, since I just provided you with one reason not to use XML, I should give you a few reasons why you should use it too.
B2B environments are ideal candidates for XML implementation. The reason for this is obvious.I If my data is in XML format, then I can access it from any operating system, as well as format it to any format, be that format another XML structure, PDF, Word documents, or HTML.
XML is also useful in isolating your content from formatting. For instance, you could easily create the content an entire Web site in XML documents are then transformed using XSL Transformations (XSLT) into HTML. Even taking this a further step, you could render your XML with XSLT into a version of HTML that was compatible with your browser, be that browser Microsoft's Internet Explorer, a Web TV device, or a Web-enabled cell phone. This justification only extends the first reason, so I shall give you another one because it's still only changing the format of XML into another format.
Most applications today, most notably Microsoft's Office XP, are incorporating XML deeply within the internal structure of their applications' basic functions. By doing this, they create opportunities for developers and IT departments to build workgroup applications, much like Lotus Notes.
The Rant
Microsoft has always enabled companies with the ability to extend their product line. For instance, Dynamic Data Exchange (DDE)- and OLE-enabled extensions to the Microsoft line. Today, so many people seem to argue against the "Big Bill" company ( Microsoft). Perhaps it is human nature to put down the most successful, like making jokes about the U.S. president. But, it think it's kind of funny that Microsoft has enabled a path for Perl, Java, and other companies' programs to run on Windows; however, try getting a Microsoft service to run on Unix. Windows runs other companies' applications; Internet Explore runs add-ons introduced by Netscape, and ActiveX controls; Netscape doesn't run ActiveX controls; and Unix won't run COM components. I see Microsoft as an enabler and the other companies as jealous little holdouts. And if I ever meet Bill Gates in person, I am going to thank him for his contributions to the IT arena, without which I'd probably be somewhere writing Watcom C. Well, bash me if you want; if you don't like Microsoft stuff, don't use it, or make something better, but stop talking about it and do something. OK, that's my two cents worth.
In any case, just remember, you need a good reason to use XML; text is transferred using 7 of the 8 bits that make up a byte, so you waste one-eighth (12.5%) of your bandwidth right there. Use common sense.
Consider this...
Database transactions are a great tool when applicable. But why would you use one on a Select statement? That would be like hiring a private detective to watch you. In other words, there's no point in it. ...or this... ...
COM+ is another great technology available for IT to use. But why reference the COM+ library in VB and not use it? Do you really think that by referencing the library and compiling your component you have created a COM+ component? If you don't access and use the interfaces exposed by the COM+ library, then you haven't created anything but a reference to the library, and quite possibly a COM component with a bunch of error-handling code that isn't even used.(I worked at a company once that asked that all COM components be COM+ compliant; I don't know what this exactly means, so I kind of laughed at this. I got the feeling they didn't know what they meant too. Looking back, I still don't exactly know what they meant, but I was sure to reference COM+ and not Microsoft Transaction Server (MTS). This was easy, too, since the reference for MTS didn't show up on the 2000 server.) ...or this... (Ok,, enough, how many technologies are there today?)...
MY POINT IS THIS. I say it again. If you don't need it, now or in the foreseeable future, don't use IT (I am using the pronoun this time).
XML is a great data-exchange format. It doesn't however, in my opinion, expose an easily accessible conversion plan for those companies that have COM components deeply embedded into their corporate operations. Most COM components, good ones anyway, modularize their architecture. Complex object hierarchies are common. For a fictitious example, many do have a Stores object that contains a collection of other Store objects. This collection class is a very robust way of implementing COM solutions, offering modularization, code logic isolation, and even enabling inheritance. However, these data containers classes ( like the Store object in this case) contain hard-coded properties about the Store (ID, Name, Owner, etc.). Further, the Stores object contains hard-coded methods for accessing and manipulating data about each store in its collection of stores. Does this look familiar to any of you?
Public Function Fetch() as Variant
'Abbreviated, more code is supposed to be here
'This is VB6 code, not VB.Net
With oRS
If Not(.EOF) Then
'I do this for performance reasons with large recordsets.
'You can read more about ADO performance issues when 'referencing ' fields
'in the fields collection at 'www.microsoft.com/data/
'Actually, the undocumented collect method would be better
'for most needs
With .Fields
Dim fID, fName, fOwner as ADODB.Field
Set fID=.Item("ID")
Set fName=.Item("Name")
Set fOwner=.Item("Owner")
End With
End With
Do until oRS.EOF
Set oStore=new Store
With oStore
.ID=fID.value
.Name=fName.value
.Owner=fOwner.value
End With
mcolStores.Add Item:=oStore, Key:=CStr(oStore.ID)
Set oStore=Nothing
Loop
Set Fetch=mcolStores
End Function
If you want to change what data is accessible to the object creator, you have to change your Store class, your Stores class, and quite possibly, your SQL statement and tables.
Now you get to ...
Recompile, unregister the old DLL, bring the Web service down, register the new DLL, and bring the Web service back up. You, and maybe some testers, would probably be there in person to do this in the middle of the night. This is a major pain and can really kill your working relationship with your coworkers.
So, what's wrong here, assuming we don't care about binary compatibility, and that's a very big assumption? I am not going to go into what is involved with breaking binary compatibility, nor am I going to explain James Joyce's Ulysses, but I'll leave you with the understanding that you could write about the same about of literature on either topic.
What's wrong here? Well, there's some hard-coding done, and it's not easy to get around this.
Some developers will argue that a client-side disconnected recordset should be returned to the calling application, instead of a collection of data containers objects. I almost did this once. These developers solve this problem by doing a SELECT * or calling a store procedure. Hopefully they use the later method, which specifies what data is returned to the middleware component. In this way, they can just change the table and/or stored procedure and the data that is returned to the caller is also changed. But you should only return the data that you need, so the SELECT * is not good, unless you are using all this information. And, wait, doesn't this approach sound kind of like the consultant's middleman, the recruiter?
Company wants skill set filled.
Consultant provides skill set.
Company contacts recruiter who contacts consultant; done deal.
(But is this all? No. Those of us who have been a consultant
know the difference between pay rate and bill rate.)
Client wants recordset.
Database provides recordset.
Client contacts middleware who contacts database; done deal.
(Hmm, yeah, there's a price here, too, on many different levels.)
The client software is expecting a recordset back so it calls a middleware component that creates and returns a recordset to the client who processes that recordset. So, what have we accomplished here ...from a recordset to another recordset? Hmm, that's bright. At least we used middleware. But why? OK, I'll give it to you: ODBC - Open Database Connectivity (ODBC) (if your using it) connection pooling; true object pooling if you put in COM+ and it's a free-threaded C++ component, and it is a disconnected recordset that the client is processing, and DB username and passwords aren't exposed to every interface developer and etc., etc., etc. This is all good. However, you've created a component dependency for your application, namely, the component that is returning the recordset to the calling client software. This component must be managed. I know, I know, a lot of you are frowning saying, another component to manage.
What else is wrong here? Hmm, well, this component is going to run on a server; if that server goes down, so does your application. Of course, you could always run it on several servers, but then you would introduce configuration dependencies. I know it seems like you get rid of one problem and others appear.
What if you want to add another object to this component? Say you want to have a Warehouses object that had a collection of Warehouse objects -- refer to my "Now you get to ..." section. What if you want to add another method to an existing component? Refer to "Now you get to ." I am not going to argue with the folks who would argue this architecture. I only ask, why use middleware at all? Well, there are a lot of reasons to use middleware and data services encapsulation is just one of them. If you do write middleware like this, I would recommend writing a batch file that does the "Now you get to." Heck, you probably already have one.
But, who would want to reuse this component architecture? You could easily end up responsible for managing that component, probably because you're the only one that uses it, in addition to your application that uses it.
NOTE: I'm sure I might get bashed for this viewpoint, but that's OK. Indeed, I'd rather return the collection of data container classes, as shown in the code above, than do this. At least I wouldn't incur the performance hit to return all the recordset and field information, like field data types. For that matter, I'd rather use ADO's GetRows and return an array.
If someone can argue the point here, I always love to learn. After all, I am in IT and my biggest job responsibility is to keep learning.
We've identified some problems here. A lot of these problems are inherent of systems of this nature. Some of these problems are often discussed in the differences between tightly coupled events versus loosely coupled events. Well, I am only concerned with two events with my architecture: success or failure, and if failure, why it failed.
The Goal
So, what is missing?
I'll tell you, I really hate doing "Now you get to ...", and so do my coworkers. My boss is a stickler about production systems and wants me and my test team there in person (no PC Anywhere to the rescue) when we do it, so we're all there in the middle of the night to make sure everything is up and running the next day. I understand his concern. He has a few thousand employees that use these systems on a daily basis.
Also, I wish I didn't have to manage the COM object. I wish I could move it into production and never have to modify it again. It's the dream of every developer to develop something one time and never have to change it. It's the dream of every architect to design a system that performs better than satisfactory for the rest of time, no matter what the load. Well, these will always be a dream. In reality, due to the changes in IT, these goals remain an ideal. But, perhaps we as developers could be happy if we just didn't have to recompile every few days, if we could just modify some text file and the newly requested functionality would just be there. If I, the developer, could add return structures and request methods without recompile, then I would be happier. As architects, if we could just have something that would scale up and out, and provide me with the same benefits that the developers want, then I would be happy.
Finally, if my customer and/or manager would just tell me exactly what they want when I meet with them the first time, then I could hard code and never recompile because version 1.0 would be exactly what they wanted. (This is just a joke, but it's also a truth about people.) As new technologies become available, new capabilities become feasible. As a result, we customers, managers, developers, and architects always want more. I am afraid this is something we as developers and architects will never get, at least not until we are no longer human. So, get used to change and adapt a mindset that takes these foreseeable change requests into account beforehand. Adopting this mindset will make the successful delivery of version 2.0 and 3.0 of your product a much easier task.
In Part 2, I shall to explore a way of reaching this ideal using the technologies listed in the "Assumptions" section. I shall solve many of our problems listed above.
About the Author
As a senior IT consultant with over 14 years industry experience, Lee architects and develops enterprise solutions for companies, including http://Unrealcity.com, of which he is the cofounder.
When Lee is not working, which is rare, he likes reading, writing, and white-water rafting. He can be reached at 15seconds@unrealcity.com.
Stonebroom.ASP2XML(c) is an interface component designed to make building
applications that transport data in XML format much easier. It can be used
to automatically pass updates back to the original data source.
Right now the latest buzzword around town is AJAX. AJAX is an acronym for Asynchronous JavaScript and XML and is a method used to implement remote calling. The problem is that AJAX is only implemented in ASP.NET 2.0. This article will show you one way to implement remote calling without using AJAX or the XMLHttpRequest object. The technique outlined can even be used from classic ASP and is sufficient for most remote calling needs. [Read This Article][Top]
This article is the third and final installment of Alex Homer's series covering the new XML support in Microsoft SQL Server 2005. In it he covers updating the contents of xml columns, comparing traditional XML update techniques with XQuery, and using XQuery in a managed code stored procedure. [Read This Article][Top]
In the second part of his series on SQL Server 2005's new XML support, Alex Homer looks at extracting data from XML columns, comparing traditional XML data access approaches with XQuery, and combining XQuery and XSL-T.
[Read This Article][Top]
Microsoft SQL Server 2005 now offers great support for and close integration with XML as a data persistence format. In the first article of his series examining this new support, Alex Homer offers an overview of how SQL Server 2005 stores XML documents and schemas, examines how it supports querying and manipulating XML documents, and provides a simple test application that allows you to experiment with XQuery. [Read This Article][Top]
In the final article of his series on reading and writing XML in .NET 2.0, Alex Homer looks at how the updated XML document store objects XmlDocument, XmlDataDocument and PathDocument can be used to read, persist and write XML documents and fragments more easily and more efficiently than in .NET 1.x. [Read This Article][Top]
In the final article of his series on reading and writing XML in .NET 2.0, Alex Homer looks at how the updated XML document store objects XmlDocument, XmlDataDocument and PathDocument can be used to read, persist and write XML documents and fragments more easily and more efficiently than in .NET 1.x. [Read This Article][Top]
Alex Homer continues his series on reading and writing XML in .NET 2.0. In part one, we focused on the reading side of things, examining the XmlReader and XmlReaderSettings classes. In this article, we move on to look at the XmlWriter and XmlWriterSettings classes, and how they can be used to write XML documents and fragments more easily and more efficiently than in version 1.x of .NET.
[Read This Article][Top]
Alex Homer continues his series on reading and writing XML in .NET 2.0. In part one, we focused on the reading side of things, examining the XmlReader and XmlReaderSettings classes. In this article, we move on to look at the XmlWriter and XmlWriterSettings classes, and how they can be used to write XML documents and fragments more easily and more efficiently than in version 1.x of .NET. [Read This Article][Top]
In the first part of his series on reading and writing XML in .NET 2.0, Alex Homer discusses the XmlReader and XmlReaderSettings classes. The XmlReader exposes several useful new features and the all new XmlReaderSettings class makes it easy to generate single or multiple instances of an XmlReader with a range of useful properties. [Read This Article][Top]
In the first part of his series on reading and writing XML in .NET 2.0, Alex Homer discusses the XmlReader and XmlReaderSettings classes. The XmlReader exposes several useful new features and the all new XmlReaderSettings class makes it easy to generate single or multiple instances of an XmlReader with a range of useful properties. [Read This Article][Top]
Mailing List
Want to receive email when the next article is published? Just Click Here to sign up.
