asp tutorials, asp.net tutorials, sample code, and Microsoft news from 15Seconds
Data Access  |   Troubleshooting  |   Security  |   Performance  |   ADSI  |   Upload  |   Email  |   Control Building  |   Component Building  |   Forms  |   XML  |   Web Services  |   ASP.NET  |   .NET Features  |   .NET 2.0  |   App Development  |   App Architecture  |   IIS  |   Wireless
 
Pioneering Active Server
 Power Search








Active News
15 Seconds Weekly Newsletter
• Complete Coverage
• Site Updates
• Upcoming Features

More Free Newsletters
Reference
News
Articles
Archive
Writers
Code Samples
Components
Tools
FAQ
Feedback
Books
Links
DL Archives
Community
Messageboard
List Servers
Mailing List
WebHosts
Consultants
Tech Jobs
15 Seconds
Home
Site Map
Press
Legal
Privacy Policy
internet.commerce














internet.com
IT
Developer
Internet News
Small Business
Personal Technology
International

Search internet.com
Advertise
Corporate Info
Newsletters
Tech Jobs
E-mail Offers

HardwareCentral
Compare products, prices, and stores at Hardware Central!

Object-Relational Persistence for .NET
By Scott Bellware
Rating: 4.2 out of 5
Rate this article


  • email this article to a colleague
  • suggest an article

    Table of Contents

    Part I - Toward a new Data Access
    Introduction
    Adoption
    Traditional, API-Based Data Access
    Object-Relational Persistence
    Mapping and Metadata
    Data Mapping Dogma
    Persistence Framework Dependencies
    Domain Schema Standards
    Part II - Code
    The ADO .NET API Way
    The ObjectSpaces Way
    Readability And Useability
    Mapping with ObjectSpaces
    Part III - Wrapping Up
    Stored Procedures and Cultural Barriers
    Conclusion
    Links

    Part I - Toward a New Data Access

    Introduction

    Only a few weeks ago the aficionados and enthusiasts of object-relational persistence for .NET might have been counted on a few sets of fingers and toes.  They were a small band of brothers and sisters bound by a belief that it is possible to eliminate all but a smattering of the verbose ADO .NET code from an application’s codebase and decouple an application’s business code from a specific data provider.  It’s almost a heretical thing to say – especially when the vast majority of .NET developers are more than willing to resign themselves to endless hours coding of an ADO .NET API-based data-access layer.

    If you were at the PDC, you may have picked up on the here-and-there hints at an addition to Microsoft’s collection of data access strategies.  Object-relational persistence (ORP) capabilities can be found in a number of places in the Whidbey-and-beyond technology set.

    Surprising to me was the number of people who attended the object-relational Birds of a Feather (BoF) session I co-hosted with Dave Foderick.  I had thoroughly expected to have a nice conversation with ten or so professionally isolated OR enthusiasts about object-relational persistence, the current OR tools for .NET, and the experiences that people have had with the various tools and frameworks.  However, over one hundred people showed up for the BoF.  The conversation was boisterous and the audience was rife with evangelists, naysayers, lurkers, and even some OR product vendors.  The following afternoon, the Microsoft ObjectSpaces team demonstrated their object-relational persistence framework for a packed house of a few hundred attendees.

    Object-relational capabilities are not just for folks using ObjectSpaces.  You can find traces of object-relational approaches throughout the upcoming release of the next generation of SQL Server codenamed “Yukon”.  Yukon can host CLR-based code, and work with CLR types and stored procedures written in .NET languages.  Business objects can be passed to Yukon stored procedures as ADO .NET 2.0 parameter values and can then be decomposed into relational structures in .NET stored procedure code.  In addition to this, Yukon tables can have columns whose data types can be a CLR-based complex type.  In such a scenario, a business object can be passed to Yukon in a parameterized query, and rather than decomposing the business object into relational structures, the business object can be persisted directly into the database as a column value.

    Consider an application that stores some simple information about Person objects.  For this example a Person object has an Id, a Name, and a Birthday.

    The data definition language (DDL) to create a SQL Server 2000 database table to represent the Person rows might look like this:

    CREATE TABLE [dbo].[Person] (

       [Id] [int] IDENTITY (1, 1) NOT NULL ,

       [Name] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,

       [Birthday] [datetime] NULL

    ) ON [PRIMARY]

    With SQL Server Yukon, you could do the following:

    CREATE TABLE [dbo].[Person] (

       [Id] [int] IDENTITY (1, 1) NOT NULL ,

       [Person] [Person] NOT NULL

    ) ON [PRIMARY]

    The significant difference between the two is that the DDL targeted at Yukon has a column of type Person.  The Person type is a .NET class.  Columns in Yukon can be typed to custom .NET classes.

    In order for a column to be typed to a CLR type, the type would first need to be compiled and then installed into SQL Server.  Here’s an idea of what the Person type might look like:

    using System;

    using System.Data.Sql;

    using System.Data.SqlTypes;

    using System.Runtime.Serialization;

     

    namespace ScottBellware.PersonExample

    {

       /// <summary>

       /// Represents a Person.

       /// </summary>

       [Serializable]

       [SqlUserDefinedTypeAttribute(Format.UserDefined)]

       public class Person

       {

           private int id;

           private string name;

           private DateTime birthday;

     

           public int Id

           {

              get{return this.id;}

              set{this.id = value;}

           }

     

           public string Name

           {

              get{return this.name;}

              set{this.name = value;}

           }

     

           public DateTime Birthday

           {

              get{return this.birthday;}

              set{this.birthday = value;}

           }

       }

    }

    Once the type is compiled an assembly containing the type is on the file system, you can register it with SQL Server Yukon.  Let’s say that the assembly containing the Person type is built to C:\Assemblies\ScottBellware.PersonExample.dll.  Here is the T-SQL code that would register the assembly with Yukon:

    CREATE ASSEMBLY PersonExample FROM 'C:\Assemblies\ScottBellware.PersonExample.dll'

    Following this, you would then need to create the type inside Yukon.  This essentially registers an alias for the type that Yukon will use to reference the CLR type contained in the assembly.  The type needs to be created inside Yukon because the DDL to create the table (above) has a column of this type.  Here is the code to create the type:

    CREATE Type Person

    EXTERNAL NAME [PersonExample]:Person

    You could then use the typical parameterized query data manipulation language (DML) to insert a Person instance into the Person table:

    INSERT INTO Person (Person) values (@Person)

    The above DML might have tickled your curiosity if you have some experience with ADO .NET 1.x DataParameter objects.  With version 1.0 of ADO .NET, DataParameter objects could encapsulate scalar values.  The above command text suggests the changes that have made to the SqlParameter class for version 2.0.  In ADO .NET 2.0, you will be able to pass complex types from client code to the database runtime engine. 

    The code pattern for using a SqlParameter instance to pass a custom type to the database is similar to the pattern used to pass scalar values using ADO .NET 1.x DataParameter objects.  With ADO .NET 2.0, the SqlParameter instance’s Value property will be assigned a reference to the object that you wish to pass.  The SqlParameter class has been augmented with the UdtTypeName property.  Using this property, the SqlParameter instance is informed of the name of the complex type that the parameter encapsulates.  As well, the SqlDbType enumeration has been augmented with the SqlDbType.Udt value to indicate that the encapsulated value is a complex or “user-defined type”.

    The following sample code demonstrates the ADO .NET 2.0 client code to execute a parameterized query with a complex type.  Assume that there is already a SqlConnection object in scope named “connection” as well as an instance of Person named “person”.  For the purposes of the example, assume also that the connection to the database is already open.

    // Create a command from the connection object.

    SqlCommand insertCommand = conection.CreateCommand();

     

    // Assign the DML to the command’s command text.

    insertCommand.CommandText = “INSERT INTO Person (Person) values (@Person)”;

     

    // Parameter object to encapsulate the person instance.

    SqlParameter parameter = insertCommand.Parameters.Add(“@Person”, SqlDbTypes.Udt);

     

    // Set the type name of the parameter’s encapsulated complex type.

    parameter.UdtTypeName = “ScottBellware.PersonExample.Person”;

     

    // Assign the person instance to the parameter’s value.

    parameter.Value = person;

     

    // Execute the command.

    command.ExecuteNonQuery();

    Looking even further into the future, Longhorn brings the promise of object-relational approaches at the platform level.  The Longhorn file system itself is based on next-generation SQL Server technologies, and ObjectSpaces plays a role in the future platform that is important enough to make it first class citizen of the Longhorn API diagram:

    You should also expect to see object-relational persistence mechanisms pop up in the yet-to-be-released Microsoft Business Framework (MBF).  The MBF provides abstractions that represent business objects and business processes that can be used to build business applications.  MBF objects can be serialized to XML for use in distributed applications, and they can also be persisted by – you guessed it – ObjectSpaces.

    Object-relational persistence is certainly not taking over for more traditional data access approaches, in fact object-relational persistence in .NET often builds on the .NET data access patterns and mechanisms already in place.  Data access commands and results pass through the layers of data access APIs even if they originate in an object-relational persistence framework.  Object-relational persistence frameworks abstract the lower level API much in the same way that ADO .NET abstracts the lower level native client libraries.  With each higher level abstraction, client source code becomes cleaner, more readable, and less massive.

    Adoption

    So why haven’t more developers caught on to object-relational persistence?  Most .NET developers traditionally look at data access with blinders – believing that their only choice for data access is to either long-code a data access layer to the ADO .NET API, or to get a code generator that will accelerate this task.  Either way the result is a great big mess of verbose ADO .NET API client code.  The believers of the Gospel of Object-Relational Persistence know that this simply isn’t the only path to data access salvation (of course, some of these folks also have their own brand of firmly-affixed blinders).

    Even with the growing impetus toward object-relational approaches we .NET developers are still largely in a time of transition from procedural coding practices to object-oriented coding practices.  After all, just because you’ve ported your VB 6 app to VB .NET doesn’t necessarily mean diddley in terms of any real understanding of the power of OO.  You can still run procedural, linearly-structured code through a .NET compiler and execute it on the CLR.  It’s like OO pioneer and luminary Tom Hadfield says, “Object languages allow advantages, but don’t provide them.”

    In order to have an object-oriented approach to data access, you need object-orientation.  With the transition to OO for .NET developers will come all of the advantages that other OO cultures have enjoyed for years.  Object-relational persistence has long been a facet of Java development, but Java was an object-oriented language and platform from the very beginning.  The vast majority of Microsoft developers may form the object-oriented rearguard, but object-oriented momentum is building in the Microsoft world and we are seeing clear signs of this now.

    I don’t want to give the impression that the Java world is an object-relational utopia where persistence frameworks abound and Java projects are consequently on-time, on-budget, on-scope, with adaptable, easy to maintain and easy to test codebases.  This simply is not the case.  If anything, the relatively low adoption of persistence frameworks in Java is often quoted by .NET folks as the reason why we should not pursue object-relational persistence in .NET.

    Despite the numerous persistence frameworks that have been available for Java for some time, the approach isn’t as prevalent as I expect it will be for .NET.  The fortunate aspect of the Java persistence models for .NET developers is that Java folks went ahead and pioneered a lot of the approaches and figured out where persistence frameworks are useful, and when they are detrimental.  As .NET developers, we gain from the experience of those who went before us into the OO fray (er… which is just about everyone at this point).

    One huge mitigating factor in the adoption of object-relational persistence approaches is the backing of a titanic vendor.  There were certainly some great object-relational products for Java availed by some great vendors, but object-relational persistence has often held an esoteric position in the realms of data access.  Many Java OR tools vendors had great products, but turned around and priced themselves out of the market before they even proved the case for their products.

    In the .NET world, we have Microsoft.  In our culture, we can expect some of the things that are priced out of reach in other development cultural experiences to be priced to move by Microsoft.  It looks like Microsoft is treating object-relational persistence facilities for .NET as a commodity rather than a precious resource to be guarded and shared with only those few deserving supplicants who belly up to the bar with huge budgets and over-flowing pockets.  I suspect that if OR vendors in the Java world had the freedom to use the Microsoft pricing models, we might be much further down the line in object-relational persistence adoption at large, and the technologies might have had an opportunity to become even more evolved.  As is stands, the open source community is the real torch bearer for affordable object-relational persistence for Java these days, and there are a few really good, mature projects producing persistence frameworks for Java that can be found on SourceForge.

    With the unveiling of Whidbey, Microsoft is stepping in to make the case for object-relational persistence and for most .NET developers, that’s all the justification that’s needed – especially when the OR product is included in your standard tool kit rather than priced on the scale of a UML modeling tool.  Microsoft gets the idea of economies of scale.  ObjectSpaces is ready to use as soon as you install Whidbey.  Simply add a reference to System.Data.ObjectSpaces.dll.

    Of course, the world of object-relational persistence for .NET isn’t all rosy just yet.  There are still many hurdles to deal with and much product maturation that still needs to happen.  However, unless there are a significant number of developers who are willing to be on the object-relational persistence vanguard for .NET, the solutions being handed to us from Microsoft may not get the community vetting that they need in order to ensure maturation.  If the crowds at Yukon and ObjectSpaces talks at the PDC are any indication of a potential ground swell towards object relational persistence for .NET, we should expect that Microsoft’s object-relational offerings will develop enough of a following to put the products through their paces and provide the real world feedback that the Microsoft folks will need to make their products meet the real world needs.

    Traditional, API-Based Data Access

    Before the advent of .NET and mainstream OO in the vast Microsoft developer culture, there was only one approach to work with relational data in an application – API-based data access.  Few people called it “API-based” data access because it was the only data access game in town. There was no need to qualify it with the “API-based” moniker.  It was simply “data access”.  Now that object-relational persistence is in play (as well as SQL XML), it’s appropriate to make the distinction between the two.

    ADO .NET is a low-level data access API.  At least this is the case when you look at it from the perspective of an object-oriented API as expressed in an object-relational framework.  ADO .NET is a relatively high-level API from the perspective of native database client libraries.  However, you wouldn’t expect to find yourself coding to the native client libraries unless you had a darned good reason to do so.

    Is there a reason to code to lower-level APIs?  There sure is – performance.  When you are trying to eek out every last drop of performance from your technology, working with high level abstractions is usually a choice that leads to less performant code.  When squeezing every last drop of performance out of your application isn’t a requirement, you should be able to defer to the higher level abstractions and take advantage of some of the facilities that these abstractions offer.

    Some would say that performance optimization is a constant, canonical goal of software development.  The performance requirements of an application should be driven like many other application requirements – by the customer and other stakeholders, and by measurement.  Often, we spend lots of time creating high-performance data access code that offers little advantage to the customer and the user since the load under which an application operates in production is insufficient to create any human-perceivable performance degradation.

    Performance zealotry costs us dearly and it’s really been allowed to go unchecked for far too long.  When I hear a coder talking about a high performance coding approach for data access that was proudly put in place on a current project, I typically like to ask (as any rascal would) how the team goes about measuring performance and defining thresholds of acceptability.  If you can’t answer these questions, chances are you might be coding for performance for its own sake rather than to support hard requirements.  Sometimes you need to do this anyway if you have no means to measure performance to begin with.  In light of this absence of measurement capability, most developers will opt for coding the highest performing data access layer they can think of – whether or not that’s what’s called for.

    So before you go running off to code to a low-level API such as ADO .NET, be sure that you have a concrete and measurable reason to do so.  You might be surprised to find that using higher level persistence abstractions provides you with a level of performance that is sufficient for the requirements of the application.

    Higher order abstractions usually lead to code that is less verbose, less massive, more anecdotal and readable, easier to initially code and subsequently maintain, and generally a more enjoyable experience.  Essentially, data access code that is based on object-relational persistence approaches and tools is typically vastly less expensive to create and maintain, and is usually more amenable to change requests and enhancements.

    Succinctly: object-relational persistence approaches lend agility to your projects and products.

    Object-Relational Persistence

    An object-relational persistence framework provides basic data CRUD (create, retrieve, update, and delete) services to your business objects.  A persistence framework is really not responsible for doing any more than this.  If your application has business logic (and I can’t imagine a business application development project that doesn’t have the requirements for business logic), then the persistence framework is not the part of the architecture that handles this.

    Coordinating business logic with persistence is the responsibility of a yet higher order framework.  This is what the Microsoft Business Framework is responsible for.  The persistence framework’s only job is to perform operations on business objects that correspond to SQL insert, select, update, and delete commands.

    A good persistence framework should also coordinate persistence messages between related business objects as well as coordinate basic transactions, manage optimistic concurrency, and propagate database-generated keys back to the middle tier.  Persistence frameworks also provide mechanisms – usually in the form of some sort of metadata – that allow business objects to be mapped to database tables, and business object properties to be mapped to table attributes.

    A persistence framework has the smarts to understand the structure of your business objects and can infer the SQL command text needed to perform CRUD operations on the underlying durable data store.

    For example, if a Person object has three properties – Id, Name, and Age – the persistence framework will understand that the SQL 92 command text to retrieve a person whose Id happens to be 123 should look like the following:

    SELECT Id, Name, Age FROM Person WHERE Id = 123

    A persistence framework knows how to do many useful things with a minimal coding effort that would otherwise take a mountain of long code to accomplish.

    Object-relational persistence frameworks understand the relationships between business objects. This allows the framework to cascade operations to associated objects.  For example, a retrieval method on an Order object could cascade to child objects, and retrieve the LineItem objects as well.  In the same vein, invoking a method to save changes to an Order object may also cascade the invocation to the child objects causing the LineItem objects to be saved.

    Well factored persistence frameworks introduce a layer of abstraction between the application domain and any specific database vendor API.  For example, business objects and the code that uses them are not tied to a specific database such as SQL Server, Oracle, MySQL, DB2, etc.  While in theory, persistence frameworks are capable of accomplishing this through the layered designs inherent in the frameworks, not all framework developers are willing to support multiple vendor APIs in practice.

    It is unknown at this point whether the ObjectSpaces team will build support for Oracle and other data providers (at least with the 1.0 version).  The effort to create the abstractions to represent the Oracle (for example) data provider is probably not overly taxing.  However, implementing a reusable, abstract test harness that can test the same set of business objects and business code against multiple providers can be an intensely time consuming task.  I would think that supporting the SQL CLR context objects and incorporating the advantages provided by and ADO .NET 2.0 is vastly more important to the ObjectSpaces team than supporting the Oracle (or any other) provider.

    Mapping and Metadata

    In .NET, one of the mechanisms that typically makes mapping possible is reflection.  By reflecting over a class, the persistence framework can get a list of all fields in that class that conform to an arbitrary set of constraints, and can retrieve any assembly meta associated with the fields that might provide specifications on how to map business objects to database tables, as well as how to map business object fields to database table columns, etc.  With the list of fields for a given business object in hand, along with the mapping information, the framework can derive not only the SQL SELECT command text, but the command text for the insert, update, and delete SQL commands as well.

    Reflection isn’t the only mechanism available in .NET.  Often you will find that object-relational persistence frameworks also make use of XML files to provide the necessary metadata.  XML files have the advantage of not requiring the recompilation of source code when mapping changes.

    The use of external XML mapping files allows for flexibility that compiled metadata doesn’t.  With external mapping files that are read at run-time, a compiled business object library provided by a third-party (or even provided by your own team) can be mapped to existing database objects (tables, views, etc) without recompiling the library.

    The goal in such a case is not to be able to change the data model of a production app and still have the object model function as normal without having to recompile, version, and re-deploy the library.  Although such a thing is possible, it’s somewhat unrealistic in practice.

    Data Mapping Dogma

    As beneficial as non-compiled mapping approaches can be, there are risks implicit in these approaches as well.  There are a couple of value propositions that developers often attribute to object-relational mapping that in reality don’t offer much value at all, and in some instances can actually be harmful.

    External XML files allow a data model to vary somewhat independently of an object model.  If a change to the data model is necessary, for example if a business object field needs to be remapped to another table column, this can be accomplished without necessarily recompiling the business object.  At first glance this sounds like a great capability, but upon deeper consideration some problems appear.

    Changes to an application’s data model are not insignificant changes.  An application’s production data model isn’t usually a highly dynamic software element (well, some are, but you can usually smell those projects from a mile away).  When an application’s data model is identified as requiring a high degree of flexibility in production, such flexibility ends up being reflected by the design choices made when modeling the application’s domain classes and its database.  When a need for a flexible data model is identified, vertical decomposition using domain-key normalization is typically the chosen pattern.  In situations where volatility in the domain model is expected, we typically don’t opt for static data models and then count on dynamic mapping mechanisms to solve for the ensuing change management activities.

    If an application’s data model changes, it’s typically a sign that a significant requirements change has occurred.  Under such conditions, it is entirely feasible to expect that a chain reaction of change management process will be triggered.  A change in an application domain’s data model is an indication that new meaning has been introduced into the domain model.  Changes to business process code, screens, reports, interfaces, and documentation can be expected to ensue, as well as changes to the business objects impacted by the changes to the data model.

    When changing the application’s domain model, you should expect some additional coding work to account for the change in the business processing code that will take advantage of the new meaning added to the model.  You should expect to execute unit tests and integration tests to make sure that the new changes have been accounted for across all extents of the application.

    The total effort to incorporate a change to an application’s domain model should be much greater than changing some mapping information.  External XML mapping files buy us little in optimizing for rapid response to change because there are more things to account for than simply the mapping.  External, non-compiled mapping files don’t exist to provide rapid response to change.  It’s a misconception that mapping is there to buy you time from a RAD perspective.  As a matter of fact, the management of not only the source files of a domain layer, but also the external mapping files can make the total change management process even more costly.  When mapping metadata is contained in external resources, we have to remember to update, version, validate, and deploy those resources along with the rest of the application’s necessary runtime resources and executable artifacts.

    The only developer who can possibly eek out a less than trivial cost savings in change management from mapping alone is probably a developer little to no test-driven focus.  The ability to remap object model elements on-the-fly to data model elements using external mapping potentially presents an opportunity for undisciplined production staff to be made to feel over-confident due to the simplicity of such changes.  They are likely to neglect the execution of unit tests and integration tests when such changes are incorporated.  By allowing mapping information to change without changing source files seems to send the message that these changes are trivial and easy and thus they are less likely to require validation.  This is an issue that should be set straight upfront by the architect or senior developers when bringing object-relational approaches into a development team, and should be enforced on an on-going basis.  Non-compiled resources bring the same risks to production support that interpreted run-time environments brought to our efforts during our more primitive recent past.

    In light of these potential pitfalls, the advantages of using external, non-compiled mapping resources are quite compelling.  With compiled metadata (NET attributes, for example) decorating your business classes, these classes become coupled to the persistence framework.  If one of your original objectives in adopting a persistence framework was to decouple your code from any specific data access API, then using compiled metadata provided by your chosen persistence framework in your business classes as mapping mechanisms, you succeed in coupling your business classes to the persistence framework’s API.  Essentially, this means that your business classes cannot be leveraged in the context of another persistence framework, and this constrains the reusability potential of your business object libraries in certain reuse scenarios.

    In my own experience using a persistence framework that uses compiled metadata as a mapping mechanism, the approach proved perfectly reasonable in practice and provided an acceptable and manageable mapping solution.  There are circumstances, particularly with business object remoting, where API dependencies aren’t acceptable at all.  In such scenarios, coupling the remoted representations of your domain to the persistence framework used in a specific application context without consideration of the greater society of interconnected enterprise apps is something that should require some thought and justification upfront.  It is perfectly reasonable to do so, but the dependency must not be allowed to cause problems outside of the boundaries of the specific application where it is leveraged.

    Persistence Framework Dependencies

    You will have dependency between your business objects and your chosen persistence framework when, for example, business objects are subclasses of a base class that is part of the persistence framework, or when metadata classes used to decorate business objects and their members are part of the framework.

    Coupling business objects to a particular persistence framework is a dependency, and dependencies need to be seriously considered before deciding whether they are permissible or not in your architecture.  While it is realistic to expect that your business objects are coupled to a business object framework, coupling business objects to a persistence framework is another thing entirely.

    Business and persistence frameworks support different architectural layers and therefore should not be overly coupled to each other.  Persistence frameworks that do not bind elements from the business layer to elements in the persistence layer express a compelling architectural advantage.  However, in the large scheme of things, having business objects that depend on elements of the persistence framework can often be a justifiable dependency for many mainstream applications.

    As far as dependencies go, this kind of dependency is somewhat forgivable.  Binding business objects’ APIs to a given framework shouldn’t cause rippling effects across a contemporary integrated enterprise.  With service-oriented architectures (SOA), encapsulation boundaries are typically drawn at the course, system level rather than the component level.  You would expect that inter-system communication will speak XML and the type system in use will more than likely be XML Schema rather than business object types.  Although it’s possible to find circumstances where a subsystem itself is highly distributed, chances are that such a subsystem is highly encapsulated and the interface it presents to its clients won’t express any of its internal dependencies.

    In this light, tying the layers of a subsystem together in practice by creating an API-based dependency across application layers is not such an architectural crime – although it is not architecture nirvana.  For example, decorating business object properties with metadata classes that are defined in a persistence framework isn’t so bad because you should expect that those business objects built for that application will more than likely use that application’s persistence layer.  The two layers are relatively dependent and bound to each other already by playing integral roles in an application’s architecture.  You shouldn't be too surprised if you find that these layers share some intimate knowledge of each other simply due to their proximity to each other.

    In all cases, an applications persistence layer’s interface to other architectural components in the same application should be well-isolated and it should itself be represented by a course, static, service-oriented façade.  Doing so helps to further limit any detriments of intra-application dependency.

    The layers of a subsystem are usually created by the same project team using the same tools and frameworks.  Creating micro API-based dependencies at the macro level - inter-system, across web services in an SOA – is another issue altogether.  And while macro-level, inter-system API-based dependencies can be caused by directly reusing another business unit’s business object libraries, this is usually done willfully with intent as part of a reuse effort that ultimately leans on the standardization on frameworks that enable the reuse effort in the first place.

    As more developers are exposed to the imperatives of service-oriented design, the knowledge gained in subsystem encapsulation implied by service-oriented approaches will go a long way in assuring that micro API dependencies will remain in isolated, manageable, application-specific (or application suite-specific) locales.

    Domain Schema Standards

    Acquiring a compiled business object library from a third-party and successfully mapping it to a production database is a bit of a rare phenomenon.  Business object libraries that are built in complete isolation of the underlying data model are often irreconcilably incompatible with in-production data models.

    It isn’t realistic to assume that you can plug any object model into any data model simply by means of a mapping layer.  In rudimentary cases, it may indeed be possible to do so.  However, it is often unlikely that one organization’s perception of a given business object is going to be similar enough to another organization’s perception that the two would end up supporting data models that are similar enough that a declarative mapping layer would account for reconciling any differences that would otherwise impede the use of a common object model.

    Often, mapping can account for minor mismatches such as naming style and rudimentary factoring, but models built in isolation of each other are often divergent enough that a more significant integration effort is required.  In all but trivial third-party mapping scenarios, the solution will more than likely involve making use of middleware mechanisms such as Biztalk that provide a framework for mapping that accounts for solving name reconciliation as well as a framework for hosting and executing custom code that will reconcile more complex differences in models that declarative mapping on its own is simply too ill-equipped to address.

    In cases where the assumption is made that a third party’s business object library can be made to work with your data model, much of the cost savings gained by object-relational persistence may be consumed by the complexity of implementing and maintaining the mapping mechanism.  It really isn’t realistic to expect that any business object model can be mapped to any data model.  A system’s object models and data models are particular aspects of a greater whole and developing them in isolation of each other - without knowledge and consideration of each other - usually doesn’t lead to emergent goodness.

    With the increase in standardized representations of common domains, the likelihood will increase that two (or more) organizations might someday share similar or even identical business object definitions supported by data models that are indeed reconcilable through declarative mapping. When this happens, you can expect that a few really powerful business object frameworks will begin to make custom business application development a much smoother process and ease integration efforts.  With standardization around domain schemas, the persistence tools that offer external XML (or other) files as their mapping mechanism will offer an increasingly important value to the application development.

    Microsoft’s ObjectSpaces persistence framework is invested in the mapping approach based on external XML mapping files.  With the advent of ObjectSpaces and its approach to mapping, coupled with the advent of the Microsoft Business Framework, augmented by certain efforts trend toward domain schema standards, we might venture to conclude that Microsoft has actually begun to pursue in practice a vision for application development based upon the direct reuse of mappable domain schemas.  Much work still remains to be done towards our greater awareness of reuse as a whole in the industry, but it is certainly refreshing to see that Microsoft has really begun to push the possibilities of reuse in the .NET era.

    One of the improvements (or detriments – depending on which side of the argument you stand) made to ObjectSpaces since the previous technology preview release is that it now makes use of the new (to .NET framework 1.2) namespaces and libraries specifically targeted to data mapping (System.Data.Mapping).  These are the same facilities used by SQL XML for its mapping.  The earlier version of ObjectSpaces used a proprietary mapping schema that didn’t particularly apply to anything else in .NET.  With the rise of mapping-based approaches to persistence in .NET, ObjectSpaces and SQL XML have found some synergy around mapping, and this is great news for .NET developers who plan to make use of either ObjectSpaces, or SQL XML, or both – there is only one mapping framework to learn!

    The mapping framework also provides facilities for creating maps in code rather than with XML files.  There are some samples and tools that have been released with the Whidbey alpha that demonstrate this capacity and you can investigate this approach for yourself by installing the Whidbey bits and familiarizing yourself with the mapping framework.  A link to these downloads will be provided at the end of this article.

    The ultimate end-goal of these efforts is the much-lauded convocation-based approaches to software development where applications are assembled from existing components where the instance configuration of these components may include code-based or XML-based mapping of business objects to relational data stores.

    While we may never get to the point where the specification of our business objects issues from standards bodies, it is likely that the on-going efforts for prescriptive architectures asset-based development will take hold in the enterprise and as a result disparate business units may share the same definition for some business objects in their domain.  In these scenarios, compiled business object libraries will need to do some application-specific mapping for their specific implementation of the underlying data model.

    The challenge for many organizations today is accepting that a certain amount of centralized architecture must take place before the reuse of business object libraries across business units can be achieved.  Selling architecture in an organization that has never had an architecture practice isn’t such an easy task, even in light of some the more self-evident benefits.

    The presence of frameworks like ObjectSpaces, servers with the capabilities of Yukon, the MBF effort and the implicit nod from Microsoft that these advances represent may do more to realistically forward the cause of domain library reuse than anything we’ve seen so far in our experience with Microsoft and the Windows platform.  As these efforts progress, we should expect to see an emerging emphasis on Asset-based Software Engineering practices (ASE), and the need for software artifact management.  Companies like Flashline should begin to play increasingly more important roles in our lives as our development practices evolve towards the vision of folks like Charles Stack, Flashline founder, CEO, and all-around reuse visionary.

    Part II - Code

    The ADO .NET API Way

    You take the low road and I’ll have a working, well factored, maintainable app weeks before you do.

    If you have ever coded a data access layer based on the ADO .NET API, you know that there is more to performing CRUD than just SQL command text.  You will use instances of DataAdapter classes, connection classes, command classes, transaction classes, parameter classes, and occasionally event args classes.  To further bloat the data access code, you’ll need code to create command objects and their associated parameters for each of the SQL insert, update, and delete commands.  You will also need code to create command objects for each select statement your application will need to represent the various data retrieval scenarios for each scenario that the application must support for each business object.

    This says nothing about the code needed to coordinate connections, transactions, and the persistence operations that must be cascaded recursively from parent to children through the hierarchy of related business objects in a given scenario.

    To help illustrate this point, let’s take a look at an example.  Let’s take a deeper look at the code needed to support CRUD operations for Person data.  As described earlier, the Person table has three attributes: Id, Name, and Birthday.

    Here is the DDL to create the person table:

    CREATE TABLE [dbo].[Person] (

       [Id] [int] IDENTITY (1, 1) NOT NULL ,

       [Name] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,

       [Birthday] [datetime] NULL

    ) ON [PRIMARY]

    The example code is by no means a well-factored solution to object persistence.  The intention of the code is to merely demonstrate the body of code required to perform rudimentary CRUD operations with ADO .NET for a single table.

    If you have any familiarity with ADO .NET, this code should look relatively similar to you.  If you haven’t used ADO .NET previously, the example is typical of the code pattern for rudimentary persistence with ADO .NET.  It should be noted that this code is by no means to be considered a recommendation for how you should implement a persistence layer for your production systems.  There are many other issues to be taken into consideration when building an ADO .NET data access layer that are not presented in the example code.

    Without further adieu, here is the ADO .NET API-based code:

    using System;

    using System.Data;

    using System.Data.SqlClient;

     

    namespace ScottBellware.PersonExample

    {

       /// <summary>

       /// Provides CRUD operations for Person data.

       /// </summary>

       public class PersonDataAccess

       {

           private readonly string connectionString =

              "Database=PersonExample; Server=localhost; Integrated Security=SSPI";

     

           SqlConnection connection ;

     

           public PersonDataAccess()

           {

              // Conenction to the database

              this.connection = new SqlConnection(this.connectionString);

           }

     

           /// <summary>

           /// Create a person record in the database.

          /// </summary>

           /// <param name="name">The person's name.</param>

           /// <param name="birthday">The person's birthday.</param>

           public void CreatePerson(string name, DateTime birthday)

           {

              // Command object to execute SQL INSERT command text

              SqlCommand insertCommand = this.connection.CreateCommand();

              insertCommand.Connection = this.connection;

              insertCommand.CommandText =

                  "INSERT INTO Person (Name, Birthday) VALUES (@Name, @Birthday)";

     

              SqlParameter parameter;

     

              // Name parameter

              parameter = insertCommand.Parameters.Add("@Name", DbType.String);

              parameter.Value = name;

     

              // Birthday parameter

              parameter = insertCommand.Parameters.Add("@Birthday", DbType.String);

              parameter.Value = birthday;

     

              // Execute the command

              try

              {

                  // Open the connection to the database

                  this.connection.Open();

     

                  // Begin a transaction and assign it to the command

                  insertCommand.Transaction = connection.BeginTransaction();

     

                  // Execute

                  insertCommand.ExecuteNonQuery();

     

                  // Commit the transaction

                  insertCommand.Transaction.Commit();

     

                  // Close the connection

                  this.connection.Close();

              }

              catch (Exception e)

              {

                  // Rollback the transaction

                  insertCommand.Transaction.Rollback();

     

                  // Close the connection

                  this.connection.Close();

     

                  // Re-throw the exception

                  throw e;

              }

           }

     

           /// <summary>

           /// Retrieve a person from the database.

           /// </summary>

           /// <param name="name">Name of the person to retrieve.</param>

           public DataRow RetrievePerson(string name)

           {

              // DataAdapter needed for retrieve into DataTable

              SqlDataAdapter dataAdapter = new SqlDataAdapter();

     

              // Command object to execute SQL SELECT command text

              SqlCommand selectCommand = this.connection.CreateCommand();

              selectCommand.Connection = this.connection;

              selectCommand.CommandText =

                  "SELECT Id, Name, Birthday FROM Person WHERE Name = @Name";

     

              // Name parameter

              SqlParameter sqlParameter =

                  selectCommand.Parameters.Add("@Name", DbType.String);

              sqlParameter.Value = name;

             

              // Assign the selectCommand to the DataAdapter so that the DataAdapter.Fill()

              // method has a command to execute

              dataAdapter.SelectCommand = selectCommand;

     

              // Instatiate a DataTable to retrieve data into

              DataTable dataTable = new DataTable("Person");

     

              // Number of rows returned

              int resultCount = 0;

     

              try

              {

                  // Open the connection

                  this.connection.Open();

     

                  // Query

                  resultCount = dataAdapter.Fill(dataTable);

     

                  // Close the connection

                  this.connection.Close();

              }

              catch (Exception e)

              {

                  // Close the connection

                  this.connection.Close();

     

                  // Re-throw the exception

                  throw e;

              }

     

              // Initialize DataRow to return

              DataRow dataRow = null;

     

              // If data retrieved, assign the retrieved row to the row returned

              if (resultCount > 0)

              {

                  dataRow = dataTable.Rows[0];

              }

     

              return dataRow;

           }

     

           /// <summary>

           /// Update Person data.

           /// </summary>

           /// <param name="id">The Id of the person to update.</param>

           /// <param name="name">The updated name.</param>

           /// <param name="birthday">The updated birthday.</param>

           public void UpdatePerson(int id, string name, DateTime birthday)

           {

              // Command object to execute SQL INSERT command text

              SqlCommand updateCommand = this.connection.CreateCommand();

              updateCommand.Connection = this.connection;

              updateCommand.CommandText =

                  "UPDATE Person SET Name = @Name, Birthday = @Birthday WHERE Id = @Id";

     

              SqlParameter parameter;

     

              // Id parameter

              parameter = updateCommand.Parameters.Add("@Id", DbType.Int32);

              parameter.Value = id;

     

              // Name parameter

              parameter = updateCommand.Parameters.Add("@Name", DbType.String);

              parameter.Value = name;

     

              // Birthday parameter

              parameter = updateCommand.Parameters.Add("@Birthday", DbType.String);

              parameter.Value = birthday;

     

              this.ExecuteCommand(updateCommand);

           }

     

           /// <summary>

           /// Delete a Person row.

           /// </summary>

           /// <param name="id">The Id of the person row to delete.</param>

           public void DeletePerson(int id)

           {

              // Command object to execute SQL INSERT command text

              SqlCommand deleteCommand = connection.CreateCommand();

              deleteCommand.Connection = connection;

              deleteCommand.CommandText =

                  "DELETE FROM Person WHERE Id = @Id";

     

              SqlParameter parameter;

     

              // Id parameter

              parameter = deleteCommand.Parameters.Add("@Id", DbType.Int32);

              parameter.Value = id;

     

              this.ExecuteCommand(deleteCommand);

           }

     

           private void ExecuteCommand(SqlCommand command)

           {

              // Execute the command

              try

              {

                  // Open the connection to the database

                  this.connection.Open();

     

                  // Begin a transaction and assign it to the command

                  command.Transaction = this.connection.BeginTransaction();

     

                  // Execute

                  command.ExecuteNonQuery();

     

                  // Commit the transaction

                  command.Transaction.Commit();

     

                  // Close the connection

                  this.connection.Close();

              }

              catch (Exception e)

              {

                  // Rollback the transaction

                  command.Transaction.Rollback();

     

                  // Close the connection

                  this.connection.Close();

     

                  // Re-throw the exception

                  throw e;

              }

           }

       }

    }

    An important point should be made now that you’ve seen the code.  The Person table is really simplistic.  Although the ADO .NET code to maintain this table seems verbose, it is really quite manageable compared to the code required to manage a table that has more than three columns… as is probably the case for most real-world business apps.

    For every column in a real-world database table, you are going to need to code a representation for the command object.  That is, every column in the database table needs an associated parameter object.  As your tables become larger, the ADO .NET code becomes exponentially more massive.

    Now that we’ve written some static data services methods to provide persistence for the person table, let’s take a look at how these methods would be invoked by client code.

    The default constructor of the PersonDataAccess class initializes a connection that the methods of this class will use to perform CRUD operations.  Before any of these operations can be invoked, the PersonDataAccess class (obviously) must be instantiated.

    PersonDataAccess personDataAccess = new PersonDataAccess();

    Now, we can call CRUD operations on the data access object.

    Create a Person row named Bob whose birthday is System.DateTime.Now:

    personDataAccess.CreatePerson("Bob", DateTime.Now);

    Retrieve Bob’s row from the database:

    DataRow dataRow = personDataAccess.RetrievePerson("Bob");

    Change Bob’s name to “Bobby” and his birthday to System.DateTime.Now.  Note that for this example, we are assuming that Bob’s Id is already known:

    personDataAccess.UpdatePerson(bobId, "Bobby", DateTime.Now);

    Delete Bob’s row from the database:

    personDataAccess.DeletePerson(bobId);

    The ObjectSpaces Way

    Now that we’ve seen what it takes to build persistence support for the person table with ADO .NET API-centric code, let’s take a look at the same code from the perspective of ObjectSpaces.

    The glaring omission from the ADO .NET persistence code is the Person business object.  Remember that ObjectSpaces is a framework for persisting the state of business objects. 

    Here’s the Person class:

    using System;

     

    namespace ScottBellware.PersonExample.ObjectSpaces

    {

       public class Person

       {

           private int id;

           private string name;

           private DateTime birthday;

     

           public int Id

           {

              get { return this.id; }

              set { this.id = value; }

           }

     

           public string Name

           {

              get { return this.name; }

              set { this.name = value; }

           }

     

           public DateTime Birthday

           {

              get { return this.birthday; }

              set { this.birthday = value; }

           }

     

           public Person() {}

     

           public Person(string name, DateTime birthday)

           {

              this.name = name;

              this.birthday = birthday;

           }

       }

    }

    There should be nothing too surprising about the Person class.  It is merely a lightweight class for hold the Person business object data.

    Here is the ObjectSpaces persistence code:

    using System;

    using System.Data;

    using System.Data.SqlClient;

    using System.Data.ObjectSpaces;

    using System.Data.Mapping;

     

    namespace ScottBellware.PersonExample.ObjectSpaces

    {

       /// <summary>

       /// Provides CRUD operations for Person business object.

       /// </summary>

       public class PersonPersistence

       {

           private SqlConnection connection = null;

           private ObjectSpace objectSpace = null;

     

           /// <summary>

           /// Initializes a new instances of PersonPersistence.

           /// </summary>

           public PersonPersistence()

           {

              // Initialize the connection to the database

              this.connection =

                  new SqlConnection(

                  "Database=PersonExample; Server=localhost; Integrated Security=SSPI;");

             

              // Initialize the objectSpace

              this.objectSpace = new ObjectSpace(@"C:\Mapping\msd.xml", connection);

           }

     

           /// <summary>

           /// Saves a person to the database.

           /// </summary>

           /// <param name="person">The Person instance to save.</param>

           public void CreatePerson(Person person)

           {

              // Start tracking changes to the Person instance

              this.objectSpace.StartTracking(person, InitialState.Inserted);

     

              // Save the person data

              this.SavePerson(person);

           }

     

           /// <summary>

           /// Retreive Person data from the data base.

           /// </summary>

           /// <param name="criteria">Expression to filter the query.</param>

           /// <returns>The person instance that matches the criteria.</returns>

           public Person RetrievePerson(string criteria)

           {

              // Instantiate a Person and retrieve the data

              Person person = (Person) this.objectSpace.GetObject(typeof(Person), criteria);

              return person;

           }

     

           /// <summary>

           /// Update the database with changes made to the Person instance.

           /// </summary>

           /// <param name="person">The Person instance to update.</param>

           public void SavePerson(Person person)

           {

              try

              {

                  // Begin the transaction

                  this.objectSpace.BeginTransaction();

     

                  // Save the changes to the Person

                  this.objectSpace.PersistChanges(person);

     

                  // Commit the transaction

                  this.objectSpace.Commit();

              }

              catch (Exception e)

              {

                  // Rollback the transaction

                  this.objectSpace.Rollback();

     

                  // Re-throw the transaction

                  throw e;

              }

           }

     

           public void DeletePerson(Person person)

           {

              // Mark the Personb instacne for deletion

              this.objectSpace.MarkForDeletion(person);

     

              // Save the changes to the Person

              this.SavePerson(person);

           }

       }

    }

    What should be immediately evident is how relatively succinct the ObjectSpaces-based code is.  Since the ObjectSpaces persistence framework is performing all of the data access, there is no ADO .NET code to write (or maintain).

    The default constructor of the PersonPersistence class initializes a connection that will be used by an instance of the ObjectSpace class to coordinate updates and retrieves to and from the database.  The constructor also initializes the ObjectSpace instance.  Before any of the persistence operations can be invoked, the PersonPersistence class must be instantiated.

    PersonPersistence personPersistence = new PersonPersistence();

    Now, we can call CRUD operations on the persistence object for a Person instance.

    Create a Person row named Bob whose birthday is System.DateTime.Now:

    Person person = new Person("Bob", DateTime.Now);

    personPersistence.CreatePerson(person);

    Retrieve Bob’s data from the database, encapsulated in the Person instance:

    person = personPersistence.RetrievePerson("Name = 'Bob'");

    Change Bob’s name to “Bobby” and his birthday to System.DateTime.Now:

    person.Name = "Bobby";

    person.Birthday = DateTime.Now;

    personPersistence.SavePerson(person);

    Delete Bob’s row from the database:

    personPersistence.DeletePerson(person);

    Readability and Useability

    Using business-object oriented syntax lends more readability to the code than can typically be achieved from API-based code.  Readability is to developers what usability is to end users.  As a user of source code, usability of the code for a developer should always be taken into consideration in that it has the same impacts on software development that usability-challenged user interfaces have on an end user’s work.

    By opting for API-based code, you are opting out of a certain level of usability and opting into higher performance.  These two elements of software development typically sit at opposite ends of a project’s value system spectrum and they should be balanced against requirements.

    The ObjectSpaces code offers really good coder experience, and this is exemplified by the clean, crisp API it presents for persistence.  Updates to the database happen through a single API element: the ObjectSpace.PersistChanges() method.

    The following code is from the ObjectSpaces example above:

    this.objectSpace.PersistChanges(person);

    The ObjectSpaces syntax for transactions greatly simplifies coding for this important facet of persistence as well.  Rather than require the developer to get a transaction instance and assign it to command objects, the ObjectSpace class conveniently encapsulates transactions and provides a clean API to control them:

    The following lines of code are from the ObjectSpaces example above:

    this.objectSpace.BeginTransaction();

    this.objectSpace.Commit();

    this.objectSpace.Rollback();

    Mapping with ObjectSpaces

    Of course, none of this light-weight code would be possible without the mapping specifications.  The ObjectSpaces example above makes use of three schema definition XML files.  One is actually quoted in the initialization of the ObjectSpace instance in the constructor:

    this.objectSpace = new ObjectSpace(@"C:\Mapping\msd.xml", connection);

    The three files used in the example are rsd.xml, osd.xml, and msd.xml.  The rsd.xml file provides for the specification of information for the data table that the Person class maps to.  The osd.xml provides information about the Person class itself.  The msd.xml is used to provide information to the ObjectSpace instance regarding where to find the other two files, as well as information on how to map the business object to the database table.

    Here are the contents of each of these files:

    rsd.xml:

    <rsd:Database Name="PersonExample" xmlns:rsd="http://schemas.microsoft.com/data/2002/09/28/rsd">

      <rsd:Schema Name="dbo">

        <rsd:Tables>

          <rsd:Table Name="Person">

            <rsd:Columns>

              <rsd:Column Name="Id" SqlType="int" AutoIncrement="true" IncrementStep="1" />

              <rsd:Column Name="Name" SqlType="nvarchar" Precision="50" />

              <rsd:Column Name="Birthday" SqlType="datetime" />

            </rsd:Columns>

          </rsd:Table>

        </rsd:Tables>

      </rsd:Schema>

    </rsd:Database>

    osd.xml:

    <osd:ExtendedObjectSchema Name="PersonSchema" xmlns:osd="http://schemas.microsoft.com/data/2002/09/20/persistenceschema">

      <osd:Classes>

        <osd:Class Name="Person">

          <osd:Member Name="id" Key="true" KeyType="AutoIncrement" Alias="Id" />

          <osd:Member Name="name" Alias="Name"/>

          <osd:Member Name="birthday" Alias="Birthday"/>

        </osd:Class>

      </osd:Classes>

    </osd:ExtendedObjectSchema>

    msd.xml:

    <map:MappingSchema xmlns:map="http://schemas.microsoft.com/data/2002/09/28/mapping">

      <map:DataSources>

        <map:DataSource Name="PersonExample" Type="SQL Server" Direction="Source">

          <map:Schema Location="rsd.xml" />

          <map:Variable Name="Person_Rows" Select="Person" />

        </map:DataSource>

        <map:DataSource Name="PersonSchema" Type="Object" Direction="Target">

          <map:Schema Location="osd.xml" />

        </map:DataSource>

      </map:DataSources>

      <map:Mappings>

        <map:Map SourceVariable="Person_Rows" TargetSelect="ScottBellware.PersonExample.ObjectSpaces.Person">

          <map:FieldMap SourceField="Id" TargetField="id" />

          <map:FieldMap SourceField="Name" TargetField="name" />

          <map:FieldMap SourceField="Birthday" TargetField="birthday" />

        </map:Map>

      </map:Mappings>

    </map:MappingSchema>

    Part III - Wrapping Up

    Stored Procedures and Cultural Barriers

    Aside from any technological and skills risks that need to mitigated, there is a mountain of human issues to overcome before we arrive at a mainstream understanding and acceptance of object-relational persistence.  Technologists are folks who often get set in their ways and sometimes it requires an act of several gods to bring in new ideas (Option Strict comes to mind).  There are a couple barbs that are not going to rub many technologists the right way when it comes to all this object-relational stuff.

    The most glaring issue that many traditional API-based data-access folks have with object-relational persistence is the edict that with object-relational persistence, the use of stored procedures is no longer a given.

    Object-relational mapping is about mapping business objects to database tables, not about mapping business objects to stored procedures.  For years we have had it drilled into our collective conscious that stored procedures are an absolute must when it comes to defining an interface between client code and a database.  If you think about the drivers for this approach, it soon becomes clear that the whole issue is not so well founded – at least in light of a good persistence framework.

    The canonical justifications for stored procedures being a must-have part of persistence architecture are performance, encapsulation and reuse, and security.  The performance and encapsulation issues are mitigated by persistence frameworks, and the security issue is actually justifiable – to a point.

    We’ve already covered how persistence frameworks derive the data manipulation language for a given business object from the business object’s structure.  For persistence frameworks that make use of parameterized queries with SQL Server, the performance degradation is minimal to the point of being practically irrelevant.  Parameterized queries benefit from the same execution plan caching that benefit stored procedures.  With a data architecture that makes multiple one call per row to the database for either insert, update, or delete operations, stored procedures and parameterized queries will yield similar performance.

    In light of this, why do we use stored procedures?  Encapsulation and reuse are certainly a couple of good reasons.  If all of our apps used parameterized queries that emanated from middle-tier components, then we might have hard coded SQL DML littered throughout our apps.  The repetition that this would cause would represent an unjustifiable cost liability.  With stored procedures, our middle-tier components only have to be made aware of the stored procedure names, and the stored procedures end up being the only place where the DML is located.  Unless you’re using a persistence framework, at which point, stored procs are mostly irrelevant.

    Since the persistence framework can derive to DML from the business objects, the DML is never hard-coded into the middle tier, and subsequently, neither are the names of the stored procedures.  The middle-tier is effectively decoupled from a specific database instance, and the database tables are not coupled to the stored procs.  Again, object-relational persistence is about mapping business objects to tables, not stored procedures.

    Stored procedures are a necessary part of an application in the absence of a persistence framework.  Otherwise, they are a hindrance to agility.  Without stored procedures and with a framework, development progresses at an accelerated rate, leading to greater efficiency and a more enjoyable project experience for all involved.  For new projects, this means the coveted quicker time-to-market, and for apps in production this means greater responsiveness to new business demands.

    When an application is initially being developed, its data model can remain (necessarily) fluid up until the very end.  If the interface to the database is through stored procedures, then every time the data model is changed all of the relevant stored procedures need to be fixed to reflect the changes to the data model.  Consider that each table will have an insert, update, and multiple retrieve stored procedures that will be affected by any changes to the data model.  They’ll need not only to be edited, but each one will need to be validated as well.

    To incorporate just the structural change to an application’s domain, the changes will need to be reflected in the data tables, the stored procedures, and the business classes.  When a persistence framework is employed, the changes to the stored procedures are not made since there are no stored procedures to change.  Stored procedures are – well – procedural.  Stored procedures are not structural elements.  It should seem odd right off the bat that effecting changes to procedural code would be necessary when incorporating a simple structural change to a domain model.  Stored procedures represent structural elements, but they are not structural elements themselves.  They are somewhat of a design aberration.

    Stored procedures are simply a hindrance to rapid prototyping.  If we accept that rapid prototyping is fundamental to achieving stakeholder acceptance and early validation of developer assumptions, then we begin to see that stored procedures introduce an inherent cost liability to a software project.  Without CRUD stored procedures, there are fewer elements to maintain, fewer things to test, fewer things to break, fewer things to deploy, fewer things to version, and fewer things to manage.  The same benefits apply to applications in production.  Changes can be incorporated quickly and put into production faster without the stored procedures.

    The maintenance and management of stored procedures add more steps to the process of software development and improvement and tend to increase TCO.  It should be elementary that any additional steps in a process are going to have associated costs.  Eliminating process points whose justifications balance tenuously on unforsaken dogma is as instrumental to achieving agility as is streamlining existing activities.

    Stored procedures are not necessarily the easiest artifact to create and maintain either.  It’s not like the tool support for T-SQL is as good as it is for C# and VB .NET.  Developers yield greater productivity when working with languages like C# and VB .NET than with T-SQL.  And since T-SQL runs in the context of the database engine, it is often not as easy to isolate and directly test without the use of specialized testing tools and frameworks.

    There are still some things that T-SQL is appropriate for, but run-of-the-mill CRUD is not one of them – especially when there are persistence frameworks that can bring more value to your efforts.

    There is still the outstanding question of security.  One of the most compelling justifications for T-SQL is that it allows for a secure encapsulation of the tables behind a secured call-level interface.  Access to the tables themselves is turned off and access to the stored procs is turned on.  The most common benefit I’ve heard for this approach is that it helps to stop someone using a data browser from making accidental, destructive changes to production data.  Well, if there are folks inside your organization who can access and change production data with a data browser, then I expect you’ve got more problems than stored procedures can solve.

    Stored procedures do add an extra measure of security.  And in our age of defense in depth, more security is often seen as a good thing.  And just like any good thing, it has to be measured in appropriate doses.  For the most part, we can get by without using stored procedures as security gateways to data.  There are other existing and emerging approaches that solve this problem without the imposition of the development overhead of stored procedures.

    So, should we expect our DBA’s to give up on stored procs just because we say so?  No, probably not.  To bring object-relational persistence into the fold, many of us are going to have to do some evangelizing to the DBA team.  But in the end, that’s going to be a good thing.  The more you can roll your DBA into your development team, the smoother your application development is going to be.

    You’re going to need to have a really good relationship with your DBA when Yukon comes along and you let him know that you’re going to start installing your CLR code into “his” database.

    DBA’s are pre-disposed by the nature of their jobs and the resources that they’re responsible for to be very protective of their data servers.  Over-coming this cultural pre-disposition is not going to be a walk in the park in all cases.  With SQL CLR, many developers are going to be asking for an unheard of level of access to the heretoforeprotected database engine.  DBA’s are going to want to know what the performance impacts are going to be.  They’re going to want to know that you’re not using CLR code when you should in fact be using T-SQL.  They’re going to want to know about security.  They’re going to want to know about deployment.  And as .NET developers, it’s largely going to be up to us to unfurrow their brows and do the legwork to get them the information they need and provide them with the benefit of our .NET skills as they too take a ride on the .NET learning curve.

    Conclusion

    Object-relational persistence is an option that is provided to you in addition to the persistence approaches that you’re already aware of.  Object-relational persistence can provide great benefits to your work in developing data-based applications on .NET.  For some applications, ADO .NET API-based persistence is still going to be the right choice.  To make those choices, we’re all going to need to become better informed.  With the numerous data access strategies in Whidbey, Microsoft is increasing our study workload, but we’re also being provided options that we might not have had previously, and these options promise to add a great deal of value to a great deal of development efforts.

    Whidbey, ObjectSpaces and Yukon are not due out until late 2004.  This gives you plenty of time to do a bit of digging and collect the knowledge you’ll need to have at the next big .NET release party.  In the mean time, there are a couple frameworks already available for Microsoft that you can use now to get into the swing of object-relational persistence.  Take a look into EntityBroker or Norpheme, or any of the tools listed at the DotNetPersistence MSN Group (link below).

    Links

    Here are a few links to some object-relational resources that will start you off on the pursuit of ever more data access reading:

    Special Thanks

    Many thanks to Dave Foderick for technical review and