Microsoft Corporation
Author: Henk van de Crommert, Dennis Mulder
Originally Published: October 2009
Applies to: BizTalk Server 2009
Summary: This document provides developers with techniques for designing, developing, and deploying solutions within Microsoft® BizTalk® Server 2009 in conjunction with Team Foundation Server 2008. This paper was based on "Developing Integration Solutions with BizTalk Server" by Angus Foreman and Andy Nash and updated for BizTalk Server 2006 R2, 2009 and Team Foundation Server. The BizTalk Server 2006 R2 version of this paper can be found here: http://go.microsoft.com/fwlink/?LinkId=184279.
Note: To download a copy of the document and sample files, go to http://go.microsoft.com/fwlink/?LinkId=135760.
This guide provides developers with techniques for designing, developing, and deploying solutions within Microsoft® BizTalk® Server 2009. It contains examples of team approaches for developing with BizTalk Server in conjunction with Visual Studio® Team Foundation Server 2008, and illustrates techniques that can increase development efficiency.
The guide also provides hints and tips that can decrease the time and effort spent debugging and testing. It emphasizes the team development process and team development setup.
This guide focuses on projects that use the core messaging and orchestration functionality of BizTalk Server. It does not cover in detail additional areas such as Business Activity Services (BAS), Business Activity Monitoring (BAM), Line of Business Adapters, RFID, the WCF Adapter or the XREF, or Business Rules Engine (BRE).
The following users should read this guide:
This guide assumes that the reader is familiar with BizTalk Server (2004 or higher) and Team Foundation Server and understands the core concepts of the products.
The document is based upon a subset of the project phases that are described in the Microsoft Solutions Framework, namely:
The Microsoft Solutions Framework (MSF) contains proven practices for planning, building, and deploying small to large information technology (IT) solutions. MSF is made up of principles, models, and disciplines for managing the people, process, and technology elements that are encountered on most projects. For more information about MSF, see the Process Templates and Tools section on the Team System Developer center at Process Templates and Tools (http://go.microsoft.com/fwlink/?LinkId=163128).
This section describes a lightweight technique for gathering integration requirements and translating them into a list of BizTalk Server artifacts to be delivered. It discusses a typical BizTalk Server integration scenario. Readers can compare this typical scenario with their own requirements and create plans and documents accordingly.
The planning phase also includes setting up the development environment and the development processes to provide a stable and efficient development environment. Making changes to the development environment or processes during the development phase can be very disruptive, so time spent planning them is likely to save time later in the project.
Planning for the integration scenario includes the following:
Like most software development problems, the successful delivery of an integration solution depends upon the collection and collation of large amounts of information prior to the design process. The following sections suggest a lightweight approach to gathering the information that is typically useful to the design and development of an integration solution. This information includes the business processes and the messages that the solution requires.
An important function of the planning process is to draw up the master list of business processes that you will deliver with the integration solution. This task ensures that a single, comprehensive list of all the business processes exists, along with a list of all the business rules that apply to those processes. You can use this list to plan and design the development of the BizTalk Server artifacts that are required to carry out those business processes.
One possible approach to developing the list of business processes is to devise logical categories of related business processes and then list all the processes within each category. For example, categories like "User Management," "Supplier Transactions," "Customer Transactions," "Catalogue Management," or "Credit Control" are often meaningful to the business and typically consist of several related business processes (with related system and messaging requirements).
In the example scenario, the business process list is documented in the following tabular format.
Table 1 - Example processes and categories
Business process category
Process name
The team now has a master list of business processes that need to be developed. The next step is to capture the process details that are significant to the integration solution. Detailed information that is typically captured about each process falls into two categories—logical process information and implementation information. You may also want to diagram the high-level processes so that developers will have a starting point for creating BizTalk orchestrations.
This information describes the process in the abstract, without being concerned about the actual implementation technologies. It is typically gathered by interviewing the owners of the business processes rather than IT or system specialists. To gather logical process information, you might perform actions or answer questions like the following:
Information related to the implementation is often dictated by the design and constraints of the existing systems being integrated. It is important to keep in mind the constraints placed around processes by the environment and other systems. These constraints often significantly influence the final design. To gather this information you might perform actions like the following:
The process of examining all business processes produces a list of the messages that are required to deliver the solution. When defining messages consider the following:
Note As part of BizTalk Server 2009 Microsoft ships Solution Accelerators that contain artifacts that can be used in certain specific industry scenarios, such as Health (HL7, HIPAA), Supply Chain (Rosettanet) and Finance (SWIFT). For more information see Accelerators (http://go.microsoft.com/fwlink/?LinkId=184282).
When describing the complexities of an integration solution, it is important that you use a strong set of terms to ensure clarity throughout the planning, designing, and development phases. You begin this process in the planning phase by identifying the naming conventions you use to describe systems and their operations. The naming conventions in the following table can be applied to the integration solution.
This document does not present a comprehensive list, but instead provides terms found useful on a variety of projects. For a comprehensive approach to integration terms, see the patterns & practices Integration Patterns whitepaper (http://go.microsoft.com/fwlink/?LinkID=128975).
Table 2 – Integration Terms
When describing data or messages between systems and the integration layer, it can also be useful to use a standard notation to avoid confusion over the direction of message flow. This is particularly useful when multiple request-response operations exist between systems.
One possible approach is to describe message instances between systems using the notation of "incoming" and "outgoing" where "incoming" and "outgoing" are understood from the point of view of the integration layer. The following messages are examples:
Messages originating from the integration layer:
Figure 1 shows the messages targeting the integration layer.
Figure 1 - Message Direction and Naming
While it can be helpful to name message instances in an orchestration, naming the message schema with a name describing direction may cause confusion in other areas of the solution, especially if the schemas are reused or exchanged with other parts of the system.
Many line-of-business messaging applications, especially financial systems, use a numeric system for identifying the messages (that is, the message type, or schema, and the destination system/direction), for example, 9820000-1104 in ISO 8583, MT 512 in SWIFT. In such cases you can either maintain the numeric notation or use a combination of numbers and text to identify messages (such as MT_512_Securities_Trade_Confirmation).
It is not uncommon for a complex process to use 10 to 20 related incoming and outgoing request and response messages. A naming convention applied to orchestration messages and variables can aid clarity. A standardized approach can reduce possible errors and confusion and make documentation more effective.
In some cases, naming conventions can produce very long names that are not easy to use in Visual Studio and the BizTalk Management Tooling, given that the dialog boxes and panes are narrow. In these cases consider using a coding scheme such as CRM_Add_Customer_IReq and CRM_Add_Customer_Oresp, where IReq is an incoming request and OResp is an outgoing response. There are different naming conventions for the BizTalk Server. One of it is the Application Naming Conventions.
When describing an integration solution, there is typically a large amount of detail to be captured about each of the systems and processes. It is useful to start by listing some common properties of each of the systems involved. It is typical to collect this information for each system participating in the integration solution. The following table shows some common system attributes.
Table 3 - Common attributes of systems
In our example scenario the planning process starts by documenting the systems as shown in the following table, with more detail being included as the planning and design processes uncover more information. The following table lists examples of the type of data that is typically recorded. The systems described here represent a customer relationship management system, and an account management.
Table 4 – Example System Information
Typically used in an interactive manner, where a change needs to be made to a customer or product within a matter of seconds or minutes. (Synchronous).
Outgoing: Batch-based file outputs typically running nightly, weekly, or quarterly processes. Files written to a UNIX file share location.
Outgoing message formats: CRM-specific XML schema. Definition available as XSD file. Actual schema used is dependent upon message type (of which six exist).
Incoming message formats: Custom XML message as defined by accounts system. Sample XML messages available.
Outgoing message formats: Positional ASCII flat file with headers and footers. Definition available as Word document.
While the system provides access to customer information, it is not the master for customer information (which is held in Invoicing).
Outgoing: Due to its batch nature, under certain circumstances the volume of data passed from the Accounts system can be very large. Typically these large volumes occur during processes like quarterly or year-end billing runs. These large volumes are significant because it is often the case that there exists a relatively short duration in which all this information must be processed. (For example, there might be 1000 files of 3 MB each.)
Outgoing: By default the CRM system's outgoing HTTP requests do not pass credentials to the target system.
Outgoing: Configurable credentials are used to write outgoing data files to a specific location.
Having previously defined the systems and business processes in scope, the next step is to document the interfaces that are required to enable the integration layer to connect to the upstream and downstream systems. One way to capture this information is to list the processes and then detail each of the interfaces required by each process. If you are designing a system to replace an existing integration solution it can be helpful to obtain actual copies of all the types of messages that are currently going through the various systems.
The following list describes typical information that you should gather while you are categorizing interfaces:
Later in the planning process this list of interfaces can be further extended to help produce a detailed list of functional requirements that can be used as the basis of a BizTalk Server project.
A helpful way to document those functional requirements is in the form of test cases with specific input files and expected output files. This typically means collecting a lot of data early in the project, but in all likelihood this data will be required when development or testing starts, and by gathering it early in the development process any unanticipated challenges can be spotted early on.
When developing an integration solution, the definition of interfaces is a significant part of the design process. The interface is typically considered as a "contract" between each of the parties involved in the development of the integration solution. The contract verifiably ensures that parties are developing compatible systems.
When documenting interfaces it is useful to be able to produce detailed descriptions of the interfaces as part of the contract. The best way to document the contract is with a domain-specific language like Web Services Description Language (WSDL), or some other consumable form of metadata. All parties can use these descriptions to validate that their development meets the interfaces expected by upstream and downstream systems. When working with XML-based solutions like Web services and BizTalk Server, standards exist to produce these detailed contract descriptions. The following table shows the contract formats for various interface types.
Table 5 - Interface types and contract formats
At the end of the information-gathering process, the high-level requirements of the integration solution are captured in a series of related documents that list unique business processes, systems, and interfaces. As the design process progresses the level of detail captured increases. Prior to development it is typical to produce a list of BizTalk Server artifacts to be developed during the build phase.
Additionally, the planning phase may well identify the number and type of test harnesses or stub systems that will be required to meaningfully develop and test the business processes. The planning phase may also identify the data requirements needed to allow development to proceed. This may include migration requirements and cross-reference data.
You use this list of requirements to aid in designating teams, planning schedules, and identifying required development resources.
This section describes tips that you can use during the development of a BizTalk Server integration scenario. Readers can use this information to apply the best practices that have been gathered during the development of BizTalk Server Integration Solutions across the world.
The approaches described in this section relate to the general approaches needed when developing within a team. Many of the concepts in this document are covered for general .NET development in the patterns & practices Guide Team Development with Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129476).
When developing a solution within a team it can be very helpful to define owners and development teams for distinct parts of the solution early in the development process. Typically named individuals (or groups of individuals) are given ownership of logical parts of the solution. Useful divisions of work can include allocating ownership of:
Typically the owner is responsible for understanding their part of the solution and acts as the "gateway" to any requested modifications. In this way when a modification to a design is proposed, the individual can assess the effect of the proposed changes and then allow or disallow that change. This is particularly important when working with shared parts of the solution, like shared message standards where a change can have a potentially significant impact upon the development team and upstream or downstream systems.
The following sections discuss solution structures and development models for a BizTalk Server project with multiple developers.
The term "solution structure" refers here to the logical and physical structure used to store all the related BizTalk Server files and folders required to produce, develop, test, and deploy a production solution.
The term "team development model" refers to the relationship between Visual Studio 2008 projects and solutions, Microsoft Team Foundation Server source configuration, and the setup and processes used for a team of developers using common resources across multiple workstations.
It is important to enforce a solution structure. A BizTalk Server solution that is built to meet business requirements typically consists of many separate related artifacts. To avoid rework and build failures, these artifacts need to be managed throughout the project from development, test, acceptance/staging up into the deployment on production. A typical BizTalk Server solution will contain the artifacts in the following table.
Note This list does not include artifacts related to Business Activity Monitoring (BAM) and the Business Rules Engine (BRE).
Table 6 - Typical Visual Studio 2008 and BizTalk Server Solution Entities
The solution structure is important to a team of BizTalk Server developers for the following reasons:
A typical BizTalk Server solution structure is composed of the following logical elements:
There are many ways to create a BizTalk Server application composed of maps, schemas, orchestrations, pipelines, and so on. The most intuitive way is to place all of the BizTalk artifacts in one project, compiled and then deployed as a single unit. This method has the benefit of simplicity but is less effective when you take into account required modifications and project upgrades (new versions).
Using this model, when modifications or upgrades are needed, you must first undeploy the complete application. This means stopping the orchestrations and potentially affecting related entities. For example, if you use custom pipelines, the undeploy operation causes the pipeline bindings to default back to the pass-through pipeline. After you deploy a change, you must reconfigure any location that uses custom pipelines. The BizTalk Documenter (see Resources) can help out here in determining what the original pipeline settings were. Other implications you might encounter are versioning requirements subsequent to your initial release. When updating your BizTalk project you will end up having to redeploy your entire application as well. Working in a single-project approach also has implications when working in teams. Chances are team members have to touch the same files often which can cause merging challenges.
As an alternative to using the single-project approach, consider how the application is structured and look for ways to split the artifacts into logical groups. A typical way to manage this is to place orchestrations, pipelines, schemas, and maps in their own projects, often on a per use-case basis. If a pipeline changes, just a single assembly can be undeployed, and there is no need to undeploy maps or schemas. In the same way, if a map changes there is no need to undeploy the schemas. This approach favors components reuse and at the same time minimizes cross-reference dependencies.
There are implications every time you version the map. You may have to update and redeploy the Orchestration if this map is referenced in there. Additionally, consider dividing suborchestrations (that is, orchestrations that are called by other orchestrations) into their own projects.
Remember that a project can only be deployed as a single unit. If the task of redeploying a single project begins to involve the rebinding of unrelated resources, then you should consider splitting the project into smaller units that allow the deployment and binding of related items only.
Note The preceding approach is a "horizontal" division of work. You may want to consider a vertical division as well (that is, by project or business process functionality). For example, if a pipeline or schema is updated it should only affect the business project that it is related to. This is relevant if you have the requirement to avoid shutting down a CRM integrator when the ordering system updates a schema.
The following table shows the structure of a typical project.
Table 7 - Typical Project Structure
Standard schemas project
("Shared Schema")
("Email Handler")
Team development with BizTalk Server has close parallels with team development of Web applications. Both models typically have dependent elements that can be shared between the team members or a user in isolation.
The development model for BizTalk Server proposed in this paper is based upon the "isolated model" of development. The models available for BizTalk Server development are "isolated," "semi-isolated," or "non-isolated." In the isolated model no elements of the developer environment are shared (with the exception of access to a common Visual Studio Team Foundation Server environment). So every developer has its own instance of BizTalk Server, the BizTalk Server databases, SQL Server etc. When using semi-isolated and non-isolated models increasing areas of the developer environment are shared.
The nature of the BizTalk Server runtime means that isolated development typically provides the most effective approach. BizTalk Server developments are typically composed of several distinct environments, as shown in the following table.
Table 8 - BizTalk Server Environments
Composed of the BizTalk Server runtime and deployment tools.
The team development model described below details an approach to configuring the development tools, developer runtime, and shared runtime to provide an isolated developer environment.
Using an isolated model a developer edits, debugs, and runs functional tests completely isolated from other developers. They have their own self-contained development workstation with a local BizTalk Server Group. Access to the master source files is controlled via Team Foundation Server. Figure 2 illustrates an isolated development model.
Figure 2 - Isolated BizTalk Server Development Model
The isolated model of BizTalk Server development provides the following benefits:
A shared or non-isolated model can be used for developing with BizTalk Server if it is absolutely required, but developer efficiency may be reduced by the lack of the benefits listed above.
In our experience of developing BizTalk Server solutions virtualization can be used to efficiently create an isolated development environment and within Microsoft Services this is the standard approach used on nearly all BizTalk Server developments. Using virtualization typically saves the development teams a considerable number of hours effort (per developer) to create the development environment and is an extremely efficient way to enforce a standard development environment. It has additional advantages including the ability to efficiently "package up" of a development environment including an easy refresh when the environment is not functioning properly anymore or new versions of products and tools become available. This technique is often used for scenarios like sharing best practice, handing over issues between teams, testing possible server configurations, etc.
For additional information on virtualization, see the following resources:
For more information about Hyper-V, the successor to Virtual Server for Server platforms which is part of Windows Server® 2008, see Virtualization with Hyper-V (http://go.microsoft.com/fwlink/?LinkId=127729).
Setting up a BizTalk Server development environment typically requires the following:
An alternative to performing these steps on every developer's workstation is to use the services provided by virtualization. The typical environment to use on Client Operating Systems such as Windows Vista® or Windows 7 is Microsoft Virtual PC. Virtual PC allows users to run multiple PC-based guest operating systems simultaneously on the host operating system. Microsoft Virtual PC allows users to run multiple PC-based operating systems simultaneously. For example a user with Virtual PC installed on their Windows computer can run Windows Server 2003 or Windows Server 2008 within the Virtual PC. The benefit to a BizTalk Server development team lies in the fact that Virtual PC uses Virtual Hard Disks files (VHD files) that can be "copied" between computers. Although there are certain caveats with this approach, described later in this document, it is a very practical approach. In essence a BizTalk Server development team can invest time in creating a standard BizTalk Server developer virtual hard disk and then push that image to every member of the development team who gains a preconfigured BizTalk Server development environment. The VHD format allows you to interchange the VHD files between all Microsoft virtualization products, including the virtualization platform released with Windows Server 2008, Hyper-V.
Follow these steps to create a virtual PC virtual hard drive containing a BizTalk Server developer environment.
Note Not sysprepping is not recommended as using multiple machines with the same name will cause issues with Team Foundation Server Working Folders.
When developing on a Virtual computer, a useful approach can be to create a second hard drive (.VHD file) that holds the source code for projects. If this approach is used across the whole team with the same drive letter assigned to the drive then this provides a consistent set of paths for the solution structures. A major advantage of this approach is that the separate VHD can be significantly smaller and as such easier to copy between remote locations.
It is possible to develop and debug BizTalk Server applications while running the BizTalk Server developer environment on a remote computer accessed by Remote Desktop or Terminal Services. It is not recommended to do this on a multi user platform, because users will have to share the same BizTalk Server installation. However, teams using this approach should be aware of the following limitations:
If the BizTalk Server environment is running on Windows Server 2003 it is possible to connect to the console session rather than starting a new user session. This can be useful in scenarios where debugging information or the like is being written to the console.
To connect to the Windows Server 2003 console using remote desktop use the "/ console" or "/admin" (Windows Vista, Windows 7 and Windows Server 2008) command-line option, for example:
mstsc.exe <RDP connection file name> /console
In Windows Vista and Windows Server 2008 the new /console option in mstsc.exe is replaced by the /admin option for backwards compatibility. Windows Vista and Windows Server 2008 don't have a console session "0" anymore.
mstsc.exe <RDP connection file name> /admin
Note Only a single user can connect to the console session at the same time.
Visual Studio solution files (with the .sln file extension) are used to group related projects together and are primarily used to control the build process. You can use solutions to control build dependency issues and control the precise order in which contained projects are built. Partitioning describes the processes of creating Visual Studio 2008 solution files that divide up the multiple projects that compose a complete BizTalk Server solution. The approach used to partition BizTalk Server solutions and projects has a significant impact on the development and build processes in a team environment. There are three main approaches to consider when partitioning solutions and projects, as shown in the following table.
Table 9 – Solution Partitioning Approaches
The partitioned single solution approach is often applicable, but is the most complex to set up. For these reasons this approach is documented in the appendix of this document. Unless you have very good reasons to use a multi-solution model, you should avoid this approach.
When developing more complex BizTalk Server solutions systems it can be advisable to reduce the number of projects and source files required on each development workstation by grouping related sets of projects together within separate sub-solution files. This approach can be particularly beneficial if the complexity of the BizTalk Server solution means that loading, navigating, and rebuilding projects can take a significant amount of time.
This partitioned single solution allows developers to work on separate, smaller subsystems within the inner-system boundary. The following diagram illustrates the partitioned single solution model. Notice how separate solution files are used to allow you to work on smaller subsystems contained within the inner-system boundary. Also note how this results in projects being contained within more than one solution file. For example, in Figure 3, Projects D and H are in a total of three solution files including the master solution.
In the partitioned single solution model:
Figure 3 - Partitioned Single Solution
For more information about partitioning Visual Studio solutions see Chapter 3, "Structuring Solutions and Projects," in the Team Development with Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129476) document referenced at the beginning of this section.
As described in the previous section, single solutions and partitioned single solutions use project references to create references between dependent BizTalk Server projects. This is significant because project references provide significant advantages over file-based (DLL) references. Specific benefits include:
When working with project or file references, do not change the default "Copy Local" attributes for a referenced project or DLL.
For more information about partitioning Visual Studio solutions see Chapter 6, "Managing Source Control Dependencies in Visual Studio Team System," in the Team Development with Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129476) document referenced at the beginning of this section.
The file system folder structure used to store the BizTalk Server integration solution must accommodate not only the BizTalk Server solutions and projects, but also all the additional artifacts that are required to develop, build, test, and deploy a BizTalk Server solution. The folder structure must also be compatible with the structure used within the team's Visual Studio Team Foundation Server repository to ensure efficient integration with the common source control tasks.
The folder structures described here is based upon typical BizTalk Server requirements and is designed to comply with common Team Foundation Server tasks. It can be modified to meet the requirements of a given development team.
When creating the source control structure, keep in mind the following points:
The drive need not be the C drive, but if a drive other than C is used, make sure that all developers have a similarly named drive available. In case you want to use drive letters that are not available on all computers, the SUBST shell command can be used to create drive letters mapped to physical folders, for example:
SUBST m: c:\BiztalkDev
When the preceding command is executed at the command prompt, it creates a drive letter m: mapped to physical folder c:\BiztalkDev. Avoid using a network share to store the project files (to avoid the security warning that arises from developing a .NET application on a non-local drive).
Create a master solution that will hold all projects. As described in the previous section, the single partitioned model for BizTalk Server development is generally recommended for medium to high complexity BizTalk Server projects. For this reason a master solution should be created that will hold all the subprojects. This master solution will typically be used as the main build solution. For notes on creating a master solution for a partitioned single solution in Visual Studio Team Foundation Server, see "Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution."
Store all Visual Studio 2008 BizTalk Server projects in a folder under the master solution. If Team Foundation Server is being used as the source control environment then it is necessary to ensure that all subprojects that are created are created in a folder underneath the folder holding the master solution that contains them. For notes on adding subprojects to a master solution for a partitioned single solution in Team Foundation Server see "Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution." For more information see How To: Create Your Source Tree in Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129479) referenced in chapter 4 of the Team Development with Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129476) document referenced at the beginning of this section.
Divide the folder structure into shared and process-specific sections It is a common practice to have the shared entities separated from the business process specific entities. The shared entities are common to multiple projects and may include helper classes. For example, the first three folders in the following list are organized by shared entity and the last two are organized by business process:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\EmailHelper
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\SharedSchema
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountValidate\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\solutiondeploy.build
Any files the build script references (MSBuild .properties files and assemblies with custom tasks) are installed in a subfolder:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Build
The process of creating and including a key file for a single project is simple, but using the same key file for multiple projects requires some additional steps. After creating the key file (see How to: Create a Public/Private Key Pair (http://go.microsoft.com/fwlink/?LinkId=193188) for more detailed steps), other projects must add a link to the created key file to the project.
Figure 4 - Adding a link to an existing key file
After this step, the key file will be available in the project properties signing panel.
Figure 5 - Project signing key configuration
It is often helpful to build a project directory structure that can be replicated upon every workstation with a specific path to the key in the structure, for example:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\Keys
Test harnesses and stubs. It is generally recommended that test harnesses and test stubs are developed early in the project because they are useful resources and they also help develop a deeper understanding of the actual interactions between the integration layer and the other systems. If test harnesses are kept under the master solution then they can be included in the source control of a specific process or under a more general shared project. For example:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\Test
Cross-reference data. If the BizTalk Server cross-reference databases are being used as part of the mapping of messages in the business processes under development then it is often necessary to load the databases with the necessary data to allow lookup operations to successfully take place in the cross-referencing functoids. Cross-reference data can be input using the BizTalk Server Cross Reference Import Tool (http://go.microsoft.com/fwlink/?LinkId=129481) (btsxrefimport.exe) and XML "seed files." When the import tool is run it empties the cross-reference databases before importing new data from the seed file. This means that prior to the development of the "all processes" deployment script all seed data must be consolidated into one seed file. For example:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared \Shared\CrossRefData\MasterSeedData.XML
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution \Shared\CrossRefData\AccountRequestSeedData.XML
Storing BizTalk binding files. When developing BizTalk Server projects binding files are produced that define the binding of a logical process to the physical transports and hosts. This binding information is initially created by using the GUI tools but is then persisted in binding files to ensure that a process can easily be deployed in a repeatable, scripted manner. Complexity arises from the fact that the physical binding information for the development environment may be different from that in the test, pre-production, and final production environments. Additionally, the binding files contain references to the specific version numbers for the DLLs being deployed and consequently these files need to be managed to be kept in sync with any DLL version number changes. To help manage this complexity the binding files should always be located within a specific location for a given process and named following a naming convention. This makes it easier to perform modifications to the binding files (either manually or automatically). Typically binding files should be kept in a folder called "bind" underneath the project folder and named according to a naming standard. For example, the binding files for the "AccountRequest" process in the development environment could be kept in a file as in the following example:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution \AccountRequest\bind\bind_AccountRequest_Stage1_DevEnv.xml
Location for file dependency DLLs. When creating the folder structure it is helpful to create a common folder to hold DLLs that are referenced as file dependencies. By ensuring that all developers follow the same folder structure, file references will not become broken when solutions are moved between developers or workstations. For example:
C:\BizTalkDev\[AppName]\Main\ReferencedBin
Test messages. When developing schemas and using messages it is common to spend considerable effort testing many different sample messages against XML schema and orchestration processes. In a real-world solution, sample messages are a valuable asset because they contain data that is meaningful to the business solution. Random or dummy values in the same message will typically cause the process to fail. For this reason sample messages should be treated with as much care as the code of the solution. To assist in managing the message files, keep test messages under a specific folder called "msgs". In cases where there are both "generated instances" and "example messages" (for example, messages from an existing system) it is useful to keep these messages separate to allow comparison between the developed schema and actual data. For example:
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\msgs\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\msgs\generated\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\msgs\examples\
Example folder structure. The example folder structure described above typically looks like the following for a sample project:
C:\BizTalkDev\[AppName]\Main\Src\Binaries\
C:\BizTalkDev\[AppName]\Main\Src\ReferencedAssemblies\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\StrongNameKeys\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\CrossRefData\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\TestHarnesses\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\SharedSchema\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\EmailSendOrchestration\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\Shared\ErrorHandlingOrchestration\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\bind\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\scripts\
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\AccReqRecPipe
C:\BizTalkDev\[AppName]\Main\Src\_MasterSolution\AccountRequest\AccReqRecPipeCode
This document assumes that the team developing the BizTalk Server solution consists of multiple developers who will be using Visual Studio Team Foundation Server as their source control tool.
For step-by-step notes on creating a master solution for a partitioned single solution in Team Foundation Server Source Control, see Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution.
Most BizTalk File Types are XML based. A typical Source Control feature is the capability to merge files when changes have been made to files by multiple developers at once. By default Team Foundation Server considers BizTalk files to be text based and therefore valid for merging. While the merging engine of Visual Studio does a good job in merging XML files it can still result in invalid XML files that cannot be opened in the BizTalk Orchestration Designer, BizTalk Mapper or BizTalk Pipeline editor. In projects where you have inexperienced developers this can cause work being overwritten. There are three approaches you can consider in order of preference:
Both steps 2 and 3 are described below.
The following steps mark the BizTalk file types as disabled for merging:
Select Team Foundation Server, and then click Source Control File Types as shown in Figure 6.
Figure 7 - Adding Team Foundation Server File Types
The following steps disable multiple check-out for a BizTalk Team Project in Team Foundation Server:
Clear the option to Enable multiple check out as shown in Figure 8.
The recommended approach to using Visual Studio Team Foundation Server is to only check in code in the Main branch when it has successfully passed functional tests and the developer is confident that the code will successfully build without breaking any related code. Applying this model to BizTalk Server results in the following guidelines:
If this model is followed, then the Main Visual Studio Team Foundation Branch will always hold a build that can be successfully built and tested. This principle is important if the approach of "nightly builds" is to be adhered to. If you are working with multiple branches, then the reverse integration of code to the main branch should always be a thought through process that is executed by experienced people. Again the Test Harnesses and Quality gates determine the way forward.
Visual Studio Team System has the notion of check-in policies that can be used to determine whether the pre-requisites prior to check-in have been met. The product doesn't come with out-of-the-box check-in policies that support the described process, but it is certainly possible to develop one that does.
An alternative approach to check-in is that of checking in "intermediate" versions. Team Foundation Server has excellent features that support this. In this approach an intermediate version will not yet have successfully passed functional tests and can be thought of as "between builds."
It is required to distinguish between the intermediate versions and build versions. Using Visual Studio Team System Team Foundation Server this can be done in a variety of ways, either automatic or process based. For example:
Please refer to the topics in the remainder of this paper or alternatively refer to the "Source Control Guidelines" in the Team Development with Visual Studio Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=129476) document referenced at the beginning of this section.
BizTalk Server uses additional files that can beneficially be versioned and stored in Source Control. The following files are examples:
If these files are related to a specific Visual Studio BizTalk Server project then these files can be included within the BizTalk Server project and managed by using the Visual Studio integrated source control functions. To include a file or folder into an existing Visual Studio project, do the following:
Figure 9 - Add an existing item to your project
If the non-BizTalk Server project files are not part of a Visual Studio project, then you can manage them by using the Source Control Explorer.
Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution contains a step-by-step guide to creating a partitioned single solution structure using Visual Studio 20085 and Visual Studio Team Foundation Server that follows the guidance listed earlier. This step-by-step guide is intended as a working example and individual projects should modify the steps listed to meet their requirements.
When developing with the .NET Framework, versioning is governed by a standard set of rules that ensure when a version number changes, the impact of that change is typically minimal. Due to certain design and implementation choices made during the development of BizTalk Server, version numbers with BizTalk Server do not always follow the standard .NET Framework rules. The following sections describe these conditions.
When a BizTalk Server project is compiled, the resulting DLL is a .NET assembly and its behavior follows the standard .NET versioning behavior. If it occurs that multiple versions of the same assembly are installed with the same assembly version number, the system will produce unexpected results. For this reason it is important to ensure that BizTalk Server assembly versioning is planned and managed.
Each DLL containing a .NET assembly has two distinct ways of expressing version information. The following table shows the difference between the assembly version and the file version.
Table 10 - DLL Version numbers
The file version is a string that is displayed by Microsoft Windows® Explorer. This number can be used to manually determine which version of a file is installed. The assembly version number together with the assembly name and culture information is part of the assembly's identity. The assembly version number is used by the runtime to enforce version policy and plays a key part in the type resolution process at run time. This version number is physically represented as a four-part string with the following format:
<major version>.<minor version>.<build number>.<revision>
The following figure shows where these two version numbers can be viewed for a DLL.
Figure 10 - DLL Version Numbers
In .NET development it is typical to update the assembly version number to the current build number when a build takes place. However when developing a BizTalk Server solution, changing the assembly version number can break the relationship between an assembly and the dependent items that reference the DLL by its assembly version number. The following table lists items that refer to a BizTalk Server assembly by using its version number and the effect of changing an assembly version number.
Table 11 - Entities affected by Assembly Version Number Changes
Binding files
Changing the assembly version number will cause any existing binding files that reference the assembly to fail. This is because the binding file references the assembly by attributes including its version number. To reuse existing files they will need to be modified or regenerated. Binding files are XML and so modification can be undertaken using tools like Visual Studio 2008, Microsoft Office Infopath® or Notepad or alternatively can be scripted or automated using a tool.
For more information on reusing existing binding files using tools or an alternative to binding files, see Patching Binding Files.
Changing the assembly version number will cause any existing BAM tracking profile definition files to fail. The BAM tracking files are a binary file format so they cannot be edited and instead must be regenerated. If BAM tracking profiles are required it may be necessary to do either of the following:
During a project you have a choice between the following:
The following table compares these two possible approaches to updating the version numbers.
Table 12 - Comparing Approaches to Update Version Numbers
Assembly version number = Fixed number
File version number = Build number
Assembly version number = Build number
Creating a policy on when to increment a BizTalk Server assembly version number depends on several variables including build processes, testing processes, number of assemblies and binding files, and the type of build being delivered.
The following table compares shipping and non-shipping build types.
Table 13 - Comparing Build Types
Following this approach, assembly versions are kept fixed and file version numbers are updated with each build, as shown in the following table.
Table 14 - Build Types for Approach 1
Following this approach means that binding files do not need to change; however the following additional requirements should be noted:
Following this approach, assembly version numbers are incremented for every build, including non-shipping, as shown in the following table.
Table 15 - Build Types for Approach 2
This approach incurs the added effort of modifying the assembly build numbers and modifying the associated dependencies that use the build numbers (like binding files).
If the development team follows the approach of incrementing assembly version numbers for every build then there are several possible ways of automating the required changes in version numbers and binding files. These options are discussed in the section Automating Build and Deployment Version Numbering.
Which versioning approach to use? For the majority of developments Approach 1 (fixed assembly versions for non-shipping builds) is typically suitable. It has the advantage of requiring fewer resources to test and as long as informational version numbers are kept up to date, issues should be detectable and resolvable using the informational version number.
Versioning in BizTalk should be a very deliberate event. Especially, if artifacts are divided into multiple assemblies how and what artifact is being versioned can easily break the production environment, because of the dependencies BizTalk artifacts have on each other.
.NET assemblies can be invoked from within a map (using the scripting functoid, found under the advanced functoids palette) and this functionality provides a great deal of flexibility for delivering custom map functionality. However it is important to understand that the internal representation of the map files references not only the assembly type name but the full assembly version number.
This is significant because it means that if the version number of the assembly called by the map changes, then any links that reference the assembly will break. In a complex map this can cause considerable rework.
To avoid this issue, we recommend that if assemblies are required to be called from a map, you create a specific assembly to hold only map functionality and the assembly version number of this assembly is fixed. In this way, other helper functions can have the assembly version updated without breaking the maps.
If an assembly referenced from a map is changed after map development, then consider updating the map file in a text based editor to reflect the updated version numbers.
To achieve this use the following steps:
The version number of helper classes referenced by project references or file reference and used by orchestrations (for example, from within expressions) can be changed without encountering the issue described above.
After the completion of the information-gathering phase, described in the first section of this document, it is usual for the team to begin to produce design documents for the BizTalk Server orchestrations. These documents are typically then used to validate the proposed designs with system and process owners prior to commencing the development phase.
One of the primary benefits of the BizTalk Orchestration model is the great clarity you can achieve when a software implementation is pictorial. Regardless of how well a developer comments code, there will always be a need to maintain a separate set of artifacts including process diagrams, Visio® diagrams (with varying shape usage), documents, and whiteboard discussions that are used to convey what the code is actually doing. This is especially true when working with a business audience.
When working with a BizTalk Orchestration, the diagram is the implementation of a piece of functionality (at least at a specific level). This provides opportunities to use an orchestration diagram in several helpful ways within a project lifecycle, as follows:
One approach to developing BizTalk Server orchestration designs is to use Visual Studio to produce "skeleton" projects. The skeleton projects provide a way of validating the assumptions about the number and type of messages and the operations taking place upon the messages to complete a process.
These skeleton projects exhibit the following attributes:
As the designs are progressed, the skeleton projects can be used as the starting point for the BizTalk Server development phase. To add documentation to a group of related workflow shapes, use a Group shape. These display as much text as you care to associate with the shape. Many of the other orchestration shapes can hold descriptions and it can be useful to use this to express planned implementation detail.
To make best use of the BizTalk Server skeleton designs, it is helpful to use a documentation tool to export the skeleton design. For more information, see the Documenting BizTalk Server Systems section.
The BizTalk Software Factory referenced in the references section can help in generating a standardized solution and project structure. It can be customized or extended to adhere to specific company standards.
This section provides guidelines on naming projects, assemblies, orchestrations, messages, and other artifacts within BizTalk Server.
Prior to starting a BizTalk Server development, it is good practice to ensure that a standard is created for new XML target namespaces. The actual standard is often less important than the uniformity it confers across the project. An example target namespace naming policy might be:
http://<;
organizationname
>/[namespaces]/<
system
>/[<
operation
>]/[<
direction
version
]>
Where:
Many artifacts within a BizTalk solution have a standard .NET namespace associated with them. The guidance on .NET namespace naming should be adhered to with BizTalk artifacts, namely:
CompanyName.TechnologyName.Feature
Note that these are Pascal-cased and nested namespaces will have dependencies on types in the containing namespace.
For more information about naming guidelines, see Namespace Naming Guidelines (http://go.microsoft.com/fwlink/?LinkId=118653).
BizTalk project names and assembly names should often match the name of the associated namespace, such as the following name:
A division into assemblies such as the following will often be quite suitable for a BizTalk project:
BizTalk Server provides a means to logically deploy BizTalk artifacts in containers called Applications. Applications can be the means of deployment and group related BizTalk artifacts together. In terms of naming, often the names of the (sub)system or overall solution can be used.
When deployed, an orchestration "type name" is the name that is used and displayed by the BizTalk Server administration tools to refer to the orchestration. When creating a new orchestration, this property defaults to "orchestration_1". When creating a new orchestration, change this property to a descriptive name to help distinguish the orchestration.
All artifacts should be named with a Pascal convention unless mentioned otherwise for the specific artifact, although underscores are used to separate logical entities.
For schemas, maps, orchestrations, and pipelines, ensure that the .NET type name matches the file name (without file extension).
The following table shows an example messaging artifact naming convention. Note that the .NET type name of the artifacts listed above should match (without the file extension) the file name. The .NET namespace will likely match the assembly name.
Table 16 - Example Messaging Artifact Naming Convention
<PropSchema>_<Standard>.xsd
<SourceSchema>_To_
<DestinationSchema>.btm
Example:
Note The .NET type name of the artifacts listed above should match (without the file extension) the file name. The .NET namespace will likely match the assembly name. Adding the application name
Establishing and following naming conventions are good practices for designating variables, messages, multipart types, and so on, but they become even more important for the workflow shapes contained within an orchestration. The goal is to ensure that the intent of each shape is clear, and that the text associated with the shape conveys as much as possible given the space constraints. In this way, a non-technical audience will be able to use the orchestration as documentation.
Note To add documentation to a group of related workflow shapes, use a Group shape. These display as much text as you care to associate with them, and can add quite a bit of documentation value to the diagram. However, if you intend to instrument the application using BAM, be aware that a group shape does not exist at runtime and as such cannot be part of a BAM tracking profile. If this is the case, you could use a non-transactional scope shape instead.
The following table shows an example orchestration shape naming convention. Shape types are not used in this naming convention to save space (allowing more detailed functional descriptions). Use the tooltip in the administration console when requiring that information.
Table 17 - Example orchestration shape naming conventions
Including brief information about transaction type may be appropriate.
Typically, MessageName will be the same (though Pascal-cased) as the name of the message variable that is being received "into" (though the message variable will be camel-cased.)
Typically, MessageName will be the same (though Pascal-cased) as the name of the message variable that is being sent (though the message variable will be camel-cased.)
Expression shapes should be named with Pascal convention (no prefix) to simply describe the net effect of the expression, similar to naming a method.
Decide shapes should be a full description of what will be decided in the "if" branch.
If-branch shapes should be a (perhaps abbreviated) description of what is being decided.
Construct Message (Assign)
<ExpressionDescription> (for expression)
If a Construct shape contains a message assignment, it should be an abbreviated name of the message being assigned.
The actual message assignment shape contained should be named to describe the expression that is contained.
which contains expression:
Construct Message (Transform)
X_<SourceSchema>To<DestSchema> (for expression)
If a Construct shape contains a message transform, it should be an abbreviated description of the transform (that is, source schema to destination schema).
The actual message transform shape contained should generally be named the same as the containing shape, except with an "X_" prefix ("X_LoanRequestToCreditRequest").
which contains transform shape:
Call/Start Orchestration
The corresponding variable name for the exception type should (often) be the same name as the exception type, only camel-cased.
Parallel shapes should be named a description of what work will be done in parallel.
Delay shapes should be named an abbreviated description of what is being waited for.
Listen shapes should be named an abbreviated description that captures (to the degree possible) all the branches of the Listen shape.
Loop shapes should be named an abbreviated description of what the exit condition is.
Describe what action an administrator must take to resume the orchestration.
More detail can be passed to error property – and should include what should be done by the administrator before resuming the orchestration.
Describe why the orchestration terminated. More detail can be passed to error property.
The policy name may need to be abbreviated.
If the shape compensates nested transactions, names should be suffixed with the name of the nested transaction – otherwise it should simply be Compensate.
or
For documentation purposes developers are recommended to add descriptive text to the shapes description property. The BizTalk Server documenter tool on Codeplexs (see Documenting BizTalk Server Systems) can generate Microsoft Word documents or compiled help file that will contain these shape descriptions. These documents can be used as the basis of system documentation.
The following table shows an example orchestration type naming conventions.
Table 18 - Example Orchestration Type Naming
Multi-part types encapsulate multiple parts. The WSDL specification indicates "parts are a flexible mechanism for describing the logical abstract content of a message." The name of the multi-part type should correspond to the "logical" document type, that is, what the sum of the parts describes.
(which might encapsulate an invoice acknowledgment and a payment voucher)
Should be named simply for the schema (or simple type) associated with the part.
Port Types
Should be named to suggest the nature of an endpoint using Pascal casing. If the orchestration is exposed as a Web service, the port name is exposed in the generated WSDL. For this reason these are not suffixed with with “PortType” to avoid the “PortType” being visible to external interface. If there will be more than one Port for a Port Type, the Port Type should be named according to the abstract service supplied.
which might have operations such as
Should be named to suggest a grouping of functionality using Pascal casing and suffixed with "Port." Typically the port name should reflect the likely name of the physical port (created in the binding files). This will aid configuration
Should be named using Pascal casing, based on the logical name of what is being used to correlate.
Should be named using camel casing based on the corresponding correlation type. If there is more than one, it should be named to reflect its specific purpose within the orchestration.
When naming messages within orchestrations, you should use a standard naming convention to avoid the confusion that can arise when messages are traveling between multiple systems in both directions.
Consider naming messages from the point of view of the integration layer (that is, incoming means incoming to the integration layer, outgoing means leaving the integration layer). It may help to include underscores (_) in the names.
The following table shows an example message naming convention.
Table 19 - Example Message Naming Convention
If the messages being described are traveling between multiple possible systems, then a naming convention based upon system names may not be meaningful. An alternative approach is to name the messages after their schema, with a description of the message usage in the orchestration, for example:
The configuration of a deployed BizTalk Server system is stored within the configuration database of the BizTalk Server group. Tools have been written to query the Configuration database and produce formatted output that lists the configuration information to help you document BizTalk Server systems.
These tools can be useful for:
You can find a BizTalk Server reporting tool that generates a compiled help file detailing the BizTalk Server configuration. For more information, see the BizTalk Server 2006 Documenter (http://go.microsoft.com/fwlink/?LinkId=117291) project on the CodePlex web site.
This tool creates compiled help (*.chm) to quickly document the many artifacts (hosts, ports, orchestration diagrams, schemas, maps, pipelines, adapters, rule engine vocabularies and policies, and more) and their dependencies within a BizTalk Server environment.
By the nature of integration development, testing BizTalk Server solutions involves exchanging data with other systems. To functionally test BizTalk Server processes, you must obtain suitable test data and have access to the appropriate method of interacting with this data. In some cases you can interact with the actual systems that the integration layer requires to exchange data with during the development stage. In many cases, however, this will not be possible. In this scenario it is typical to use test harnesses and test stubs to allow development to proceed in the absence of the actual systems.
Test data is a very significant resource when developing an integration solution. Without valid test data you often cannot test a business process fully. When planning an integration development, you must ensure that the project includes sufficient resource & time to obtain valid data to allow testing. This is particularly true when the test data needs to be produced from systems that have never previously been integrated.
If existing data flows already exist it can be extremely beneficial to the design and testing of the integration solution if example messages are captured for all existing business operations.
For integration environments, test harnesses, test stubs and mocking can be defined as the following:
Example test harnesses include:
Example test stubs include:
Example mocks include:
It is generally recommended that test harnesses , test stubs and mocks are developed early in the project because they are a useful resource and they also help develop a deeper understanding of the actual interactions between the integration layer and the other systems.
For more information about example test stubs and harnesses, see the Additional Resources section.
By default BizTalk Server does not perform schema validation against incoming XML messages entering the BizTalk Server orchestration runtime. In this configuration BizTalk Server only examines the root node and target namespace of the incoming document. Comparing every incoming XML instance against its schema is a resource-intensive process and this default setting allows messages to enter BizTalk Server with the maximum throughput. Typically the default "no validation" setting is used for production environments.
During the development period, we recommend that runtime schema validation is switched on. By validating messages during development potential errors relating to badly formed messages are caught early in the development process rather than later, when they are typically more difficult to fix.
In some cases it may be applicable to switch on message validation in the production system (especially if messages may be coming from systems that may not have been through a rigorous integration testing process). If runtime validation is switched on, then be aware of the impact on throughput and capacity. Consider using the BizTalk Server performance counters to understand the impact.
To turn on runtime schema validation:
<?
xml
=
"1.0"
?>
<
configuration
>
xlangs
Configuration
Debugging
ValidateSchemas
"true"
/>
</
For more details on the runtime configuration, see Runtime Validation for the Orchestration Engine (http://go.microsoft.com/fwlink/?LinkId=129482) in the BizTalk Server documentation.
The preceding approach switches on validation of messages entering orchestrations. To enable validation of incoming messages in a "messaging only" solution requires turning on validation by setting the configuration options in the XML and FF disassemblers and by using the XML validator component in pipelines. For more information, see XML Disassembler Pipeline Component (http://go.microsoft.com/fwlink/?LinkId=201205) and Flat File Assembler Pipeline Component (http://go.microsoft.com/fwlink/?LinkId=201206) in the BizTalk Server documentation.
When putting in place the BizTalk Server team development and test environments, it is important to make certain the platform and environment is stable. The following sections detail techniques and approaches that will help deliver a stable environment and stable development process.
Your Team Foundation Server environment provides a central environment that contains all of the source code. Having a backup strategy for the Team Foundation Server is crucial in order to return to consistent state during disaster recovery. For more information about setting up a backup strategy, see the Administration Guide for Microsoft Visual Studio Team System 2008 Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=125172).
The recommended approach to using Source Control is to only check in code in the main branch when it has passed functional testing. This means that a considerable period of time can pass between check-in operations, during which time the code is not on the backed-up Source Control environment. To mitigate against the failure of a developer workstation it is important to ensure that a backup of the source code takes place, so make sure you are using the features described in the Checking In Intermediate Versions section and make use of the Shelving and Branching features of Team Foundation Server.
When developing orchestrations it is important to be able to understand the status of an orchestration both during and after execution. The BizTalk Management Console provides the capability to view the flow of an orchestration after it has completed through the Tracked Message Events and Tracked Service Instances queries. To place a breakpoint within an orchestration, see Debugging an Orchestration (http://go.microsoft.com/fwlink/?LinkId=185049) in the BizTalk Server documentation. In BizTalk Server 2009, the BizTalk compiler also creates a .cs file for each Orchestration in your solution. The .cs file can be found in the obj\Debug\BizTalk\XLang folder. You can open the .cs file in Visual Studio 2008 and leverage the Visual Studio debugger to attach to the btsntsvc.exe process and debug the code. Although Tracking is effective, it can also be useful to develop orchestrations that write out debugging information as they run.
One of the techniques most commonly used when developing debugging traditional code is to write out useful data during code execution to allow observation of the data and processes that are actually executing. The same functionality can be achieved in BizTalk Server orchestrations by using the following technique.
Within the .NET class library, the System.Diagnostics.Trace class allows the outputting of trace and debug information to listeners, which can capture and log the information. It is possible to write listeners using .NET code (see "Debug Class" in the .NET documentation), but it is also possible to use readily available utilities.
One such utility is DebugView for Windows v4.76 (http://go.microsoft.com/fwlink/?LinkId=129486).
When DebugView is running, it will catch and log the output of any debug statements, allowing the developer to view progress and actual data within the process. Note that debug statements also exist within other Microsoft applications (including BizTalk Server), so expect to see debug output from other applications too.
The following steps describe how to output a hard-coded string containing a process name and version number:
System.Diagnostics.Trace.WriteLine(
"Entering Orchestration)"
);
Activate the BizTalk Server process and observe the output in DebugView as the orchestration is started by the BizTalk Server runtime.
The trace.writeline technique described above can also be used to write out the contents of messages and variables at runtime. This can be useful when working with complex orchestrations, for example, multi-map transformations where it would normally not be possible to observe the intermediate messages produced by maps inside the orchestration.
The following procedure describes how to write out the contents of a populated orchestration message (named myMsg) to a debug listener:
myString declared
as
an orchestration variable of type
'string'
xmlDoc declared
'xmlDocument'
(type found under .Net classes, System.XML)
myMsg
is
the orchestration message whose contents are to be output
xmlDoc = myMsg;
//create XML doc from the BTS message
System.Diagnostics.Trace.WriteLine(xmlDoc.OuterXml);
// output the xml as a string
xmlDoc.LoadXml(myString);
myMsgFromString = xmlDoc;
To achieve this, the following requirements are necessary:
Viewing the map output is an important step when testing maps. The resulting XML produced by the map is saved to disk, even if the output is not valid according to the destination schema. Examining this output can be valuable in understanding why the output of the map is not valid.
Compare the resulting output XML against the schema definition to determine why the error is occurring. The map output document can be seen by looking in the temporary location for the output of a map, for example:
The output document is also available by CTRL-clicking the file name in the output window after performing the test map operation.
When developing maps, if map links or functoids don't seem to be producing the expected output or are producing errors, then consider examining the XSLT to work out what node data is being processed and where the results are being written. The XSLT is produced when validating a map and it represents the actual mapping that the BizTalk Server runtime will perform. By reading the generated XSLT it is sometimes possible to observe the reason that a map is producing unexpected output.
The XSLT is usually quite simple to follow, typically consisting of many simple XPATH queries. By reading the XPATH statements it is often possible to clearly see the input values and any operators followed by the nodes in which the output is written.
The XSLT is produced during a map validation and is located at:
The Visual Studio debugger supports debugging XSLT. The debugger supports setting breakpoints, viewing XSLT execution state, and so on. When working with mappings from Visual Studio 2008, the XSLT debugger can be started from the solution explorer by right clicking a mapping and select “Debug map”.
For more information on debugging XSLT in Visual Studio, see Debugging XSLT (http://go.microsoft.com/fwlink/?LinkId=201198).
BizTalk Server supports the creation of maps that use the scripting functoid. Users can create their own .NET methods and use them from within the scripting functoid to allow better reuse of custom code and functionality. Debugging these functions when testing a map can be very useful to determine if your custom map functions are working properly, but the steps to debug these custom assemblies are not obvious. Here are the steps you need to take in order to directly debug your scripting assembly while testing your map:
Error: "Map contains a reference to a schema node that is not valid"
This design-time error can occur when the map being opened uses a project dependency for one of its schema. If this error occurs ensure that the dependency has been recompiled to reflect any recent changes.
To understand why this error arises, consider the following scenario:
This is significant when a source control system is being used to store projects. By default Visual Studio 2008 does not store DLLs automatically and when retrieving a BizTalk Server project containing schema the project will need to be recompiled to produce the DLLs that dependent projects need.
Error: "Element cannot contain text or white spaces. Content model is empty"
This runtime error can occur when using Complex Content elements (like <ANY> or <SEQUENCE>). Complex content elements are not allowed to contain text unless their “Mixed” property is set to “True.”
You can debug adapters by using a similar approach to that outlined in the Debugging Custom .NET Methods within the Scripting Functoid on a Map section. Be aware that although adapters do not need to be in the global assembly cache for the runtime to find them, they do need to be in the global assembly cache for Visual Studio 2008 to find them when debugging. Make sure that adapter assemblies are in the global assembly cache before attaching to the BTSNTSvc.exe to debug them.
A common integration requirement is to a call a Web service from the orchestration, and then debug the Web service to determine the functionality taking place within the Web service when called from BizTalk Server. The following steps describe how to debug this scenario. The orchestration is shown in the following figure. It receives a message from the MessageBox, transforms it into the message expected by the Web service, and sends it to the Web service, and receives a response.
Figure 12 - Orchestration Calling a Web Service
To debug the Web service, perform the following steps:
Figure 13 - Process Dialog Box
Figure 14 - Attach to Process Dialog Box
The Web service is now ready to be debugged. Send the message to the orchestration and Developer Studio will stop at the breakpoint.
Pipeline assemblies are often used in BizTalk Server solutions to provide special handling of files before they are sent into or out of the BizTalk MessageBox. Debugging pipeline assemblies in Visual Studio 2008 is critical in order to quickly troubleshoot and fix problems.
After you build the pipeline assembly successfully, you will need to copy the files to the "Pipeline Assemblies" directory in the BizTalk Server installation directory (C:\Program Files\Microsoft BizTalk Server 2009\Pipeline Components) or register them in the Global Assembly Cache. If you are updating an assembly that has already been installed and you cannot copy over the existing assembly there are two possible causes:
After the file has been copied, you attach to the BizTalk process:
For information about the dialogs displayed when attaching to a process, see Debugging Adapters and Pipelines.
Note After changing a pipeline, the BizTalk Server runtime will not immediately pick up the change. The changes will not be visible until the runtime refreshes its configuration. This can be forced by enabling the Restart Host Instances option on the project properties as shown in Figure 15 or by restarting the hosts manually through BizTalk Server Administration or via a script.
Figure 15 - Restart Host Instances option
Note If the Redeploy option is set to True, this allows a modified pipeline to be redeployed without undeploying a previous version. However be aware that this redeploy action will reset any send or receive ports that used this pipeline to "PassThrough." After the redeploy, you need to reassign the port to the custom pipeline.
It is also possible to debug custom pipelines using Pipeline.exe, as described in Pipeline Tools (http://go.microsoft.com/fwlink/?LinkId=184286) in the BizTalk Server help. This tool, which can be found in the <InstallationFolder>\SDK\Utilities\PipelineTools directory, allows the developer to debug the pipeline without deploying it to an actual BizTalk Server. To use pipeline.exe, perform the following steps:
It can be useful to include helper classes within a BizTalk Server project to perform business logic tasks or functions that aid the development process. A typical task of a developer helper class would be to write messages out to the file system to aid debugging and diagnostics.
Included in the samples coming with this whitepaper is a template for a generic BizTalk Server helper class that manipulates messages. It provides limited functionality to aid the debugging of messages including a "DumpMessageToFile" function that writes a BizTalk Server message to the file system.
This sample can be modified to handle more complex functionality. To use the sample within a BizTalk Server project complete the following steps:
GACUTIL /I GeneralHelper.dll
DebugHelper.Utils.DumpMessageToFile(<MessageName>,
"<path>"
,
"<filename>"
,<timestamp>);
<MessageName>: String, Name of the BizTalk Server message in the orchestration to be written out <path>: String, Path for file to be written to, with backslashes denoted with "\\" <filename>: String, Filename for the resulting file <timestamp>: Boolean, prepends a timestamp to the filename to create quasi-unique filenaming for messages
Another example:
DebugHelper.Utils.DumpMessageToFile(AccountUpdateIncoming,
"c:\\tmp"
"AccountUpdateIncoming.xml"
true
Note It is a best practice to always mark BizTalk Helper classes with the “Serializable” keyword, so that the state of the class can be serialized when a BizTalk orchestration is being dehydrated, it is properly stored in the MessageBox as well.
The following tools and documents are useful to have available within the BizTalk Server developer's environment:
This tool adds a BizTalk Server search pane to the standard Windows Explorer, which can be accessed from the folders bar or using the View -> Explorer Bar -> BizTalk Server Search.
This tool also allows you to search across the deployed assemblies for any of the BizTalk artifacts types and to view the detailed information about the artifacts found.
The following figure shows the location of the tool.
Figure 16 - BizTalk Server Windows Explorer Extension
This section provides information, toolkits, templates, and tools useful to teams looking to develop efficient build and deployment phases.
When a BizTalk Server developer is developing a solution, there is a common requirement to perform the following steps: build, deploy, test, and undeploy. The Visual Studio 2008 BizTalk Server Tools assist in carrying out these tasks. On the project-level there are two settings:
With these settings set to True as shown in Figure 17, developers can just right-click on a project and select Deploy to redeploy the already deployed project incorporating changes made.
Note Be careful with projects that depend on other projects. If you redeploy the project that is referenced by another project, the last project won’t always be automatically redeployed.
Note Unfortunately this doesn't provide a developer experience on the local sandbox that is consistent with the way BizTalk solutions are deployed on other environments like integration, test, and eventually production environments. While this feature can be beneficial in the early stages of a project, we recommend you work on consistent deployment tooling based on MSBuild or Powershell.
Figure 17 - Redeployment Settings
BizTalk Server provides command-line and WMI interfaces to the BizTalk Server deployment functionality. This allows the creation of command and script files. These files can enable a developer to redeploy simple or complex solutions with a single operation.
These scripts are also beneficial when working with orchestrations that have dependencies upon other orchestrations. BizTalk Server enforces the dependencies when starting/stopping and deploying/undeploying orchestrations, and a script is an efficient way to ensure that the dependency order is always followed.
To aid the creation of simple redeployment scripts, the code that comes with this whitepaper contains MSBuild scripts that are using the MSBuild SDC Tasks on Codeplex (http://sdctasks.codeplex.com/) and the MSBuild Community Tasks (http://msbuildtasks.tigris.org/).
To aid the reusability of the above scripts, a team may wish to parameterize the deploy and undeployment scripts to accept paths for the DLL and binding file folder locations. In this way, the same scripts can be reused from a deployment package that contains the final compiled DLLs and production binding files in alternative locations.
The scripts rely on the SDCTasks MSBuild task assembly being available in the accompanying sample. These tasks are used to control and deploy the BizTalk artifacts. The sample scripts are provided as samples to demonstrate the steps required. You can use these scripts as a basis for your own solution(s). It is recommended that the same scripts are used as part of the MSIs that are used to deploy the solution on other (non development) environments by creating an MSI and a custom deployment task that calls the MSBuild scripts.
When developing BizTalk Server solutions, there is no way to test a modified orchestration without actually deploying and binding the resulting assembly. This means that changing just one line of code requires you to stop, unbind, rebuild, redeploy, rebind, and restart an assembly. The redeployment scripts listed above provide an automated "one-step" operation to achieve the necessary functionality; however, you can use an alternative approach in some circumstances.
Note The above approach is not officially supported, and should be restricted to local development workstations.
Steps to set up quick redeploy:
DEVPATH=
"C:\BizTalkDev\[AppName]\Main\AccountUpdate\bin\development\"
From this point on the assembly is loaded from the DEVPATH location, not the global assembly cache. To make a simple change to the assembly, simply recompile the updated solution and restart the BizTalk Server service or the BizTalk Server host hosting the orchestration. For more information about the DEVPATH environment variable and machine.config settings, see Locating Assemblies Using DEVPATH (http://go.microsoft.com/fwlink/?LinkId=43667). Note This procedure doesn't work all the time. Once the boundaries of the solution are touched, like incoming / outgoing message schemas, port names, pipelines, maps, etc, most of the time a redeployment is necessary. It can be hard to determine why a solution isn’t working if one only updates the assembly and the information in the BizTalk Configuration is out of sync. While this feature can be beneficial in the early stages of a project, we recommend you work on consistent deployment tooling based on MSBuild or Powershell.
You can also use the following approach when making internal changes to an orchestration that does not require port binding changes. As in the quick redeploy option above, this approach is purely a development shortcut and is not officially supported.
When developing BizTalk Server on a local workstation, it is common to make a change and then redeploy the updated assembly to retest the functionality. Because the BizTalk Server runtime loads assemblies into memory periodically, BizTalk Server does not load up the new version until it reloads the assemblies, unless the updated version has an updated assembly version. To ensure that the BizTalk Server runtime picks up the updated version immediately, stop and restart the BizTalk Server runtime hosts.
Restart the hosts manually through BizTalk Server Administration or via a script. Alternatively, you can start and stop the BizTalk Server runtime by creating a command file that contains the following two lines of code:
net stop
"BizTalk Service BizTalk Group : BizTalkServerApplication"
net start
Note If you have multiple hosts (with different names), you have to add and update the command line. Recycling hosts is quicker than starting and stopping the BizTalk Server service and is recommended, unless you have modifications such as BizTalk Server runtime configuration changes that need to be picked up quickly.
Alternatively, you can restart the Enterprise Single Sign-On service. The BizTalk Server Host service has dependencies on this service, so a restart of the SSO Service will cause the host services to restart as well.
As the Planning Team Development section of this document discussed, a development team may wish to automate the updating of project version numbers and binding file version numbers. This section provides some tools to assist in these processes.
When a development team is producing frequent build and deployment operations of multiple BizTalk projects with the version information on the DLLs changing frequently, it can be helpful to automate the modification of the version numbers or physical addresses in binding files.
The SDC Tasks provide a MSBuild task that can replace a single element or attribute in any XML file by using an XPath expression to identify the element or attribute to replace. This can be very useful as part of your build process to update bindingfiles, version attributes, BizTalk project files, etc., to incorporate values specific to the environments that you are targeting. The following example provides a sample how to use the MSBuild task:
Project
Target
Name
"Test"
Xml.ModifyFile
Path
"C:\SourceFiles\file1.xml"
XPath
"/TestNodes/TestNode"
AttributeName
"NodeID"
NewValue
"MODIFIED"
ShowMatches
"Yes"
</Xml.ModifyFile>
For more information about XPath Expressions, see XPath Examples (http://go.microsoft.com/fwlink/?LinkId=94638).
An automated build process is designed to enable a team with a large number of team members, or a large number of code assemblies under development, to efficiently perform a unified build process. The unified build process produces a set of assemblies that can be used for testing the whole integration solution (as opposed to the isolated testing of individual processes on developers' workstations).
An automated build process is heavily dependent upon flexible access to developer's source code. Typically an automated build process integrates with a source management system like Visual Studio Team Foundation Server Source Control and a build environment based on a Visual Studio Team Foundation Server Team Build Server.
The automated build example described here is a simplified version of a typical build process used in a customer project. It is provided as an example from which a team could develop a fully automated nightly build process with related individual build processes for developers.
The process is described here as an example of the type of steps typically required, because an individual team's requirements will vary. All scripts mentioned in this section are included in the "BtsAutomatedBuildSample" folder of the accompanying samples.
The example assumes that a workstation or server is available as a Team Foundation Server build server and that the file assembly version number is incremented with each build. It is assumed that Assembly Versions are updated manually based on stringent versioning strategies.
The following list gives you additional information and hints about the automated build process:
There are two versions of the build process:
The main difference between the two processes is that the developer process leaves no trace on Source Control and actually (re)deploys the solution on the local developer’s sandbox, whereas the actual build process increments the build number and other numbers and possibly (re)deploys the solution on the build server or a different test environment.
The command file that drives the build process in TeamBuild is called TFSBuild.proj, which defines the Build Type created in Team Explorer. The scripts can be found in the “BtsAutomatedBuildSample\BtsAutomatedBuildSample\TeamBuildTypes\BizTalk Build” folder.
The following steps outline the process:
TeamBuild provides an optional parameter called IsDesktopBuild to indicate it should run a local (developer) build.
The following table describes the automated build files.
Table 20 - Description of Automated Build Files
This section covers how a team developing with BizTalk Server can achieve an efficient process to collectively automate the deployment of the resulting solution. It focuses on automating the deploying process to allow nightly builds and test processes to be run on a set of test servers. It does not focus on the topic of deployment to a production environment, although some of the techniques may be applicable to this task.
Deploying a "real-world" BizTalk Server solution typically consists of several related tasks, including:
Typically a deployment also performs operational tasks like enlisting and starting processes (in the right order) and clearing down logs and tracking databases.
The following steps provide a sample structure for creating a partitioned single solution that integrates with Visual Studio Team Foundation Server and holds the non-BizTalk Server project entities that are required (for example, test data or scripts).
This example assumes that you are following the partitioned single solution model for larger or more complex solutions. You can create the single solution model by following the same steps and excluding the "Create a partitioned solution for customer operations only" section.
Use the following steps as a sample and not as a definitive guide. They are presented here to assist developers new to team development with BizTalk Server and Visual Studio Team Foundation Server with a "quick start" approach. This approach allows the team to familiarize themselves with the partitioned single solution approach and evaluate which Source Control approach will best suit the team needs.
This section provides the steps and information necessary to create an example Visual Studio 2008 solution structure that integrates with Visual SourceSafe.
Figure 18 - Team Foundation Server New Team Project option.
Figure 19 - Team Explorer Source Control option
Create a new local source control workspace. In Source Control Explorer, select the Workspace option (see Figure 20). And create a new workspace mapped to the folder you just created c:\BizTalkDev (see Figure 21).
Figure 22 - Creating subfolder in Source Control Explorer
Figure 23 - Check In Pending Changes
c:\BizTalkDev\[AppName]\Main\Src\_MasterSolution
Repeat this process for other shared projects.
Figure 24 shows the Add New Project dialog box.
Figure 25 - Solution Explorer showing Include In Project
Figure 26 - Retrieving specific solutions
The following resources may be of use to BizTalk Server designers, developers, and project managers:
It is our hope that through this guide, solution designers have gained an understanding of the constituent parts of a BizTalk solution and the possible project structures used to develop BizTalk solutions.
We also trust that developers have gained a better understanding of the process of developing a BizTalk solution in a team by using Team Foundation Server. Project managers will also have gained an overview of the main project phases and gained a better understanding of the typical tasks necessary when planning a BizTalk Server team development.
This guide was created by the following team members:
This whitepaper was based on Developing Integration Solutions with BizTalk Server 2004 by Angus Foreman and Andy Nash. The BizTalk Server 2006 R2 version of this paper is available at Developing Integration Solutions using BizTalk Server 2006 and Team Foundation Server (http://go.microsoft.com/fwlink/?LinkId=201209).
Many thanks to the following contributors and reviewers:
Another important place to find BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.