Biztalk Cross Reference

Reference Data:

Reference data means, it is an object within a system that is restricted to specific list of values. In integration scenarios when you are passing a message from one system to another these lists of reference data are different between the systems. (Where a particular field of the source maps to another field in destination. But their values aren’t so agreeable)

If we use an EAI (Enterprise Application Integration) pattern, you can have one input and more than one output flow. So, in this case, you must translate the property Value or Id in a Reference Value or Id.

In this case consider the example diagram:

Example Scenario:

We have a field called status in Application1 which field named mapped to status in Application2 and status in Application3. But the possible list of values are different.

Field	Application 1	Application 2	Application 3
Status	Open	Entered	Started
Status	Closed	Done	Finished

How to solve this issue?

There are many possible ways.
Write an inline script / xslt in a Scripting Functoid

1.        Use an External dll that takes an input and returns an equivalent output
2.        Use of BizTalk Server Cross Reference Functoids.
3.        Custom Functoids and Lots of more options….

Explained here for pros and cons of each solution.

This article provide solution by using Cross Reference Functiods. The cross referencing database functoids use data from tables stored in the BizTalkMgmtDb SQL Server database. The functoids available and XML files that represents the reference data listed below.

Functiods available in Biz talk:

Get Application ID	Retrieves an identifier for an application object.
Get Application Value	Retrieves an application value.
Get Common ID	Retrieves an identifier for a common object.
Get Common Value	Retrieves a common value.

All these functiods are listed under Database functiods group.

XML files that represents the cross reference data:

SetUp-Files	This will have the path to all other files
listOfAppType	This will have the list of applications. In our case it is “Application1″ ,“Application2″ and “Application3”
listOfAppInstance	For each Application created, an instance should be created
listOfIDXRef	This XML file will hold the IDs for which we create Data. In our case it is the “Status”
listOfIDXRefData	This file will hold the data for Common ID, “Application1″ data “Application2″ Data, and “Application3” data
listOfValueXRef	This XML file will hold the IDs for which we create Data. In our case it is the “type”
listOfValueXRefData	This file will hold the data for Common value, “Application1″ data “Application2″ Data, and “Application3” data
listOfMessageDef	This XML file hold the information related to message codes
listOfMessageText	This XML file hold the information related to message binding to the message code

You can find in MSDN for more information about document structure and how to import data for Cross referencing functoids.

Note: we can have any name for file but the format of data should be as per documentation.

Solution Approach:

1.        Create a XML files that represents the cross reference data.
2.        These XML files (data) are imported to a set of Tables inside BizTalk Management Database (BizTalkMgmtDb) using BizTalk Server Cross Reference Import tool (btsxrefimport.exe)

Cross Reference Types:

There are 2 types of Cross Reference – 

1.        ID Cross Reference
2.    Value Cross Reference

Deserializing CSV file to Collection

Every one know that CSV file stores the tabular data in plain text format.A CSV file consists of any number of records, separated by line breaks of some kind; each record consists of fields, separated by some other character or string, most commonly a literal comma or tab. Usually, all records have an identical sequence of fields. I have a requirement like I want to read the CSV file and Convert to Collection of objects. Steps to convert the CSV file to collection as follows.

1. Read all the lines from the CSV file
2. Take the first line as the header and remaing lines as records
3. Convert the data to data table by parsing first line to data columns and remaining lines to data rows
4. Convert data table to collection

Sample code to implement step 1 to 3

 public static DataTable ToDataTable(string csvFileNameIncludingPath)
        {
            var dt = new DataTable();

            var s = File.ReadAllLines(csvFileNameIncludingPath);
            string[] tableData = s.Split("\r\n".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
            var col = from cl in tableData[0].Split(",".ToCharArray())
                      select new DataColumn(cl.TrimStart('"').TrimEnd('"')); // In my case all the field values are included in the double quotes ex: "FirstName". So used TrimStart and TrimEnd methods to remove double quotes
            dt.Columns.AddRange(col.ToArray());

            (from st in tableData.Skip(1)
             select dt.Rows.Add(st.Split(",".ToCharArray()).Select(str => str.TrimStart('"').TrimEnd('"')).ToArray())).ToList(); // In my case all the field values are included in the double quotes ex: "jhon". So used TrimStart and TrimEnd methods to remove double quotes

            return dt;
        }

Sample code to implement step 4

 public static List<T> ToCollection<T>(this DataTable dt, IList<KeyValuePair<string, string>> changedNames = null) // changeNames parameter used for mapping the column name change with object propertyname
        {
            List<T> lst = new List<T>();
            Type tClass = typeof(T);
            PropertyInfo[] pClass = tClass.GetProperties();
            List<DataColumn> dc = dt.Columns.Cast<DataColumn>().ToList();
            T cn;
            foreach (DataRow item in dt.Rows)
            {
                cn = (T)Activator.CreateInstance(tClass);
                foreach (PropertyInfo pc in pClass)
                {
                    // Can comment try catch block. 
                    try
                    {
                        DataColumn d = dc.Find(c => c.ColumnName == pc.Name || (changedNames != null && changedNames.Any(x => x.Key == c.ColumnName)));
                        var ds = dc.Where(x => x.ColumnName == pc.Name);
                        if (d != null)
                            pc.SetValue(cn, item[pc.Name], null);
                    }
                    catch
                    {
                    }
                }
                lst.Add(cn);
            }
            return lst;
        }

I hope this will helps you.

DataSet Deserialization from Webservice

I observed that few of the webservices or REST services returns the DataSet with the xml read mode as DiffGram. But our requirement is to deserialize into collection of object of sepecified type. The deserialization of dataset is different from the normal xml or json deserialization. Please find the below steps to deserialize dataset.

Step1: Create the response to DataSet object as follows.

private DataSet GetDataSet(string url)
        {
            DataSet ds = new DataSet();
            using (var client = new WebClient())
            {
                var xml = client.DownloadString(sURL);
                using (var reader = XmlReader.Create(new StringReader(xml)))
                {
                    ds.ReadXml(reader, XmlReadMode.DiffGram);
                }
            }
            return ds;
        }

Step 2: Use the following helper method to Convert Data Table to Collection

 public static List<T> ToCollection<T>(this DataTable dt)
        {
            List<T> lst = new System.Collections.Generic.List<T>();
            Type tClass = typeof(T);
            PropertyInfo[] pClass = tClass.GetProperties();
            List<DataColumn> dc = dt.Columns.Cast<DataColumn>().ToList();
            T cn;
            foreach (DataRow item in dt.Rows)
            {
                cn = (T)Activator.CreateInstance(tClass);
                foreach (PropertyInfo pc in pClass)
                {
                    // Can comment try catch block. 
                    try
                    {
                        DataColumn d = dc.Find(c => c.ColumnName == pc.Name);
                        if (d != null)
                            pc.SetValue(cn, item[pc.Name], null);
                    }
                    catch
                    {
                    }
                }
                lst.Add(cn);
            }
            return lst;
        }

Step 3: Use the methods as follows to Convert DataTable to collection

var ds= GetDataSet(url);
ds.Tables[DataTables.LotTable].ToCollection<Lot>();

I hope this helps you..

Cannot compare elements of type 'System.Collections.Generic.List

We may get an exception when we are working with lamda expression as follows.

Cannot compare elements of type 'System.Collections.Generic.List`1'. Only primitive types, enumeration types and entity types are supported.

The solution to resolve this issue is, make sure don't have any null condition checks in the where clause.

Get Bin Directory Path using C# for web application as well as class library

We may get the requirement while coding in C# like we need to get the bin path  of class library or Web application.

the following is the sample code which works for class library as well as web application.

string binPath = Path.GetDirectoryName((new System.Uri(Assembly.GetExecutingAssembly().CodeBase)).LocalPath)

Application pool always getting stopped

I have a problem like SecurityTokenServiceApplicationPool always getting stopped when I am opening Sharepoint manager 2013.
After research for this problem on google and gone through the EventViewer and found that this is the problem with the user credentials for the applicationpool running under the specified user. Password for my user got expired.

Solution: Make sure password for the user is correct and it is not expired. If it is expired please update with new password.

Xml Serialization Behavior

Xml Serializer behaves differently based on the datatype and the behaviour of the property(Nullable/Not Nullable). Some time we may get confusion on why the xml serializer throwing an exception. Find the below table contains when the xml serializer throws an exception or when the object is instantiated.

Data Type	Property Behavior	Tag removed	i:nil="true"	Empty Tag	Invalid Content	Comments
Int, decimal	Nullable	Null	Null	Exception	Exception
	Not Nullable	0	Exception	Exception	Exception
DateTime	Nullable	Null	Null	Exception	Exception	DateTime should be the following format, otherwise throws an exception 1) yyyy-MM-ddTHH:mm:ss 2)yyyy-MM-dd 3)HH:mm:ss
	Not Nullable	0001-01-01T00:00:00	Exception	Exception	Exception
Collections	-NA-	Initialized new object with collection count as zero	Initialized new object with collection count as zero	Initialized new object with collection count as zero	Initialized new object with collection count as zero	Decorate property with [XmlElement(IsNullable=true)] attribute to set the property as null instead of initializing new object
Csharp Object	-NA-	Null	Initialized to New Object	Initialized to New Object	Initialized to New Object	Decorate property with [XmlElement(IsNullable=true)] attribute to set the property as null instead of initializing new object
Enum	Nullable	Null	Null	Exception	Exception
	Not Nullable	Initialized with firt enum element	Exception	Exception	Exception