blog.mha.dk
The on-line blog of Michael Holm Andersen

LINQ to SQL – Group 2 rows into 1

Thursday, 12 August 2010 20:55 by mha

Jacob and I have been struggling with some “nasty” LINQ today where we need to get the aggregated value of a Quantity column from a transaction table, however a little “twist” was that we need to sum all the positive values into one group and all the negative values into another, and in the result set end up with only ONE row containing a PositiveQuantitySum and a NegativeQuantitySum … at first it sound pretty easy, but turned out not to be the case.

After a few attempts we ended up with this LINQ which actually gives the correct result, but we have 2 rows (one “positive”, one “negative”) – here is the LINQ:

from test in
(from t in AxInventTransactions
where t.Quantity >= 0 &&
              t.DateFinancial >= DateTime.Parse("01-04-2010") &&
              t.DateFinancial <= DateTime.Parse("31-08-2010") &&
              t.Customer.StatisticsGroupName.Contains("S")
              orderby t.ItemNumber, t.CustomerAccountNumber
              group t by new { t.ItemNumber, t.CustomerAccountNumber, t.VendorAccountNumber, t.Customer.StatisticsGroupName, CustomerName = t.Customer.Name, ProductName = t.Product.Name }
              into myGroup select new { myGroup.Key.ItemNumber,
              myGroup.Key.CustomerAccountNumber, 
              myGroup.Key.VendorAccountNumber, 
              myGroup.Key.StatisticsGroupName, 
              myGroup.Key.ProductName, 
              myGroup.Key.CustomerName, 
             Quantity = myGroup.Sum(i => i.Quantity)
})
.Union(from t in AxInventTransactions
where t.Quantity < 0 &&
              t.DateFinancial >= DateTime.Parse("01-04-2010") &&
              t.DateFinancial <= DateTime.Parse("31-08-2010") &&
              t.Customer.StatisticsGroupName.Contains("S")
              orderby t.ItemNumber, t.CustomerAccountNumber
              group t by new { t.ItemNumber,
                               t.CustomerAccountNumber,
                               t.VendorAccountNumber,
                               t.Customer.StatisticsGroupName,
                               CustomerName = t.Customer.Name,
                               ProductName = t.Product.Name} 
              into myGroup select new { myGroup.Key.ItemNumber,  
                             myGroup.Key.CustomerAccountNumber, 
                             myGroup.Key.VendorAccountNumber, 
                             myGroup.Key.StatisticsGroupName, 
                             myGroup.Key.ProductName, 
                             myGroup.Key.CustomerName, 
Quantity = myGroup.Sum(i => i.Quantity) } ).OrderBy(x => x.ItemNumber).ThenBy(x => x.CustomerAccountNumber)
orderby test.ItemNumber, test.CustomerAccountNumber
select new
{
            ItemNumber = test.ItemNumber,
            CustomerAccountNumber = test.CustomerAccountNumber,
            VendorAccountNumber = test.VendorAccountNumber,
            QuantityPlus = test.Quantity >= 0 ? test.Quantity : 0,
            QuantityNeg = test.Quantity < 0 ? test.Quantity : 0,
            StatisticsGroupName = test.StatisticsGroupName,
            ProductName = test.ProductName,
            CustomerName = test.CustomerName,
}

The above LINQ which uses UNION to merge the two result set (positive / negative)produces a result set something like this:

result1

We would like to have this result as ONE row, containing a column for the positive sum (6) and a column for the negative sum (-3).

While eating dinner tonight (I now, geeky!), I realized that we had to skip UNION and do it all “in one go”. The problem however is that (as you probably already know): Anonymous types can't be null

… and grouping all the positive and negative values of Quantity “in one go” would result in a NULL value for each “alternate” value (or whatever we should call it), causing an InvalidOperationException in this case: “The null value cannot be assigned to a member with type System.Int32 which is a non-nullable value type.”

So .. we need to make sure that IF the value returned by Sum() is actually NULL (which does not play together with System.Int32 :-) we instead return 0 (as it’s harmless to either add/subtract zero, this will of course not affect the result).

I ended up with this LINQ:

from t in AxInventTransactions
where     t.DateFinancial >= DateTime.Parse("01-04-2010") && 
          t.DateFinancial <= DateTime.Parse("31-08-2010") && 
          t.Customer.StatisticsGroupName.Contains("S") 
          orderby t.ItemNumber, t.CustomerAccountNumber 
          group t by new {
t.ItemNumber,
                           t.CustomerAccountNumber, 
                           t.VendorAccountNumber,
                           t.Customer.StatisticsGroupName,
                           CustomerName = t.Customer.Name, 
                           ProductName = t.Product.Name } 
          into myGroup select new { myGroup.Key.ItemNumber,
                         myGroup.Key.CustomerAccountNumber, 
                         myGroup.Key.VendorAccountNumber, 
                         myGroup.Key.StatisticsGroupName, 
                         myGroup.Key.ProductName, 
                         myGroup.Key.CustomerName, 
QuantityPlus = myGroup.Where(x => x.Quantity >= 0).Sum(i => i.Quantity) != null ? myGroup.Where(x => x.Quantity >= 0).Sum(i => i.Quantity) : 0, 
QuantityNeg = myGroup.Where(x => x.Quantity < 0).Sum(i => i.Quantity) != null ? myGroup.Where(x => x.Quantity < 0).Sum(i => i.Quantity) : 0 }

Which produces a result set like this:

result2

As you can see we now have ONE row with a column for “all positive values aggregated” and column for “all negative values aggregated”, which is exactly what I want.

The ‘trick’ is to make sure you don’t get any NULL values into the quotation, which is done using the ternary operator (very useful in LINQ also!), like so:

QuantityPlus = myGroup.Where(x => x.Quantity >= 0).Sum(i => i.Quantity) != null ? myGroup.Where(x => x.Quantity >= 0).Sum(i => i.Quantity) : 0

Which “translates” to something like this:

If Quantity is equal or larger than 0, add the value of Quantity to the aggregated “QuantityPlus” (System.Int32), however if the Quantity value does NOT meet the criteria (aka is a negative value) then NULL would be added (which of course will cause an exception!), SO INSTEAD we need to add the value 0 to the aggregated “QuantityPlus” (using the ternary operator takes care of this).

Hope the above makes sense and helps anyone which is facing similar problem finding the solution a bit easier :-) .. if anyone knows an alternate way of solving this issue, please feel free to write me an email.

AutoComplete, PageMethod and LINQtoSQL

Wednesday, 12 May 2010 15:49 by mha

I’m using AutoComplete to enhance the functionality of a TextBox and make it act like the below image, where the user enters the customer accountnumber, which is then displayed together with their name (the blurry stuff in the picture :-)

autocompletesearch

As I have to use this functionality on many pages, I needed it to be a UserControl.

I wanted to use a PageMethod (instead of a Web Service) to populate the AutoCompleteExtender, but since the page method must reside on the page (in the .aspx.cs file, it does not work if it’s placed in the .ascx.cs file – dammit MS), I wanted to limit the code needed on the .aspx.cs page to a bare minimum.

The number of entries displayed in the AutoComplete DropDown list had to be limited, but the user need to know how many matches the prefix entered.


The Page Method (ServiceMethod) of the AutoCompleteExtender requires a string[] to work and I wanted the Page Method to contain as little code as possible. I ended up with this:

[System.Web.Services.WebMethodAttribute(), 
System.Web.Script.Services.ScriptMethodAttribute()] public static string[] GetDataCustomerAccountNo(string prefixText,
int count, string contextKey) { CustomerRepository customerRepository = new CustomerRepository(); return customerRepository.GetCustomerAccountNumbersByPrefix(prefixText); }


The GetCustomerAccountNumbersByPrefix looks like this:

/// <summary>
/// Retrieves all AccountNumbers which match prefix (AutoComplete)
/// </summary>
/// <param name="prefix">Prefix to match</param>
public string[] GetCustomerAccountNumbersByPrefix(string prefix)
{
    var customers = from e in db.Customers
            where e.AccountNumber.StartsWith(prefix)
            orderby e.AccountNumber
            select e;

    int count = customers.Count();
    string[] result = customers.Take(21).Select(x => x.AccountNumber + " – " 
                                         + x.NameAlias).ToArray<string>();

    if (result.Count() > 20)
    {
       result[20] = ">> antal: " + count.ToString();
    }

    return result;
}


If the ResultSet contains more than 20 rows I want to provide information about this to the user, displaying the total number of rows, which is why I do a Count() before Take()

I use .Select() to transform the properties AccountNumber and NameAlias into a single string, which is then returned as a string[] using ToArray(), in the process I limit the number of rows to a maximum of 21 using Take()

And all there’s left to do is to optionally change the 21th entry to show the total number of rows information. As the string[] is fixed it’s much easier simply to take out an extra row - Take(21) – and use this last row to show the information.

Hope the above example is useful to others who might want to use the AutoCompleteExtender in a UserControls only to find out that the PageMethod actually DO have to reside in the .aspx.cs file.

Categories:   C# | ASP.NET | LINQ
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

LINQtoSQL – Grouping and selecting

Wednesday, 21 April 2010 15:00 by mha

I have a Order->OrderLine table setup where I need to check if any “duplicate” orders exists. In order to check this I need to be able to select select various columns *not* contained in the group (key) – playing around in LinqPad I quickly came up with this query:

from line in AxSaleLines
where line.QuantityRemaining > 0
group line by new {line.AxSale.AccountNumber, line.ItemNumber, 
ProdName = line.Product.Name, ProdFormat = line.Product.ProductFormat.Name}
into myGroup
where myGroup.Count() > 1
from s in AxSaleLines
where s.ItemNumber == myGroup.Key.ItemNumber && 
      s.AxSale.AccountNumber == myGroup.Key.AccountNumber
select new { s.AxSale.SalesId, s.AxSale.AccountNumber, 
             CustomerName = s.AxSale.Customer.Name, s.ItemNumber, 
             ProductName = s.Product.Name, s.Product.ProductFormatName, 
             s.Quantity, s.AxSale.SalesOrigin}
 


When bound to a grid, I can then show this to the user:

groupedgrid

Categories:   LINQ | ASP.NET
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

LINQ and PagedList

Tuesday, 26 January 2010 18:04 by mha

If you’re using LINQ you probably know about the .Skip and .Take methods. However often you need to do paging, hence you probably use something similar to this to get around that:

public PaginatedList(IQueryable<T> source, int pageIndex, int pageSize)
    {
        PageIndex = pageIndex;
        PageSize = pageSize;
        TotalCount = source.Count();
        TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);

        this.AddRange(source.Skip(PageIndex * PageSize).Take(PageSize));
    }


However you don’t need the above code – instead take a look at the CodePlex project called PagedList located at http://pagedlist.codeplex.com/

PagedList allows you to take any List<T> and by specifying the page size and desired page index, select only a subset of that list, there’s also a StaticPagedList if you should need that.

Simply download the source, run “Release.bat” to build a release version and reference the .dll file in your solution, then you can have code like this:

var somePage = list.ToPagedList(4, 50); // fifth page, page size = 50

Using LINQPad to query the database

Monday, 18 January 2010 18:17 by mha

If you’re working with LINQ, you simply can’t live without LINQPad. Below is a screenshot from the query:

from ans in UserAnswers
group ans by new {ans.AnswerId, ans.Answer.Content, 
                  Q=ans.Answer.Question.Content}
into myGroup
select new {myGroup.Key.Q, myGroup.Key.Content, Antal = myGroup.Count()}
 

- being able to do realtime queries against the database is simply awesome. And for only $29 you’ll have Autocompletion (which of course is a must-have / timesaver).

The query shows simple statistics about a poll, where question (Q), answer (Content) and total votes (Antal) is pulled out from the database – I’ve blurred the value of Count() in order to “protect” the clients data…

limfjordspoll_linq

Categories:   LINQ | Tools
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

LINQ Projection and SelectListItem

Sunday, 17 January 2010 15:16 by mha

A small example of where LINQ Projection might come in handy is when you’re working with MVC. In the below example LINQ Projection is used to transform a enumeration of EmployeeCategory instances into an enumeration of SelectListItem instances:

var rep = new mhaRepository();
var person = rep.GetPersonByID(id);
ViewData["EmployeeCategories"] = from c in rep.EmployeeCategories
                         select new SelectListItem
                         {
                             Text = c.Name,
                             Value = c.ID.ToString(),
                             Selected = (c.ID == person.CategoryID)
                         };

In the View all that is needed to render the DropDownList is this:

<%= Html.DropDownList("EmployeeCategories") %>

Categories:   LINQ | ASP.NET MVC
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

InvalidCastException on linq DB submit with non-integer key – aka LinqToSQL bug

Friday, 27 November 2009 09:02 by mha

Wasted a whole day on a (confirmed) bug in LinqToSQL. The problem is this:

“When database contains two tables, both with automatic integer primary keys, and a relationship between a unique char field in one and a non-unique char field in the other, inserting new rows into the second table fails on submit with InvalidCastException"

The error returned be LinqToSQL is: specified cast is not valid

I have a table which contains a char column which is ForeignKey to a unique char column in another table. In my case the column is “CurrencyISO” (e.g. the value DKK) in one table which has a FK relation to a unique ISO4217_ISO column in my Currency table.

Running on my development machine (Windows 7, x64) the code executes without problem – however running on my development server (Windows Server 2008 Web edition, x64) LinqToSQL at random (more or less!) throws the exception: specified cast is not valid

After having spend hours on the bug (I thought it was related to different OS and/or differences in MS SQL Server or something other “weird”), I found this article on Microsoft Connect which confirms the bug and the message “I can confirm that this bug has been fixed for .NET 4.0, which is why it's still failing with .NET 3.5 SP”

However a hotfix is available (and it will fix this bug!) – You can download yourself from Connect or if you’re running an x64 OS (Vista and/or Windows Server 2008), you can simple download the below .ZIP file which contains the hotfix (I’ve installed this on my Windows Server 2008 Web edition (x64), rebooted (an IIS reset should be enough!) and finally the bug was gone!!

Grab the hotfix (NDP35SP1-KB963657-x64.exe) for x64 platforms here:

Categories:   LINQ | SQL/Database
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

“Dynamic” LINQ queries – using query composition and deferred execution

Tuesday, 10 November 2009 18:03 by mha

dynlinqtable

LINQtoSQL is GREAT, however we often need to perform dynamic queries, which is not directly supported by LINQ – however if we use the capability of LINQ to do query composition and deferred execution we can perform most of the ‘dynamic’ queries we need (without having to use the PredicateBuilder).

Take a look at this article which shows how to do “dynamic” LINQ to SQL queries just be using query composition and deferred execution:

http://blogs.msdn.com/swiss_dpe_team/archive/2008/06/05/composable-linq-to-sql-query-with-dynamic-orderby.aspx

If you like LinqDataSource (and the benefits it provide, e.g. automatic handling of sort etc.), but would like it to behave a bit more “dynamic”, you should take a look at the “Selecting” event. Scott Gu has an excellent article about just that – Enjoy! :-)

Categories:   LINQ
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

LINQtoSQL Samples

Monday, 9 November 2009 18:46 by mha

If you’re looking for some great LINQtoSQL samples look no further than MSDN.

As part of the 101 LINQ samples project, Microsoft has put together some nice LINQtoSQL samples – take a look at:

Categories:   SQL/Database | LINQ
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

LINQ - SqlMethods.Like method

Friday, 17 July 2009 14:22 by mha

Using LINQ we can generate SQL which uses the LIKE behaviour using .Contains, like this:

var qMails = from m in db.Mails
             where m.Subject.Contains("M")
             select new { m.MailID, m.ToEmail, m.Subject };

However in some situations the auto-generated SQL created by LINQ will not perform very good. E.g. the query:

var qMails = from m in db.Mails
             where m.Subject.StartsWith("M") && m.Subject.Contains("o")
             select m;

..will cause the generated SQL to include:

WHERE  Subject LIKE [M%]
AND    Subject LIKE [%o%] 

And that's not exactly what we want. So in order to force LINQ to generate proper SQL we can simply use the SqlMethods (located in the System.Data.Linq.SqlClient namespace) helper like this:

var qMails = from m in db.Mails
             where SqlMethods.Like(m.Subject, "M_o%")
             select m;
 

Now, the generated SQL will contain: WHERE Subject LIKE [M_o%] .. which is exactly what we want :)

Categories:   ASP.NET | LINQ
Actions:   E-mail | del.icio.us | Permalink | Comments (1) | Comment RSSRSS comment feed