Filtering a list - efficiently

[abcd]

I have a list of items. Some of the items are another list and that list item
could be another list. Now I want to filter the main list by the inner list
item value based on some criteria...

My fileter criteria is, A and B and C and D and E then get the filtered
item...where A, B , C ,D, E are passed from the frong end.

How can I efficiently filter the main list. Any generic efficient logic so
that I can make it more generic code...

I was thinking if

- I can convert the list into DataView and filter by SQL queries?
- I can conver the list into XML file and use XPATH and XQuery?

Any better thoughts...

The slection of the technology is the key to the performance for our
application

THanks

 

[Marc Gravell]

It sounds like you have a tree/hive of data. Could you perhaps throw the
data into a databas with a self-referencing, indexed table and just do
it in the database?

If you want to do it in-memory, then the best approach would probably
depend on how exactly you want to filter it. For example, if it is
equality, you could build a dictionary of values against a (list of)
paths to nodes (or the nodes themselves).

So: what would a typical filter look like?

Marc

 

[abcd]

Marc,

Thanks for the reply. I am working in SOA based application. My boundaries
are make a request and get a response. DB intereaction is not directly
involved. Its airline reservation system so filters are show me the flights
which are direct, min stops, certain carrier, etc etc. All the filter
criteria is in AND condition.

In a list I get flights which has certain properties and those properties
value I get as a list item.

Our design is like this...We get the response and we will business object
and business object is filling a list...classes are desicgned like that...

Any thoughts,...

 

[Marc Gravell]

Well, if you are doing repeated AND queries, then one of the easier ways
to build an overall query is using LINQ-to-Objects (no database here) -
something like shown below - but this is still a very basic approach
that doesn't do anything special re indexes. If you need something more
sophisticated you'll need to give a lot of attention to likely queries.

Re the nested data, I don't understand enough from your description to
make much comment. It might be that the below is fine, i.e. if you just
want to check things like (read in context of main body below):

if (via != null)
{
query = query.Where(f => f.Stops.Contains(via));
}

Which again is very simplified; I'd probably suggest a
Dictionary<string, Airport> for things like Stops, so you can say:

string via = "LAX"; // via UI
if (via != null)
{
query = query.Where(f => f.Stops.ContainsKey(via));
}

But again - not a huge amount of context here...

Marc

using System.Linq;
static class Program
{
static void Main()
{
List<Flight> flights = new List<Flight>();

var query = flights.AsQueryable();

int? maxStops = 3; // from UI
if (maxStops != null)
{
query = query.Where(f => f.Stops <= maxStops);
}

Carrier carrier = null; // from UI
if (carrier != null)
{
query = query.Where(f => f.Carrier == carrier);
}

// etc
}
}

 

[abcd]

Marc

THanks for the detailed email. We are right now on VS 2005. So it will take
a while to go with LINQ... but I will create a POC and can push to use the VS
2008 if required...

Thanks

 

[Peter Morris]

I use a tool named "Enterprise Core Objects" for my business objects /
persistence. One of the features is has is an OCL (object constring
language) parser, so you can execute select queries etc on your object
space. I take stuff like this for granted


--
Pete
=========================================
I use Enterprise Core Objects (Domain driven design)
http://www.capableobjects.com/
=========================================

 

[Marc Gravell]

Well, considering is costs approx the same as Visual Studio (pro), I
think I'll stick to LINQ ;-p

But if you like it...

Marc

 

[Peter Morris]

Well, considering is costs approx the same as Visual Studio (pro), I think

I'll stick to LINQ ;-p

I've been using it for years, it does a lot more than LINQ. So yes, I do
like it




--
Pete
=========================================
I use Enterprise Core Objects (Domain driven design)
http://www.capableobjects.com/
=========================================

 

[Alun Harford]

Marc

THanks for the detailed email. We are right now on VS 2005. So it will take
a while to go with LINQ... but I will create a POC and can push to use the VS
2008 if required...

The syntax is nicer with extension methods, but Select(), Where(), etc
are not difficult to implement in C#2.

An example (I've not tested or compiled it so I may have done something
stupid) for Select(...) is:

public static class MyLinq
{
public delegate Tresult SelectionDelegate<T, Tresult>(T obj);
public static IEnumerable<Tresult> Select<T, Tresult>(IEnumerable<T>
input, SelectionDelegate<T, Tresult> func)
{
foreach(T item in input)
{
yield return func(item);
}
}
}

Other 'LINQ' functions are similarly trivial.

The great thing about C# 3 is not LINQ in itself, but the fact that I
can now expect other developers to have no problem understanding what
I'm doing (and the better syntax).

Alun Harford

 

[Marc Gravell]

The syntax is nicer with extension methods, but Select(), Where(), etc

are not difficult to implement in C#2.

In fact LINQBridge does all this for you if you want LINQ-to-objects
on 2.0.

Marc