Lazy load of properties in DLinq

[Andrus]

I tried query like

var list=Db.Customers.ToList();

and noticed that all properties are retrieved immediately.

Only few properties are actually used in each unit of work.
Also properties contain large amout of data (images, attachments, text
documentes).

Retireving all properties over internet is slow.
Also big properties take a lot of memory in users computer.

How to force DLinq to retrieve those properties only which are are actually
used?

Andrus.

 

[Jon Skeet [C# MVP]]

I tried query like

var list=Db.Customers.ToList();

and noticed that all properties are retrieved immediately.

Only few properties are actually used in each unit of work.
Also properties contain large amout of data (images, attachments, text
documentes).

Retireving all properties over internet is slow.

Do you really have live *internet* access to the database? Why isn't
this behind a web service or something similar?

Also big properties take a lot of memory in users computer.

How to force DLinq to retrieve those properties only which are are actually
used?

By specifying which ones you want:

var list = Db.Select(customer =>
new { customer.Name, customer.Age }).ToList();

 

[Jon Skeet [C# MVP]]

By specifying which ones you want:

var list = Db.Select(customer =>
new { customer.Name, customer.Age }).ToList();

I should have expanded on this. I don't know if there's any way of
retrieving one property at a time, only when you need it. It's possible
that this isn't implemented in LINQ to SQL. (Personally I've never
needed that for simple properties, only for associations.)

I wouldn't be surprised to find it's available in the Entity Framework,
mind you.

It wouldn't be too hard to fake if you had to - create another class
(e.g. CustomerShell) which is given the ID, and which lazily loads the
properties as and when they're requested. The latency involved would be
pretty hideous though, fetching a single property at a time.

 

[Frans Bouma [C# MVP]]

I tried query like

var list=Db.Customers.ToList();

and noticed that all properties are retrieved immediately.

Only few properties are actually used in each unit of work.
Also properties contain large amout of data (images, attachments,
text documentes).

Retireving all properties over internet is slow.
Also big properties take a lot of memory in users computer.

How to force DLinq to retrieve those properties only which are are
actually used?

var list = from c in db.Customers
select new Customer { ID = c.ID, .. //rest of fields you want
};

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------

 

[Marc Gravell]

Re the discussion of web-service; I was talking to Guy Smith-Ferrier about
what ADO.NET Data Services (aka Astoria), which sounds like it will be a
reasonable option for many "normal" LINQ implementations over a web-service,
but note that it only (AFAIK) supports full entities, not bespoke
projections.

Still, something for consideration as it matures.

Of course, I'm not sure that even a RESTful web-service could code very well
with the dynamic type-creation model... ditto for any standard client proxy.

Marc

 

[Jon Skeet [C# MVP]]

Re the discussion of web-service; I was talking to Guy Smith-Ferrier about
what ADO.NET Data Services (aka Astoria), which sounds like it will be a
reasonable option for many "normal" LINQ implementations over a web-service,
but note that it only (AFAIK) supports full entities, not bespoke
projections.

Last time I asked Guy about it, he wasn't aware of any attempt to use
LINQ on the Astoria *client* - it was a possibility I was somewhat
excited about. (LINQ on the server to actually fetch the data which
would then be returned, sure - but that's old hat by now!

Still, something for consideration as it matures.

Of course, I'm not sure that even a RESTful web-service could code very well
with the dynamic type-creation model... ditto for any standard client proxy.

There's an open source project which has similar goals:
http://www.codeplex.com/glinq

No idea how far it's got, but I should look into it at some point...

 

[Marc Gravell]

Last time I asked Guy about it, he wasn't aware of any attempt to use

LINQ on the Astoria *client*

Well, unless I had a brain-freeze, that is *precisely* what he was demoing
in my office earlier this month ;-p

You can get the PPT from his site (www.guysmithferrier.com); note slide 30:
"LINQ to ADO.NET Data Services" - i.e. LINQ to Astoria *at the client*.

I must admit, I haven't tried the CTP yet (the install is a little hairy),
but I'm very keen to when I get a chance (and enough time to refresh a VM
that I can trash).

Marc

 

[Jon Skeet [C# MVP]]

Well, unless I had a brain-freeze, that is *precisely* what he was demoing
in my office earlier this month ;-p

Cool. It was a few months ago that I asked him, admittedly

You can get the PPT from his site (www.guysmithferrier.com); note slide 30:
"LINQ to ADO.NET Data Services" - i.e. LINQ to Astoria *at the client*.

Fabulous. I'd love to think this was in response to my feedback, but I
strongly suspect it was already in the works but not in the version he
was demo-ing at the time.

I must admit, I haven't tried the CTP yet (the install is a little hairy),
but I'm very keen to when I get a chance (and enough time to refresh a VM
that I can trash).

It is a great use of VMs...

 

[Andrus]

Jon,

By specifying which ones you want:

var list = Db.Select(customer =>
new { customer.Name, customer.Age }).ToList();

Thank you.
I need to create updatable customer list containing dynamic columns.
I tried to use Dynamic Linq Library Select() function

IQueryable<Customer> gqueryable =
(IQueryable<Customer>)db.Customers.Select("new(CustomerID,City)");

and

IQueryable<Customer> gqueryable =
db.Customers.Select("new(CustomerID, City)").Cast<Customer>();

Both statements cause cast error at runtime.

How to create updatable Customer instance list from dynamic Customer
property list ?

Andrus.

 

[Andrus]

Frans,

var list = from c in db.Customers
select new Customer { ID = c.ID, .. //rest of fields you want
};

Thank you. Excellent.
List of properties required is only known at runtime in my application.
How to build list of properties for this command at runtime ?
I tried Dynamic Linq Library Select() function but this does not allow to
create Customer type entity.

Andrus.

 

[Jon Skeet [C# MVP]]

I need to create updatable customer list containing dynamic columns.

I suspect you may find that tricky/impossible. I don't know for sure,
but I doubt that you'll be able to load updatable entities partially.

 

[Andrus]

Jon,

I suspect you may find that tricky/impossible. I don't know for sure,
but I doubt that you'll be able to load updatable entities partially.

I tried Frans great suggestion in this thread to load and update using
partial columns:

Northwind db = CreateDB();
var cust = (from c in db.Customers
select
new Customer {
CustomerID = c.CustomerID,
City = c.City
}).First();

cust.City = "Tallinn";
db.SubmitChanges();

This works OK.
Why do you suspect that this is tricky or impossible ?
Unresolved thing is to pass property list dynamically to this statement.

Solutions:

1. Create expression tree or only select part programmatically

2. Modify MS Dynamic Linq library Select() to allow create customer class
using syntax like

Select( "new Customer ( " + columnlist+")" );

and allow to use generic IQueryable

3. Add Dynamic Select() to Marc Dynamic query extension library.

4. Use dynamic compilation.

Which is the best solution ?

Andrus.

 

[Jon Skeet [C# MVP]]

I tried Frans great suggestion in this thread to load and update using
partial columns:

Northwind db = CreateDB();
var cust = (from c in db.Customers
select
new Customer {
CustomerID = c.CustomerID,
City = c.City
}).First();

cust.City = "Tallinn";
db.SubmitChanges();

This works OK.
Why do you suspect that this is tricky or impossible ?

I just didn't expect it to work if it had only loaded some of the
columns. I'm glad for you that it does though.

Unresolved thing is to pass property list dynamically to this statement.

Solutions:

1. Create expression tree or only select part programmatically

2. Modify MS Dynamic Linq library Select() to allow create customer class
using syntax like

Select( "new Customer ( " + columnlist+")" );

and allow to use generic IQueryable

3. Add Dynamic Select() to Marc Dynamic query extension library.

4. Use dynamic compilation.

Which is the best solution ?

Not having done any of them, I'm not really in a position to recommend.

 

[Marc Gravell]

1 is sounds achievable via Expression.New with Expression.Bind, but
won't be trivial; I'd start by compiling something similar and reverse-
engineering it
2 is the same as 1? Just with writing a parser too... extra work
3 I don't undersand
4 You already know my thoughts on this, but it would be consistent
with your other code, and probably less work than 1?

I know I've said it before, but can I just say it again? You are
fighting LINQ every step of the way... I fully expect that after /
this/ issue, it won't be long before you're trying to out-maneuver
LINQ in /another/ issue...

Marc

 

[Marc Gravell]

This works OK.

Why do you suspect that this is tricky or impossible ?

Jon's gut feeling was well-justified; this might work with DbLinq, but
it doesn't work with LINQ-to-SQL; I haven't tried EF (I'm waiting for
vNext on this CTP...).

Additionally, and importantly, note that there is a major risk of data
loss with this approach: if the back-end decides to set all columns,
it will set most of them to blank/0/null etc. This is pretty-much
guaranteed if you are using SPs to update data, but if you use dynamic
(generated) CRUD it will /probably/ be OK. On your head be it, etc...

I posted more details to the parallel chain:
http://groups.google.co.uk/group/microsoft.public.dotnet.languages.csharp/msg/c9f0cf167b355e20

Note that this also contains a "best guess" at option 1. It works with
LINQ-to-objects, but I haven't tried it on DbLinq.

In particular, note that my expression code isn't the problem: the
following raises exactly the same error in LINQ-to-SQL:

select new Customer { CustomerID = c.CustomerID, City = c.City }

Annoyingly, I wrote the code using LINQ-to-objects while waiting for
SQLExpress+management studio to download, only to find it fail at the
first hurdle.

Oh well. Let me know how the code behaves with DbLinq.

Marc