-->
These old forums are deprecated now and set to read-only. We are waiting for you on our new forums!
More modern, Discourse-based and with GitHub/Google/Twitter authentication built-in.

All times are UTC - 5 hours [ DST ]



Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 25 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Lazy loading problem
PostPosted: Wed Aug 27, 2003 7:47 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
The problem is in Hibernate's lazy loading: you can't load non-initialized objects after session is disconnected/closed.

Quote:
Gavin King:
>but what other possibility _is_ there??

How about transparent distributed non-transactional lazy loading? I hate to say this, but CocoBase supports it :)

Quote:
Gavin King:
the whole point behind a layered architecture is that you cant access the lowest layer (the db) from the much higher layers (the web tier).

Out-of-session lazy loading doesn't break anything. You can treat it as more sophisticated way to pass objects to another layer.

Quote:
Gavin King:
In practice, it isn't a nightmare at all.

It _IS_ a nightmare in a layered environment with _complex_ graphs of objects (huh, try to preload 50Mb of interlinked objects from 200 tables).

Proposed solution to this problem: add support for plugable lazy-loading proxies.

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Wed Aug 27, 2003 9:54 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 3:00 pm
Posts: 1816
Location: Austin, TX
I can't speak to the feasibility of implementing plugging support for proxy generators.

However, it seems to be that the focus for Hibernate is OR mapping. It does that very very well. What you are asking for is not OR mapping functionality; it is getting more into the realm of application frameworks.

I will say that at the time I moved to Hibernate, I also evaluated OJB. At that time, OJB was testing out adding exactly what you are talking about. The way they initially tried it (this was a year or so ago, so this probably has changed) was through a session ejb which the proxy could call back into.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Aug 27, 2003 10:03 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
steve wrote:
I can't speak to the feasibility of implementing plugging support for proxy generators.

But why? It won't affect existing Hibernate users, they will just use the default configuration of a new proxy layer :)

Quote:
However, it seems to be that the focus for Hibernate is OR mapping. It does that very very well. What you are asking for is not OR mapping functionality; it is getting more into the realm of application frameworks.

Generaly speaking, lazy loading also isn't OR mapping functionality. Should we throw it away?

Quote:
I will say that at the time I moved to Hibernate, I also evaluated OJB. At that time, OJB was testing out adding exactly what you are talking about. The way they initially tried it (this was a year or so ago, so this probably has changed) was through a session ejb which the proxy could call back into.

I know about OJB, but it's query language and OR mapping are poor in comparision with Hibernate.

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Wed Aug 27, 2003 11:51 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 3:00 pm
Posts: 1816
Location: Austin, TX
Quote:
But why? It won't affect existing Hibernate users, they will just use the default configuration of a new proxy layer :)

Because I am not a hibernate developer, nor am I very familiar that that piece of the code base.

Quote:
Generaly speaking, lazy loading also isn't OR mapping functionality

I beg to differ. Name a single OR tool that does not support lazy loading. There's a reason for this. Otherwise you'd pull your entire database into memory just loading some small entity with myriad realtions. Remote resolution of proxies, however, is clearly outside the realm of pure OR mapping.

I'm not saying this is necessarily undesirable. All I'm saying is that Hibernate rocks because it does what it is meant to do very, very well. But there are a lot of other projects out there that have suffered from trying to "add to the periphery" of their core purpose (ala OJB).


Top
 Profile  
 
 Post subject:
PostPosted: Wed Aug 27, 2003 2:07 pm 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
NO!! This is very evil functionality! It is essential to the performance and transaction isolation characteristics of Hibernate that the beginning and end of a session (ie. a connection to the database) is demarcated by the developer!

An auto-reconnect feature might look superficially appealing but it will reult in a system that performs like a dog and has doubtful transaction semantics.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 4:44 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
gavin wrote:
NO!! This is very evil functionality! It is essential to the performance and transaction isolation characteristics of Hibernate that the beginning and end of a session (ie. a connection to the database) is demarcated by the developer!

An auto-reconnect feature might look superficially appealing but it will reult in a system that performs like a dog and has doubtful transaction semantics.

Non-transactional lazy loading is _NOT_ auto-reconnect!!!! Currently there is _NO_ way to efficiently pass complex graphs of objects to another level (maybe over the network).

Lazy loading shouldn't have any side-effects if you are not using database-level hacks. After the object is loaded from the database it won't have 'persistent state' in any Hibernate session , so any changes to this object won't be persisted to the database and transactional isolation won't suffer a bit.

Non-transactional lazy loading is just a more sophisticated way of passing objects from one layer to another.

In any case, now I'm myself writing Hibernate remoting project in my free time :)

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 5:31 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Quote:
Currently there is _NO_ way to efficiently pass complex graphs of objects to another level (maybe over the network).



Why is there "no way"? I actually don't understand this. Every serious Hibernate app I have ever written is multi-tiered and I simply havn't found this to be a problem.

Quote:
any changes to this object won't be persisted to the database and transactional isolation won't suffer a bit.



Just because it is a read only situation does NOT mean that it is correct to fetch data nontransactionally! Transaction isolation requires that data be READ transactionally, as well as written transactionally.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 6:27 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
gavin wrote:
Quote:
Currently there is _NO_ way to efficiently pass complex graphs of objects to another level (maybe over the network).

Why is there "no way"? I actually don't understand this. Every serious Hibernate app I have ever written is multi-tiered and I simply havn't found this to be a problem.

I have found this to be a problem in my project. My current project is a geologic analytical application, it has 317 persistent classes (mapped to 205 tables) with _VERY_ complex relations. Client fetches a single 'entry point' instance over the network and then begins analysis of the data retrieved. Analysis requires extensive link navigation and its pattern can't be predicted, so we can't just prefectch necessary objects. Analysis can't be moved to server (it sometimes requires hours of CPU time).

We hadn't found how this can be done with Hibernate :(

Quote:
Quote:
any changes to this object won't be persisted to the database and transactional isolation won't suffer a bit.

Just because it is a read only situation does NOT mean that it is correct to fetch data nontransactionally! Transaction isolation requires that data be READ transactionally, as well as written transactionally.

Yes, but transactional reads are not always a neccessary requirment.

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 6:35 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
Hmmmm. Fair enough. I have probably been so immersed in web applications all my life that I forgot people still do client-side stuff in Java.

I admit that Hibernate does not currently provide for this kind of usage.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 6:36 am 
Hibernate Team
Hibernate Team

Joined: Tue Aug 26, 2003 12:50 pm
Posts: 5130
Location: Melbourne, Australia
(My definition of "multitiered" has always been multiple tiers on the server-side; standard web architecture. Slightly different.)


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 28, 2003 8:10 pm 
Pro
Pro

Joined: Tue Aug 26, 2003 8:07 pm
Posts: 229
Location: Brisbane, Australia
cyberax wrote:
I have found this to be a problem in my project. My current project is a geologic analytical application, it has 317 persistent classes (mapped to 205 tables) with _VERY_ complex relations. Client fetches a single 'entry point' instance over the network and then begins analysis of the data retrieved. Analysis requires extensive link navigation and its pattern can't be predicted, so we can't just prefectch necessary objects. Analysis can't be moved to server (it sometimes requires hours of CPU time).

We hadn't found how this can be done with Hibernate :(


What about some kind of command pattern, perhaps implemented in a transaparent fashion using CGLib type proxying funcionality?


http://article.gmane.org/gmane.comp.jav ... evel/1433/

It describes a solution, where the logic for a lazy loading from client to server is contained in a CGLIB-based interceptor that is added to the "hibernated" objects. Maybe you could do something like this in your client?

Just a thought.

Cheers,
Shorn.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 29, 2003 12:30 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
stolley wrote:
cyberax wrote:
I have found this to be a problem in my project. My current project is a geologic analytical application, it has 317 persistent classes (mapped to 205 tables) with _VERY_ complex relations. Client fetches a single 'entry point' instance over the network and then begins analysis of the data retrieved. Analysis requires extensive link navigation and its pattern can't be predicted, so we can't just prefectch necessary objects. Analysis can't be moved to server (it sometimes requires hours of CPU time).

We hadn't found how this can be done with Hibernate :(


What about some kind of command pattern, perhaps implemented in a transaparent fashion using CGLib type proxying funcionality?


http://article.gmane.org/gmane.comp.jav ... evel/1433/

It describes a solution, where the logic for a lazy loading from client to server is contained in a CGLIB-based interceptor that is added to the "hibernated" objects. Maybe you could do something like this in your client?

Just a thought.

Cheers,
Shorn.

We're now doing exacly this thing (dynamic proxies + EJB session beans) :)

But it is wiser to add support for it in the Hibernate's core. All we need to is add plugable proxy generation support. I will contribute code for this change when it is finished, and I hope it will become a part of standart distribution.

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 29, 2003 12:31 am 
Newbie

Joined: Wed Aug 27, 2003 7:31 am
Posts: 9
Location: Russia, Izhevsk
stolley wrote:
cyberax wrote:
I have found this to be a problem in my project. My current project is a geologic analytical application, it has 317 persistent classes (mapped to 205 tables) with _VERY_ complex relations. Client fetches a single 'entry point' instance over the network and then begins analysis of the data retrieved. Analysis requires extensive link navigation and its pattern can't be predicted, so we can't just prefectch necessary objects. Analysis can't be moved to server (it sometimes requires hours of CPU time).

We hadn't found how this can be done with Hibernate :(


What about some kind of command pattern, perhaps implemented in a transaparent fashion using CGLib type proxying funcionality?


http://article.gmane.org/gmane.comp.jav ... evel/1433/

It describes a solution, where the logic for a lazy loading from client to server is contained in a CGLIB-based interceptor that is added to the "hibernated" objects. Maybe you could do something like this in your client?

Just a thought.

Cheers,
Shorn.

We're now doing exacly this thing (dynamic proxies + EJB session beans) :)

But it is wiser to add support for it in the Hibernate's core. All we need to is add plugable proxy generation support. I will contribute code for this change when it is finished, and I hope it will become a part of standart distribution.

_________________
Sapienti sat!


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 29, 2003 4:32 pm 
Beginner
Beginner

Joined: Fri Aug 29, 2003 3:39 pm
Posts: 33
Location: San Francisco, CA
gavin wrote:
Quote:
Currently there is _NO_ way to efficiently pass complex graphs of objects to another level (maybe over the network).

Why is there "no way"? I actually don't understand this. Every serious Hibernate app I have ever written is multi-tiered and I simply havn't found this to be a problem.

I assume you mean that you initialize all of your proxies before handing them off? Is your graph of objects fairly flat or do you have some tricky code for traversing an arbitrary graph of objects?

Even if initializing the entire graph ahead of time is possible, I'm not sure it's always desirable--you obviously lose some of the benefits of lazy loading, especially if the next tier is not going to touch most of the objects.

gavin wrote:
Quote:
any changes to this object won't be persisted to the database and transactional isolation won't suffer a bit.

Just because it is a read only situation does NOT mean that it is correct to fetch data nontransactionally! Transaction isolation requires that data be READ transactionally, as well as written transactionally.

I can buy that, but why does it have to be the same transaction? For example, I would like to commit all changes in my business layer, and then start a new transaction for the View layer to read in the proxies. Yes, the loaded objects may be "out of date" with respect the objects already loaded by the business layer, but depending on the isolation level this could happen within a single transaction too (I think it resembles "read committed").

What I would like to see is:
Code:
Hibernate.setFallbackSession(session);

Internally this would use a ThreadLocal. The specified session will be used when loading a proxy and the original session has been closed. Users would be required to call this method explicitly, so there is no "auto-reconnect" here, just a notion of the "current" session.


Top
 Profile  
 
 Post subject: My personal experience with Lazy Loading
PostPosted: Tue Sep 02, 2003 12:34 pm 
Beginner
Beginner

Joined: Tue Sep 02, 2003 12:15 pm
Posts: 33
In my company, we had so much data that it was impossible to use the direct value object mapping schemes. Some queries would return maybe a set of 100 parent classes, but with child mappings, it would turn into thousands even tens of thousands of objects. What we did to get around this was implement key classes for each value object. The key class holds just the primary key information for each class. This way, we have direct (pseudo lazy) access to the value objects without all the data. If I see I need the children of a primary value object, I just made my findByPrimaryKey(List keys) through my proxy and they are returned. This way I keep all the data access and loading away from the client and on the server where it should be as well as the performance benefits. I have only recently begun working with hibernate, but I am in the middle of porting this design to work with hibernate. I think hibernate itself would benefit from a design like this rather than the dynamic lazy loading of classes. Just let the programmer decide if they need the data or not and make 1 network call. It really isn't that expensive or intrusive.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 25 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
© Copyright 2014, Red Hat Inc. All rights reserved. JBoss and Hibernate are registered trademarks and servicemarks of Red Hat, Inc.