|
String fileds vs Memory usage
|
Joined: Nov 2003
Posts: 18
Member
|
OP
Member
Joined: Nov 2003
Posts: 18 |
Hello,
Why DOA in TOracleDataSet allocates full memory for string fields?
For example I declare two TStringFields: NAME with size 100 and DESCRIPTION with size 200; all records in DataSet uses all memory for these fields (it doesn't matter how many characters these fields contain for each record). So when I declare these fields 100/200 long, DataSet uses (it's example) 3068KB, but with sizes 50/100 it uses "only" 1644KB. Even if all records have empty strings in these fields.
Is there any reason that DOA does not support dynamic memory allocation for strings? When value in record for string field with Size=100 has 15 characters, then it should using memory for 15 characters, not for all 100 chars.
Best regards, Daniel Stoch
|
|
|
Re: String fileds vs Memory usage
|
Joined: Aug 1999
Posts: 22,216
Member
|
Member
Joined: Aug 1999
Posts: 22,216 |
I believe this is the nature of the TDataSet and TField classes. It is all based on fixed size buffers.
Marco Kalter Allround Automations
|
|
|
Re: String fileds vs Memory usage
|
Joined: Nov 2003
Posts: 18
Member
|
OP
Member
Joined: Nov 2003
Posts: 18 |
Sorry but I don't agree with you. It is possible to use dynamic size buffers with TDataSet and TField classes. What does it mean "I believe"? Probably you are not sure. It is very strange for me: you write such components as DOA and you are not sure how does VCL classes are implemented and how to implement dynamic size buffers?!!! Sorry for this example, but ODAC components from CoreLab uses dynamic size buffers, so as you can see it is possible. Your DOA components are very good, but dynamic size buffer can make them much better .
|
|
|
Re: String fileds vs Memory usage
|
Joined: Nov 2003
Posts: 18
Member
|
OP
Member
Joined: Nov 2003
Posts: 18 |
Hello, Could someone reply to my previous post? Or maybe you are implementing "dynamic" string buffers now? Best regards Daniel Stoch
|
|
|
Re: String fileds vs Memory usage
|
Joined: Aug 1999
Posts: 22,216
Member
|
Member
Joined: Aug 1999
Posts: 22,216 |
I looked into this, and it is indeed possible to implement a feature to compress the dataset result set storage. The record buffer needs to comply to TDataSet/TField rules, but one can store the entire result set as one sees fit. Maybe we can add this in a future release.
Marco Kalter Allround Automations
|
|
|
Re: String fileds vs Memory usage
|
Joined: Oct 1999
Posts: 138 Eschborn, Germany
Member
|
Member
Joined: Oct 1999
Posts: 138 Eschborn, Germany |
Hi Macro, is it possible to say more then maybe? I think this is a very very helpfull feature for people working with large datasets and large varchar fields, so i would realy appreciate if we can see a 4.1 in January or February 2004 with this feature Greetings Jens
|
|
|
Re: String fileds vs Memory usage
|
Joined: Aug 1999
Posts: 22,216
Member
|
Member
Joined: Aug 1999
Posts: 22,216 |
For read-only datasets this is relatively simple to implement. For updateable datasets we have to rewrite a large part of the memory management. We have to look into this before I can say anything more specific than "maybe".
Marco Kalter Allround Automations
|
|
|
Re: String fileds vs Memory usage
|
Joined: Nov 2003
Posts: 18
Member
|
OP
Member
Joined: Nov 2003
Posts: 18 |
Hi,
Thanks for your answer Marco and thank you Jens for joining this topic. I think it would be great if you will be able to implement this feature. In our application we process huge amount of data, so effective memory management is a very important for us.
Good luck, Daniel
|
|
|
Re: String fileds vs Memory usage
|
Joined: Nov 2003
Posts: 89 Germany
Member
|
Member
Joined: Nov 2003
Posts: 89 Germany |
[quote]Originally posted by Daniel Stoch: Hi,
In our application we process huge amount of data, so effective memory management is a very important for us.
[/quote]I dont't want to meddle with your discussion, but cost of intensive memory allocation/deallocation for dynamic buffer managment may (or may not) heavily overweight benwefits from memory saving.
Yours,
Oleksandr
|
|
|
Re: String fileds vs Memory usage
|
Joined: Oct 1999
Posts: 138 Eschborn, Germany
Member
|
Member
Joined: Oct 1999
Posts: 138 Eschborn, Germany |
You may be right.
Because of this reason it should be an optional parameter.
Greetings Jens Fudickar
|
|
|
|
|
|