I use TExportX to export data to Text/HTML/Excel/Word, but I found TOracleDataset always cache the data from the query, so the application will eat many many many memory when my query returns large dataset, how can I ommit the cache feature of TOracleDataset? (TExportX only support the TDataset not TOracleQuery)
TOracleDataSet and TExportX
- Thread starter Che Ming
- Start date
You're are absolutely right and I always wonder why 3-d party dataset manufactures never think about the fact that the customer could have some really huge data. I do remember how many problems I had with Raima Velocis dataset, it was enough just to press Ctrl-End in the quite a table and after a while your application crashed with message "system is out of memory". They did the same thing as DOA do (just keeping all read data in memory cache). I've completely redone the implementation of that dataset and added some kind of buffered file for that cache support. And now application with that dataset never go out of assigned buffer memory size, just do some kind of tidy swapping to temporrary file. If AllAround is interested in I can send the source of that buffered cache stream class.
For batch processing of large numbers of records, we always advice to use the TOracleQuery. It doesn't buffer records in memory, and doesn't have any TDataSet overhead. The TOracleDataSet is primarily designed for use with data-aware controls.
However, there are circumstances where people need or want to use a dataset to process hundreds of thousands of records. Therefore we are considering a TOracleDataSet.UniDirectional property. When True, the TOracleDataSet will not buffer any records. Furthermore we are considering a TOracleDataSet.RecordBufferSize property, which determines how many records can be held in memory before flushing the eldest records to disk.
Andrey, thanks for the offer to send us your buffered cache stream class. Just e-mail the source to me and I'll see if it (or some of it) is suitable for our requirements.
------------------
Marco Kalter
Allround Automations
However, there are circumstances where people need or want to use a dataset to process hundreds of thousands of records. Therefore we are considering a TOracleDataSet.UniDirectional property. When True, the TOracleDataSet will not buffer any records. Furthermore we are considering a TOracleDataSet.RecordBufferSize property, which determines how many records can be held in memory before flushing the eldest records to disk.
Andrey, thanks for the offer to send us your buffered cache stream class. Just e-mail the source to me and I'll see if it (or some of it) is suitable for our requirements.
------------------
Marco Kalter
Allround Automations
Similar threads
