Print Thread
Importer - Text or ODBC
#60624 10/23/19 03:18 PM
Joined: Jan 2015
Posts: 89
Cambridge, UK
C
cct Offline OP
Member
OP Offline
Member
C
Joined: Jan 2015
Posts: 89
Cambridge, UK
I know I have raised this in the past, but just had to import 136654 rows (of 10 simple fields) into an empty table.

Still chuntering along, now looking to be complete in over the hour.

This time with the ODBC importer, but comparable with the text inmporter.

Have you had a chance to review the processing, as this not acceptable performance, and this time SQL Developer is not an option, as on an Oracle 8 database.

Thanks


Chris

Re: Importer - Text or ODBC
cct #60626 10/24/19 08:56 AM
Joined: Aug 1999
Posts: 22,207
Member
Offline
Member
Joined: Aug 1999
Posts: 22,207
Can you check in the database if the import is still loading records? In other words: is it slow or has it stopped?


Marco Kalter
Allround Automations
Re: Importer - Text or ODBC
Marco Kalter #60639 10/28/19 11:25 AM
Joined: Jan 2015
Posts: 89
Cambridge, UK
C
cct Offline OP
Member
OP Offline
Member
C
Joined: Jan 2015
Posts: 89
Cambridge, UK
It is loading records. Initially the count goes up, and one can estimate the time remaining fairly accurately. Sometimes, but not always the count ceases to display, but the load seems to continue at the same rate.

This is not a new problem - I have reported this a number of times before

Thanks


Chris

Re: Importer - Text or ODBC
cct #61851 07/09/20 12:12 PM
Joined: Jul 2020
Posts: 4
B
Member
Offline
Member
B
Joined: Jul 2020
Posts: 4
hi, I registered just to chip in on this.

If you're importing a large number of records (>5000), use sqlldr which is the command line tool (it probably calls the same libraries) for which every Oracle consultant since the year dot carried a manual in their car.

Check it out on the web and particularly look at direct=true parameter.

136k rows would take around 30 seconds in my experience. Setup for a 10 field table (you have to map the data file so SQLLDR knows how to process it) would take a few minutes at most. Dates and non-integer numbers need a little care, varchar2 are mapped as CHAR and numbers without precision are mapped as EXTERNAL INT...and your needs to be in a csv.

SQLLDR should be in your Oracle client installation and accessible from the command line.

The reason PLSQL Dev import is slow is that it's committing each batch of data but if your data is uniform, there's no need to do this. (And also I don't think the memory management is quite right.)


Moderated by  support 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 7.1.33 Page Time: 0.038s Queries: 14 (0.010s) Memory: 2.5105 MB (Peak: 3.0380 MB) Data Comp: Off Server Time: 2024-05-02 03:42:03 UTC
Valid HTML 5 and Valid CSS