Large Array DML problem

I am attempting to insert a very large number of records (66,736 in the case I'm workign on) into a table using Array DML in C++Builder 5 with Direct Oracle Access 3.4.6.4. In order to keep the user informed that something is going on I'm using the TOracleQuery.ExecuteArray method, pushing 1,000 records at a time. When the ExecuteArray uses the values of 65001 for the index, I get a key violation. If I don't halt execution and let it continue I end up with 65,536 records in the table.

If I break up the records into a block of 60,000 and a block o 6,736 then all the records post fine.

Any thoughts?

------------------
Jeff Watson
 
Perhaps you can use the OnArrayError event to log the key value that causes the error? This may provide a clue.

------------------
Marco Kalter
Allround Automations
 
Hi Marco,

Thanks for the quick response.

I constructed a small program in Delphi 6 to try and duplicate this problem as follows:

I'm using a table with three fields each of which is a NUMBER(10) and all of which are part of the Primary Key.

I created a new Project, dropped a TOracleSession and a TOracleQuery object on the form with a button to use to test. I set the connection information at design time. The OnClick of the button is as follows:

procedure TForm1.Button1Click(Sender: TObject);
var
V_ISSUEID: Variant;
V_ALIASID: Variant;
V_INCLUDEDID: Variant;
i: integer;
begin
V_ISSUEID := VarArrayCreate([1,70000], varVariant);
V_ALIASID := VarArrayCreate([1,70000], varVariant);
V_INCLUDEDID := VarArrayCreate([1,70000], varVariant);

for i := 1 to 70000 do begin
V_ISSUEID := 66666;
V_ALIASID := i;
V_INCLUDEDID := i;
end;

OracleSession1.Connected := True;
with OracleQuery1 do begin
DeclareVariable('ISSUEID', otInteger);
DeclareVariable('ALIASID', otInteger);
DeclareVariable('INCLUDEDID', otInteger);
SQL.Add('INSERT INTO ALIASES (ISSUEID, ALIASID, INCLUDEDID) ' +
'VALUES (:ISSUEID, :ALIASID, :INCLUDEDID)');
SetVariable('ISSUEID', V_ISSUEID);
SetVariable('ALIASID', V_ALIASID);
SetVariable('INCLUDEDID', V_INCLUDEDID);

executeQueryIncrementally(OracleQuery1, 1000, 70000);
end;
OracleSession1.Commit;
end;

The executeQueryIncrementally function is as follows:

procedure executeQueryIncrementally(q: TOracleQuery; incSize, numRecords: integer);
var arrayPos, numArray: integer;
begin
arrayPos := 0;
while arrayPos < numRecords do begin
numArray := numRecords - arrayPos;
if (numArray > incSize) then begin
q.ExecuteArray(arrayPos, incSize);
Inc(arrayPos, incSize);
end
else begin
q.ExecuteArray(arrayPos, numArray);
Inc(arrayPos, incSize);
end;
end;
end;

When the 65001-66000 ExecuteArray is executed I get a key violation. Given that every value in the IssueID variant is the same and every value in the AliasID and IncludedID are unique within an array location, this should have worked, yes?

I'm I doing something wrong with Array DML that isn't supported? Any suggestions would be appreciated.

------------------
Jeff Watson
 
If you can send me this project and the create table statement for this table, I will check it out.

------------------
Marco Kalter
Allround Automations
 
I tested your demo, and apparently the array size cannot be larger than 65534 elements. You should modify your code so that the actual variant arrays that are passed to the array variables of the query do not contain more than 65534 elements.

------------------
Marco Kalter
Allround Automations
 
Thanks for the investigation Marco. I've make changes to my code to not attempt to process variant arrays larger than 60,000 records.
 
Back
Top