Mismatch between SourceDB and TargetDB by 500K ?

Posted by Adrian.wright on 27-Dec-2018 09:31

Hi All 

we started a Bullkload Proccess on a Table with 70 million Records which took about 7 days plus to complete,  but when the Bulk upload completed there is a difference of about 500K records missing from Target\SQL server, 

I hoped when we enabled the Table Triggers this would autmatically catch up\replicate the missiing 500K which after 4 days its only provided the Table Trigger updates, so the questions I have are....

1. why\how did this occur, is this beacuse we didn't enable the Database Triggers at the same time as the Bulkload ? and is this something we should do for furture tables ?

2. The missing 500K is probably the difference between the Bulkload starting and the 7 days it took to finish,  so that would imply that the Record count is taken at the start of the bulkload ? so everythin after that should be an update ? and should the Table trigger not have transferred the 500K ? 

3. There also in the "REPLQUEUE" Table 18,000 records that have all got the "Bulk-copy-err" status,  When I have tried to the bulkload for the Same Table,  it returns the "Bulkload for the this table is complete"  How do I move or load these records from PRO2 into the Target Database ?

4. the most important,  how do we rectify the missing 500K, would stating a Bulkload again for that table be a risk ? as it is 70 millions records in the table I would not want to have truncate the Target Table and start again from scratch especially just for 500K records

Many Thanks in Advance

Ade 

   

All Replies

Posted by Satya Prasad on 28-Dec-2018 09:44

1. why\how did this occur, is this beacuse we didn't enable the Database Triggers at the same time as the Bulkload ? and is this something we should do for future tables ?

Yes. This might be the reason for out of sync. The right sequence is, enable database triggers and then bulk-load the tables. Once bulk-load completes then turn on the replication threads to process the updates in the replqueue.

2. The missing 500K is probably the difference between the Bulkload starting and the 7 days it took to finish,  so that would imply that the Record count is taken at the start of the bulkload ? so everythin after that should be an update ? and should the Table trigger not have transferred the 500K ? 

This 500k is the difference between the time-gap of enable trigger vs bulk-load. More the gap among them will cause more records to be out of sync.

3. the most important,  how do we rectify this, would stating a Bulkload again for that table be a risk ? as it is 70 millions records in the table I would not want to have truncate the Target Table and start again from scratch especially just for 500K records

I am not aware of any solution to rectify this other than bulk-load of that particular table. However, if you can know those 500k rowid’s then you can rectify by creating replqueue records for those 500k records.
 
Hope this Helps, Satya
 

This thread is closed