[librecat-dev] Handling server errors in long running oai-pmh harvests

"Dan Michael O. Heggø" d.m.heggo at ub.uio.no
Mon Dec 5 14:49:59 CET 2016


Do you have any tips on handling intermittent server errors occuring in 
long running harvests?

For a harvest of mine I got around 800.000 records before hitting an error:

http://bibsys-k.alma.exlibrisgroup.com/view/oai/47BIBSYS_UBO/request : 
all at all@oai_komplett at marc21@3535592670002204 : Server closed connection 
without sending any data back

Eliminating random server errors doesn't seem realistic, so I'm 
wondering if Catmandu bails out immediately or does multiple retries, 
and if this is something that can be configured? If not, is the best way 
to handle it to write a small Perl script that catches certain exceptions?

Also, is it possible to pass in the continuation token to the OAI 
importer as a flag or something to resume the harvest from where it stopped?

Thanks alot for your feedback!

Dan Michael

More information about the librecat-dev mailing list