[librecat-dev] loading MARC records

Christoph Krempe krempe at ub.fu-berlin.de
Mon Mar 27 13:32:58 CEST 2017


It works if I first convert MARC to JSON and than load JSON into
Elasticsearch. Trying to load MARC directly I get an error:

Oops! Undefined subroutine &Catmandu::Store::Elasticsearch::Bag::IGNORE
called at
/usr/local/share/perl/5.20.2/Search/Elasticsearch/Client/5_0/Role/Bulk.pm
line 202, <GEN0> line 100.


Am 24.03.2017 um 16:56 schrieb Patrick Hochstenbach:
> Usually we process the MARC before we put in in ElasticSearch to extract the fields of our interest. But, with an easy fix you can turn the record into MARC-in-JSON when importing into ElasticSearch:
> 
> $ catmandu import -v marcxml --fix 'marc_in_json()' to almabib < marc.xml
> 
> 
> ________________________________________
> From: Christoph Krempe <krempe at ub.fu-berlin.de>
> Sent: Friday, March 24, 2017 4:47 PM
> To: Patrick Hochstenbach; librecat-dev at lists.uni-bielefeld.de
> Subject: Re: [librecat-dev] loading MARC records
> 
> Works, thank you!
> 
> As result of the import I get the whole MARC record in one field
> "record". I'd like to have one fields for every MARC category in the
> record. Are there fixes missing?
> 
> 
> Am 24.03.2017 um 16:15 schrieb Patrick Hochstenbach:
>> ElasticSearch versions >= 2.0 don't allow for underscore fields. You need to add the key_prefix option to your store.
>> E.g.
>>
>> $ catmandu import MARC --type XML  to ElasticSearch --index_name almabib --bag bib --key-prefix bib_ < marc.xml
>>
>> Or create a catmandu.yml configuration file:
>>
>> ---
>> importer:
>>   marcxml:
>>      package: MARC
>>      options:
>>        type: XML
>> store:
>>   almabib:
>>     package: ElasticSearch
>>     options:
>>       index_name: almabib
>>       bag: bib
>>       key_prefix: bib_
>> ...
>>
>> and then do:
>>
>> $ catmandu import -v marcxml to almabib < marc.xml
>>
>> See also: https://metacpan.org/pod/Catmandu::Store::ElasticSearch#COMPATIBILITY
>>
>> Patrick
>> ________________________________________
>> From: librecat-dev-bounces at lists.uni-bielefeld.de <librecat-dev-bounces at lists.uni-bielefeld.de> on behalf of Christoph Krempe <krempe at ub.fu-berlin.de>
>> Sent: Friday, March 24, 2017 2:40 PM
>> To: librecat-dev at lists.uni-bielefeld.de
>> Subject: [librecat-dev] loading MARC records
>>
>> Hi *,
>>
>> I want to load a bunch of MARCXML bibliographic records into
>> ElasticSearch without any additional transformation using catmandu as
>> loader.
>>
>> With
>>
>> catmandu import MARC --type XML  to ElasticSearch --index_name almabib
>> --bag bib < marc.xml
>>
>> I get an empty ES index and error messages like
>>
>> org.elasticsearch.index.mapper.MapperParsingException: Field [_id] is a
>> metadata field and cannot be added inside a document. Use the index API
>> request parameters.
>>
>> Any idea how to get rid of this errors?
>>
>> Thanks!
>>
>> --
>>
>> Mit freundlichen Grüßen
>>
>> Ch. Krempe
>>
>> __________________________________________________
>>
>> Christoph Krempe
>> Abt. Datenverarbeitung / Fachreferat Philosophie
>> Universitätsbibliothek der FU Berlin
>> Garystraße 39
>> D-14195 Berlin
>>
>> Tel.: 030 83854583
>> Fax: 030 838454583
>>
> 
> 
> --
> 
> Mit freundlichen Grüßen
> 
> Ch. Krempe
> 
> __________________________________________________
> 
> Christoph Krempe
> Abt. Datenverarbeitung / Fachreferat Philosophie
> Universitätsbibliothek der FU Berlin
> Garystraße 39
> D-14195 Berlin
> 
> Tel.: 030 83854583
> Fax: 030 838454583
> 


-- 

Mit freundlichen Grüßen

Ch. Krempe

__________________________________________________

Christoph Krempe
Abt. Datenverarbeitung / Fachreferat Philosophie
Universitätsbibliothek der FU Berlin
Garystraße 39
D-14195 Berlin

Tel.: 030 83854583
Fax: 030 838454583

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.uni-bielefeld.de/mailman2/unibi/public/librecat-dev/attachments/20170327/234686ee/attachment.asc>


More information about the librecat-dev mailing list