[librecat-dev] fixes and tests

[Patrick Hochstenbach]

I get some personal emails asking to clarify the replies on this subject. Here are my two cents:

What is the use of testing?

The Test::More question is not quite clear. What catmandu (and catmandu-marc) does is transforming records from one format to another format. To do these transformations many Catmandu fixes were developed. All Catmandu fixes are tested to check if they actually do what they claim to do. For instance,

marc_set('100a','Bla)

should set the 100a field of a MARC input record not just a MARC input record, but any MARC input record. If this 'marc_set' doesn't do what it claims to do, then it is a bug and has to be reported at out GitHub issues https://github.com/LibreCat/Catmandu-MARC/issues

Now, if you trust Catmandu and that it tested and works as it should, does that mean that your own fixes automatically do everything what you want to them to do?

Here Catmandu isn't different then ..lets say .. Excel. We (catmandu programmers) can't guess which combinations of calculations our end users are going to do on their own datasets. If Excel gives the wrong answer for tax returns, is this an error in the Excel program or the spreadsheet. How can Microsoft make a test for a random spreadsheet? This is something that needs to be done by the person creating the spreadsheet. Microsoft can't guess which spreadsheet one is going to create. 

The same holds for Catmandu. To test the catmandu fix file itself one needs a way to check if the MARC output is still valid to local catalogging rules. Alas, this MARC validation isn't part of the Catmandu Fix language. There a generic Catmandu::Importer::MARC::Lint that has some validation rules given by MARC::Lint. If you have a MARC marc and a Fix you can create an output like

   $  catmandu convert MARC --type RAW --fix myfixes.fix to MARC --type USMARC < input.mrc > output.mrc

And check both the validatity of the input and output

  $ catmandu convert MARC --type Lint <  input.mrc
  $ catmandu convert MARC --type Lint < output.mrc

And see if you like the results or not. But most probably..very certainly ..every library has its own local validation rules what is an 'okay' record. This falls outside the scope of the Catmandu Fix language currently. One needs to create an own validation program like Johan suggested with for instance MARC::Record to test the local validation rules for any type of MARC record.

Cheers

Patrick 

________________________________________
From: librecat-dev-bounces at lists.uni-bielefeld.de <librecat-dev-bounces at lists.uni-bielefeld.de> on behalf of Rolschewski, Johann <Johann.Rolschewski at sbb.spk-berlin.de>
Sent: Tuesday, September 6, 2016 1:29 PM
To: aina at openmailbox.org
Cc: librecat-dev at lists.uni-bielefeld.de
Subject: Re: [librecat-dev] fixes and tests

Hi,

> i think the example of the use of combination of fixes and tests, would be
> enough if an example of a search and replace fix would be used:
>
> if marc_match('100a','AWrongSpelledName')
>          marc_set('100a','CorrectlySpelledName')
>     end

yes, you could use a fix like this:

do marc_each()
    if marc_match(100a,"OldName")
        marc_set(100a,"NewName")
    end
    if marc_match(245a,"OldTitle")
        marc_set(245a,"NewTitle")
    end
end

You can add some "log"-fixes <https://metacpan.org/pod/Catmandu::Fix::log> to generate some output, but it will not fit your requirements... I would recommend to write a script for this task using a more "low level" module like MARC::Record <https://metacpan.org/pod/MARC::Record>.

Regards,

Johann

_______________________________________________
librecat-dev mailing list
- send list mails to librecat-dev at lists.uni-bielefeld.de
- to unsubscribe or change options, visit https://lists.uni-bielefeld.de/mailman2/cgi/unibi/listinfo/librecat-dev
- project website: http://librecat.org/