[librecat-dev] collaborate with Catmandu ?

[Patrick Hochstenbach]

Hi Marc

I see you still had a question why we used a flow controlled by an exporter. The answer is twofold: 1) yes we use exporters to create a uniform interface around many coversion/load-into-database processes that are very similar 2) you don’t need to use this mechanism, it is possible to create a stream of data you control. E.g.

use Catmandu;

my $importer = Catmandu->importer('Mock');
my $fixer       = Catmandu->fixer(['add_field(hello,world)']);

while (my $data = $importer->next ) {
    # Fix the data
    $fixer->fix($data);

    my $str = Catmandu->export_to_string($data,'YAML');
    print "$str\n";
}

Patrick

> On 23 Jan 2019, at 12:15, Marc Chantreux <mc at unistra.fr> wrote:
> 
> hello Patrick,
> 
> thanks for that and the hour we took monday to exchange ideas.
> 
>> use Catmandu;
>> my $importer = Catmandu->importer('Null');
>> my $fixer    = Catmandu->fixer(['add_field(hello,world)']);
>> my $output   = '';
>> my $exporter = Catmandu->exporter('YAML', file => \$output);
>> $exporter->add_many($fixer->fix($importer));
>> print "$output”;
> 
> which made me remember a very old discution we had: the idea was
> that the flow is controled by the exporter and i wanted to bet on
> perlude for that. i actually still do ... which means "a lot of
> redundant logic".
> 
> can you explain the benefit of catmandu exporter over a simple stream?
> 
> anyway, i'll take the workflow part aside for the moment: i'll focus on
> 
> * see how to use a catmandu filter and importer as a part of a perlude
>  stream
> * try to get the result of the parsing of the fix langage to see how it
>  is possible to use it in combination with the DataVM.
> 
> i'll do that on my spare time for the moment so don't expect a fast pace
> progress :)
> 
> regards.
> marc
>