[librecat-dev] Catmandu and Hadoop/Spark?

Jonas Smedegaard dr at jones.dk
Wed Feb 17 13:49:38 CET 2016


Quoting Günter Hipler (2016-02-17 12:07:42)
> it would be nice to integrate Catmandu in such processes. But I think 
> the integration of a Perl based framework is less natural compared to 
> e.g. Python. All these "Big Data" components are Java/Scala based and 
> Perl is not part of the JVM world (might change in the future with 
> Perl6). Spark and Flink (https://flink.apache.org/) are providing 
> specialized Python clients.
> 
> I know we already have had this discussion more than one year ago ;-) 
> and for me this was one important reason to use Metafacture for our 
> project (swissbib). But I still hope both frameworks (Catmandu / 
> Metafacture) are coming closer together in the future.

I don't see how PYthon should be any simpler than Perl to handle big 
data - whether it be link against or reimplement java-based tools.  Only 
reason as I see it is that Python is perceived as more popular and 
therefore wrappers etc. are more often created for that.

One way to link Perl to java is to use Inline::Java: 
https://metacpan.org/pod/Inline::Java


> On 02/17/2016 10:28 AM, Jakob Voß wrote:
>> I just got asked whether Catmandu (or Perl in general) can be used 
>> with Hadoop or Spark. Has anyone of you tried this before?

...but sorry, that doesn't really address your question, Jakob.

Perhaps some of these may help further: 
https://metacpan.org/requires/distribution/Inline-Java


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: signature
URL: <http://lists.uni-bielefeld.de/mailman2/unibi/public/librecat-dev/attachments/20160217/d6bc046d/attachment.asc>


More information about the librecat-dev mailing list