[librecat-dev] Catmandu and Hadoop/Spark?

Jakob Voß Jakob.Voss at gbv.de
Wed Feb 17 10:28:44 CET 2016


Hi,

I just got asked whether Catmandu (or Perl in general) can be used with 
Hadoop or Spark. Has anyone of you tried this before? This is what I 
found for Spark:

https://wiki.ufal.ms.mff.cuni.cz/spark:recipes:using-perl-via-pipes

Although we successfully do processing of large data sets with Catmandu, 
I guess it has its limitations with "big data" (whatever that means). 
Maybe it's worth to use Catmandu on top of existing big data frameworks 
such as Hadoop and Spark instead of extending Catmandu with big data 
features such as massive parallel processing?

Just a thought,
Jakob

-- 
Jakob Voß <jakob.voss at gbv.de>
Verbundzentrale des GBV (VZG) / Common Library Network
Platz der Goettinger Sieben 1, 37073 Göttingen, Germany
+49 (0)551 39-10242, http://www.gbv.de/


More information about the librecat-dev mailing list