[librecat-dev] expanding limit and maximum_page_size

Patrick Hochstenbach Patrick.Hochstenbach at UGent.be
Wed Jan 16 11:55:26 CET 2019


Every Catmandu::Searchable (not only in the LibreCat project) gives an advice on the absolute maximum number of items that can added to
a Catmandu::Hits generator. In the current version of Catmandu this absolute maximum is set to 1000 but can be changed with a configuration parameter
in general Catmandu projects:

use Catmandu;

my $store = Catmandu->store(‘ElasticSearch'
			index_name => 'librecat',
			bags => { 
				publication => { 
					maximum_limit => 10000 
				}
			})->bag('publication');

my $hits = $store->search(limit => 10000);

warn $hits->count; #10000 when the store contains 10000 or more items

This said…in LibreCat is better got get a dump of all the records via OAI-PMH, SRU or via the command line tools

 bin/librecat export search —bag publication > public/mydump.json

and access it via http://yourserver/librecat/mydump.json

Or, to create a new route where you can page through the results list.

Br
Patrick 

> On 15 Jan 2019, at 15:28, Patrick Hochstenbach <patrick.hochstenbach at ugent.be> wrote:
> 
> This can be set in your catmandu.local.yml file parameter `maximum_page_size`
> 
> The default is set to `1000` to protect the server. If many users request 100000 record page results it can crash the server.
> 
> Br
> patrick
> 
>> On 15 Jan 2019, at 13:39, Jonathan NORRIS <jnorris at ist.ac.at> wrote:
>> 
>> Hello,
>> 
>> I was trying to get an export of the total amount of records through the browser.
>> 
>> I have the 'maximum_page_size' set to 100000
>> 
>> And I'm using a link like so: https://librecat.dev.ist.ac.at/export?fmt=json&limit=100000
>> 
>> Yet I can only get 1000 records returned to the browser or in the export file.
>> 
>> The thing is I can print out '$hits->total;' in the search function of 'LibreCat/Search.pm' and see that the total number of records is actually found, yet only 1000 is returned to the browser or file.
>> 
>> Would anyone have any ideas as to why this could be happening?
>> 
>> 
>> Thanks,
>> Jonathan
>> IST Austria
>> _______________________________________________
>> librecat-dev mailing list
>> - send list mails to librecat-dev at lists.uni-bielefeld.de
>> - to unsubscribe or change options, visit https://lists.uni-bielefeld.de/mailman2/cgi/unibi/listinfo/librecat-dev
>> - project website: http://librecat.org/
> 




More information about the librecat-dev mailing list