[librecat-dev] expanding limit and maximum_page_size
Patrick Hochstenbach
Patrick.Hochstenbach at UGent.be
Wed Jan 16 11:55:26 CET 2019
Every Catmandu::Searchable (not only in the LibreCat project) gives an advice on the absolute maximum number of items that can added to
a Catmandu::Hits generator. In the current version of Catmandu this absolute maximum is set to 1000 but can be changed with a configuration parameter
in general Catmandu projects:
use Catmandu;
my $store = Catmandu->store(‘ElasticSearch'
index_name => 'librecat',
bags => {
publication => {
maximum_limit => 10000
}
})->bag('publication');
my $hits = $store->search(limit => 10000);
warn $hits->count; #10000 when the store contains 10000 or more items
This said…in LibreCat is better got get a dump of all the records via OAI-PMH, SRU or via the command line tools
bin/librecat export search —bag publication > public/mydump.json
and access it via http://yourserver/librecat/mydump.json
Or, to create a new route where you can page through the results list.
Br
Patrick
> On 15 Jan 2019, at 15:28, Patrick Hochstenbach <patrick.hochstenbach at ugent.be> wrote:
>
> This can be set in your catmandu.local.yml file parameter `maximum_page_size`
>
> The default is set to `1000` to protect the server. If many users request 100000 record page results it can crash the server.
>
> Br
> patrick
>
>> On 15 Jan 2019, at 13:39, Jonathan NORRIS <jnorris at ist.ac.at> wrote:
>>
>> Hello,
>>
>> I was trying to get an export of the total amount of records through the browser.
>>
>> I have the 'maximum_page_size' set to 100000
>>
>> And I'm using a link like so: https://librecat.dev.ist.ac.at/export?fmt=json&limit=100000
>>
>> Yet I can only get 1000 records returned to the browser or in the export file.
>>
>> The thing is I can print out '$hits->total;' in the search function of 'LibreCat/Search.pm' and see that the total number of records is actually found, yet only 1000 is returned to the browser or file.
>>
>> Would anyone have any ideas as to why this could be happening?
>>
>>
>> Thanks,
>> Jonathan
>> IST Austria
>> _______________________________________________
>> librecat-dev mailing list
>> - send list mails to librecat-dev at lists.uni-bielefeld.de
>> - to unsubscribe or change options, visit https://lists.uni-bielefeld.de/mailman2/cgi/unibi/listinfo/librecat-dev
>> - project website: http://librecat.org/
>
More information about the librecat-dev
mailing list