[librecat-dev] expanding limit and maximum_page_size

Jonathan NORRIS jnorris at ist.ac.at
Wed Jan 16 09:56:27 CET 2019


Ok interesting, I logged the size an saw that it was 1000.

total = 5000
size = 1000

If I log the limit that comes back from '_set_limit($p->{limit})' I get 
100000.

https://github.com/LibreCat/LibreCat/blob/master/lib/LibreCat/Search.pm#L58

What sets this 'size' limit of 1000? It would seem that if you set the 
'maximum_page_size' above 1000 you would also want the 'size' limit to 
also match it right?

Thanks,
Jonathan

On 1/16/19 09:29, Nicolas Franck wrote:
> $hits->total is different from $hits->size,
> the first means the total number of hits found,
> the second the total number of records returned.
>
>> On 16 Jan 2019, at 09:20, Jonathan NORRIS <jnorris at ist.ac.at> wrote:
>>
>> Hey,
>>
>> The thing is, I already have maximum_page_size set to 100000 in catmandu.local.yml.
>>
>> The debug logs are showing me that I am getting the full set of records back from the search and there is even a log of the file contents of what is to be sent back, and I can see that it is indeed the full set of records. Yet somehow when I actually download the file there is only ever 1000 records in it. If I change the limit to be less than 1000 then I get the desired amount, but I can't seem to get more than 1000.
>>
>> Also, this will only be for admin functionality and will not be ran very often so I don't fear any server issues with it.
>>
>>
>> Thanks,
>> Jonathan
>>
>> On 1/15/19 15:28, Patrick Hochstenbach wrote:
>>> This can be set in your catmandu.local.yml file parameter `maximum_page_size`
>>>
>>> The default is set to `1000` to protect the server. If many users request 100000 record page results it can crash the server.
>>>
>>> Br
>>> patrick
>>>
>>>> On 15 Jan 2019, at 13:39, Jonathan NORRIS <jnorris at ist.ac.at> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I was trying to get an export of the total amount of records through the browser.
>>>>
>>>> I have the 'maximum_page_size' set to 100000
>>>>
>>>> And I'm using a link like so: https://librecat.dev.ist.ac.at/export?fmt=json&limit=100000
>>>>
>>>> Yet I can only get 1000 records returned to the browser or in the export file.
>>>>
>>>> The thing is I can print out '$hits->total;' in the search function of 'LibreCat/Search.pm' and see that the total number of records is actually found, yet only 1000 is returned to the browser or file.
>>>>
>>>> Would anyone have any ideas as to why this could be happening?
>>>>
>>>>
>>>> Thanks,
>>>> Jonathan
>>>> IST Austria
>>>> _______________________________________________
>>>> librecat-dev mailing list
>>>> - send list mails to librecat-dev at lists.uni-bielefeld.de
>>>> - to unsubscribe or change options, visit https://lists.uni-bielefeld.de/mailman2/cgi/unibi/listinfo/librecat-dev
>>>> - project website: http://librecat.org/
>> _______________________________________________
>> librecat-dev mailing list
>> - send list mails to librecat-dev at lists.uni-bielefeld.de
>> - to unsubscribe or change options, visit https://lists.uni-bielefeld.de/mailman2/cgi/unibi/listinfo/librecat-dev
>> - project website: http://librecat.org/



More information about the librecat-dev mailing list