Type Alias PaginationOptions<ElementType, BodyType>

PaginationOptions<ElementType, BodyType>: {
    transform?: ((response: Response<BodyType>) => Promise<ElementType[]> | ElementType[]);
    filter?: ((data: FilterData<ElementType>) => boolean);
    paginate?: ((data: PaginateData<BodyType, ElementType>) => OptionsInit | false);
    shouldContinue?: ((data: FilterData<ElementType>) => boolean);
    countLimit?: number;
    backoff?: number;
    requestLimit?: number;
    stackAllItems?: boolean;
}

All options accepted by got.paginate().

Type Parameters

  • ElementType
  • BodyType

Type declaration

  • Optionaltransform?: ((response: Response<BodyType>) => Promise<ElementType[]> | ElementType[])

    A function that transform Response into an array of items. This is where you should do the parsing.

    Default

    response => JSON.parse(response.body)
  • Optionalfilter?: ((data: FilterData<ElementType>) => boolean)

    Checks whether the item should be emitted or not.

    Default

    ({item, currentItems, allItems}) => true
  • Optionalpaginate?: ((data: PaginateData<BodyType, ElementType>) => OptionsInit | false)

    The function takes an object with the following properties:

    • response - The current response object.
    • currentItems - Items from the current response.
    • allItems - An empty array, unless pagination.stackAllItems is set to true, in which case, it's an array of the emitted items.

    It should return an object representing Got options pointing to the next page. The options are merged automatically with the previous request, therefore the options returned pagination.paginate(...) must reflect changes only. If there are no more pages, false should be returned.

    Example

    import got from 'got';

    const limit = 10;

    const items = got.paginate('https://example.com/items', {
        searchParams: {
            limit,
            offset: 0
        },
        pagination: {
            paginate: ({response, currentItems}) => {
                const previousSearchParams = response.request.options.searchParams;
                const previousOffset = previousSearchParams.get('offset');

                if (currentItems.length < limit) {
                    return false;
                }

                return {
                    searchParams: {
                        ...previousSearchParams,
                        offset: Number(previousOffset) + limit,
                    }
                };
            }
        }
    });

    console.log('Items from all pages:', items);
  • OptionalshouldContinue?: ((data: FilterData<ElementType>) => boolean)

    Checks whether the pagination should continue.

    For example, if you need to stop before emitting an entry with some flag, you should use ({item}) => !item.flag.

    If you want to stop after emitting the entry, you should use ({item, allItems}) => allItems.some(item => item.flag) instead.

    Default

    ({item, currentItems, allItems}) => true
  • OptionalcountLimit?: number

    The maximum amount of items that should be emitted.

    Default

    Infinity
  • Optionalbackoff?: number

    Milliseconds to wait before the next request is triggered.

    Default

    0
  • OptionalrequestLimit?: number

    The maximum amount of request that should be triggered. Retries on failure are not counted towards this limit.

    For example, it can be helpful during development to avoid an infinite number of requests.

    Default

    10000
  • OptionalstackAllItems?: boolean

    Defines how the property allItems in pagination.paginate, pagination.filter and pagination.shouldContinue is managed.

    By default, the property allItems is always an empty array. This setting can be helpful to save on memory usage when working with a large dataset.

    When set to true, the property allItems is an array of the emitted items.

    Default

    false

Settings

Member Visibility