Pagination

Many times, it is required to paginate the results that the server returns in order to retrieve extra older/newer items because the server mostly does not return how many items you want.

Pagination specification

The pagination collection is placed in the root level of Request Specification, along with url, method , body and response.

Below is the list of all pagination directives. They should be placed inside the pagination collection.

Key

Type

Description

Boolean

Specifies the condition if to execute pagination or not

IML String

Specifies the URL that should be called.

IML String

Specifies the HTTP method, that should be used when issuing a request.

IML Flat Object

A single level (flat) collection, that specifies request headers.

IML Flat Object

A single level (flat) collection that specifies request query string parameters.

Any IML Type

Specifies a request body.

IML String or Boolean

Determines if to execute current request or never.

mergeWithParent

This directive specifies if to merge pagination parameters with the original request parameters, or not. Default value is true, which means that all of your pagination request parameters will be merged with the original request parameters.

If the value is set to false, then no parameters will be merged, but directly used to make a pagination request.

url

This directive specifies the URL that will be called when the pagination request is executed. It will override the original request URL no matter the value of mergeWithParent.

method

This directive specifies the HTTP method to be used when executing the pagination request. It will override the original request method no matter the value of mergeWithParent.

headers

This directive specifies the request headers to be used when executing the pagination request. It will merge with headers of the original request, overriding headers with the same name when mergeWithParent is set to true(default).

qs

This directive specifies the request query string parameters to be used when executing the pagination request. It will merge with query string parameters of the original request, overriding ones with the same name when mergeWithParent is set to true (default).

body

This directive specifies the request body when the request method is anything but GET to be used when executing the pagination request. It will override the original request body no matter the value ofmergeWithParent.

condition

This directive specifies whether to execute the pagination request or not. It has all context variables available to it as response collection, such as body, headers, temp etc. Use this to stop paginating if the server is sending you info about the number of pages or items, or if the next page is available or not.

Limits

As we cannot paginate to infinity and beyond, there are four limits on the paginated requests.

Name

Total limit of ...

Value

Max Request Count

... all requests performed by a single module

100

Max Pagination Request Count

... pagination requests performed by a single module

50

Max Past Record

... paginated records

3200

Max Call Timeout

... seconds

40

Examples

Total pages

Common way of pagination is using total pages or pages count . Basically, the service gives you information about how many pages of content (the total count) are available and you have to iterate over those pages.

{
  "Data": [
    {...},
    {...},
    {...},
    {...},
    {...}
  ]
  "TotalItems": 42,
  "TotalPages": 10
}

This is a sample response from the service. As you can see, the service returns some items in the Data array, TotalItems, and TotalPages metadata. The TotalPages is the one we will use for setting up pagination in Make.

Some services index pages from 0, and some index from 1. The pagination.page directive indexes pages from 1. You'll need to check that to set up the pagination correctly.

Limit - offset

This is another common way how pagination is being implemented. You can set the paging cursor using two parameters:

  • limit - sets the maximum amount of records being retrieved per page

  • offset - sets the number of skipped records from the beginning of the data set

Using these two parameters, you're able to control the pagination.

[
    {...},
    {...},
    {...},
    {...},
    {...}
]

The service doesn't even have to return you any pagination info. Sometimes the service returns something like PageSize , but that's not important for us now.

The pagination will stop automatically after the response with 0 records or once the condition response.limit is met. If that behavior does not fit your needs, you can specify the additional condition.

Next Page Token

Next page token is basically a unique identifier of a next page, which is provided in the response. You can get the next page without counting some pages or offsets using this token only. Sometimes the "Next page URL" is provided instead of the token only, but the flow is quite similar.

{
    "data": [
        {...},
        {...},
        {...},
        {...},
        {...}
    ],
    "nextPageToken": "ABCDEFGHHGFEDCBA"
}

The service returns a nextPageToken for retrieving the next page.

Next Page URL

The next page URL is a link to the next page which is provided in the response.

If the next page URL is provided, you can map it directly in the pagination directive.

{
    "items": [
        {...},        },
        {...}
    ],
    "_links": {
        "next": {
            "href": "https://api.abc.com/locations?page=ZmFsc2V8aHR...."
        },
        "previous": null
    }
}

Has More Items

When using this method, you will get information on whether the data set contains more records or not.

{
    "data": [
        {...},
        {...},
        {...},
        {...},
        {...}
    ],
    "has_more_items": true
}

The service returns a has_more_items boolean which tells you if you can get more pages or not.

Page Only

This type of pagination isn't the best, but many services use this. Basically, you keep asking for the next page and when you get an empty response, you know you're done.

[
    {...},
    {...},
    {...},
    {...},
    {...}
]

No additional info about pagination is provided.

The pagination should stop after an empty response or empty data in the body. But some services think that it's a good practice to keep returning the last page after the maximum amount of pages has been reached. Always check the behavior of the service when using this type of pagination.

Debugging pagination

pageDebugging of pagination in list/search modules

Last updated