Pagination with API

Question

I'm currently building an opensource administration interface to help people easily manage their contens/websites.

Application built on Vue.js and heavily rely on json configurations for which fields to show or which data to send server via forms.

Since rest apis doesnt have popular standardization (besides jsonapi) is there any popular pagination solution for apis?

1. Currently i'm using headers like "total_count, page_index, limit" to paginate records my server-side testing app.
2. Another approach is "_links" property in json result, but i believe its designated to use navigate next-prev pages.

so, tldr; What is best or common way to handle pagination on rest api?

Answer 1

There has been quite a lot written on the subject of api pagination. I think it's fair to say that the naive approach 'get page 5, 20 items per page' doesn't work well with changeable data, filters, sorting etc.

ie,

• if I get 20 items and then apply a filter, i don't have 20 items
• if I request page 2 of sorted data then I also need to send what I'm sorting by
• if a new record is added while I am paging through, then every page after that record is changed. I'm going to miss (for deletion) or see repeat records as I page through.

Pagination is a function of the total dataset, and yet the reason we are doing pagination is so that we don't have to get the whole dataset at once.

If you want a solution that works perfectly then you need to have the server side remember the total dataset for your particular query and then return pages from it as requested. You then send a request for 'page 5 of query 1234'

Which can eat a lot of memory.

There isn't really a good solution that I've seen for dynamic data. I would opt for either:

• a full download and let the client do all the pagination
• a full server side query with filters and sorting, cache results and return by page.
• a streamed/infini-scroll download where the client just gets the next page of data continualy

The user experience for someone paging through data, can be quite jarring when they see 19 of the expected 20 items, or a repeated item from the last page. It makes them think the application is broken and should be avoided.

Answer 2

Paging in multi-user environments where data can change between requests for pages is tricky at best, pointless at worst.

Say your first request got you a page of 10 items out of a total of 100. Now you do another request for the second page of 10 items, but the number of items has changed to 101, with the new item existing in what would be the first page.

Now what do you return? The request was no doubt intended for 10 records not yet retrieved, but should that new record be returned or not? In other words, should we return records 12-21, 12-20 + the new record, 10 records starting at the new record, records 11-20, or what?

Applications like Twitter handle it by just returning the X most recent data points and allowing users to retrieve more at either end as desired, but their data is structured such that no data will fit within an interval already retrieved (or if it does, like when following someone new who's posted in that interval) they simply ignore it.

If your application can retrieve data based on custom filters and ordering sequences this is of course not an option, and the problems I described start rearing their ugly head. In those cases, a hybrid solution where you select a broad chunk of data using only filters that won't cause a situation where new data can enter the chunk, then filtering and sorting further on the client is often the best solution.

Overall then, there is no best way. It depends greatly on your data, your presentation requirements for it, and the acceptable network load (returning larger data volumes of course means higher network load) and security requirements (in some domains, returning more than the absolute minimum data to the client is considered a security risk).

Answer 3

If your URI for accessing orders is /api/orders then I would recommend that page 1 of orders be /api/orders/page/1.

Then for convenience, if you happen to use the same URI for put, post, and delete requests, you could make it work as if the URI were actually /api/orders (where applicable of course).

I would argue against allowing the user to set the number of items per page (simple is better in my humble opinion), however, if that might be something you wish to do, you could add an optional parameter for setting items per page, so:

/api/orders/page/5?itemsPerPage=20

Answer 4

What is best or common way to handle pagination on rest api?

RFC 5005 describes link relations for communicating the semantics of paging links to generic clients. It looks about as you expect - each relation includes an identifier of the resource of the appropriate page.

It also specifically introduces a distinction between paged feeds (which are lossy) and archived feeds (which are not lossy). That gives clients a way to choose the appropriate strategy for their use case.

As you noticed, that doesn't solve the problem of choosing the correct identifiers to use. A good approach is described in Working With Timelines.

You could use headers, I suppose, but this is the sort of thing I would expect to see in the effective request identifier; each "page" is a different resource, and the meta data allows you to control the caching rules for the different pages from the origin.