Infusion.Runtime.Paginate (infusion_runtime v0.1.0)

Copy Markdown View Source

Turns a list operation into a lazy stream of every item, fetching pages as they are needed.

{:ok, repos} = GitHub.Repos.list_for_org_all("elixir-lang", per_page: 100)
repos |> Stream.map(& &1.name) |> Enum.take(250)   # fetches 3 pages

The operation's paginate map says how the API pages:

  • style: :cursor_after, param: "after": while has_more, ask for the items after the last one's id (OpenAI, Stripe). Paging backwards (before, ending_before) works the same way from the first item.
  • style: :cursor, param: "cursor", from: ["result_info", "cursor"]: send back the cursor the response holds (Cloudflare, Stripe search).
  • style: :next_url, from: ["next_page_uri"]: follow the URL the response holds (Twilio).
  • style: :page, param: "page": follow the Link header when there is one (GitHub), else ask for the next page number until a page comes back empty, short, or past total_pages.
  • style: :offset, param: "offset", limit: "limit"
  • style: :link_header: follow rel="next" in the Link header.

plus items (where the list is in the response, [] for the response itself) and item (the item type).

The first page is fetched right away, so its errors come back as {:error, error}. A later page that fails raises Infusion.Runtime.Error (after the usual retries). Stopping early fetches no more pages.

Summary

Functions

stream(config, operation, path_args, params, opts \\ [])

Runs operation and returns {:ok, stream} of all its items.

Functions

stream(config, operation, path_args, params, opts \\ [])

@spec stream(
  Infusion.Runtime.Config.t(),
  map(),
  [term()],
  map() | keyword(),
  keyword()
) ::
  {:ok, Enumerable.t()} | {:error, Infusion.Runtime.Error.t()}

Runs operation and returns {:ok, stream} of all its items.