Poll and paginate

A search runs in the background and adds results as it finds them. This guide shows how to wait for a search and read everything it found, without missing a result or reading one twice.

You need the id of a search you started. See the quickstart. Reading status and results is free, so the code here spends nothing.

The two signals

Two fields answer two different questions. Confusing them is an easy way to miss results.

Field	Where	Tells you
`status` / `search_status`	Status response, and every results page	Whether the search is still working
`has_more`	Results page	Whether there are more stored results to read right now

has_more: false while the search is running means you have read everything found so far. The search is not finished, and more results may arrive.

Wait, then read everything

The simplest approach is to wait until the search reaches a , then read every page of results. Each page ends with a that fetches the next one.

import os, time, requests

API = "https://api.avina.io/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['AVINA_API_KEY']}"}
TERMINAL = {"completed", "failed", "cancelled"}

def wait_for_search(search_id: str, kind: str = "companies") -> dict:
    """Poll until the search finishes. Returns the final status."""
    delay = 2.0
    while True:
        response = requests.get(f"{API}/{kind}/search/{search_id}/status", headers=HEADERS)
        if response.status_code == 429:
            time.sleep(int(response.headers.get("Retry-After", "5")))
            continue
        response.raise_for_status()
        search = response.json()
        if search["status"] in TERMINAL:
            return search
        time.sleep(delay)
        delay = min(delay * 1.5, 10.0)

def read_all_results(search_id: str, kind: str = "companies") -> list[dict]:
    """Read every stored result, following cursors."""
    results, after = [], None
    while True:
        response = requests.get(
            f"{API}/{kind}/search/{search_id}/results",
            headers=HEADERS,
            params={"limit": 100, **({"after": after} if after else {})},
        )
        response.raise_for_status()
        page = response.json()
        results.extend(page["data"])
        after = page["next_cursor"]
        if not page["has_more"]:
            return results

search = wait_for_search(SEARCH_ID)
companies = read_all_results(SEARCH_ID)
print(search["status"], search["stop_reason"], len(companies))

const API = "https://api.avina.io/v1";
const headers = { Authorization: `Bearer ${process.env.AVINA_API_KEY}` };
const TERMINAL = new Set(["completed", "failed", "cancelled"]);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

/** Poll until the search finishes. Returns the final status. */
async function waitForSearch(searchId, kind = "companies") {
  let delay = 2000;
  for (;;) {
    const response = await fetch(`${API}/${kind}/search/${searchId}/status`, { headers });
    if (response.status === 429) {
      await sleep(Number(response.headers.get("Retry-After") ?? 5) * 1000);
      continue;
    }
    if (!response.ok) throw new Error(`Status failed: ${response.status}`);
    const search = await response.json();
    if (TERMINAL.has(search.status)) return search;
    await sleep(delay);
    delay = Math.min(delay * 1.5, 10000);
  }
}

/** Read every stored result, following cursors. */
async function readAllResults(searchId, kind = "companies") {
  const results = [];
  let after;
  for (;;) {
    const url = new URL(`${API}/${kind}/search/${searchId}/results`);
    url.searchParams.set("limit", "100");
    if (after) url.searchParams.set("after", after);
    const response = await fetch(url, { headers });
    if (!response.ok) throw new Error(`Results failed: ${response.status}`);
    const page = await response.json();
    results.push(...page.data);
    after = page.next_cursor;
    if (!page.has_more) return results;
  }
}

const search = await waitForSearch(SEARCH_ID);
const companies = await readAllResults(SEARCH_ID);
console.log(search.status, search.stop_reason, companies.length);

Pass "people" as kind for a people search. Everything else is the same.

Stream results as they arrive

To show results while the search is still running, read new results on each poll and remember the cursor.

Python

def stream_results(search_id: str, kind: str = "companies"):
    """Yield each result once, as soon as it is found."""
    after, delay = None, 2.0
    while True:
        response = requests.get(
            f"{API}/{kind}/search/{search_id}/results",
            headers=HEADERS,
            params={"limit": 100, **({"after": after} if after else {})},
        )
        if response.status_code == 429:
            time.sleep(int(response.headers.get("Retry-After", "5")))
            continue
        response.raise_for_status()
        page = response.json()
        yield from page["data"]
        after = page["next_cursor"]  # always returned, even on an empty page

        if page["has_more"]:
            continue  # more stored results: read them now, without waiting
        if page["search_status"] in TERMINAL:
            return    # caught up and the search has finished
        time.sleep(delay)
        delay = min(delay * 1.5, 10.0)

for company in stream_results(SEARCH_ID):
    print(company["name"])

This works because results behave predictably:

Results are only ever added. A result’s position never changes, and results are never removed.
next_cursor is always returned. Even an empty page gives you a cursor, so you can continue from exactly where you stopped.
Every page includes search_status, so you do not need a separate status call to know when to stop.

Pagination reference

Parameter	Default	Notes
`limit`	25	Page size, 1 to 100.
`after`	(none)	The `next_cursor` from the previous page. Omit on the first call.

Treat a cursor as an opaque value: store it and send it back unchanged. Do not parse it, construct one yourself or do arithmetic on it. Only use a cursor with the search it came from. A malformed cursor returns 400 invalid_request. You can store a cursor and come back later. Reading from a stored cursor returns everything found since, which makes it easy to resume after a restart.

Polling intervals

Wait 2 seconds before the first poll, then a little longer each time, up to 10 seconds.
Polling counts toward your rate limit of 120 requests per minute per key. One search polled at this pace uses a small part of it.
If you receive a 429, wait the number of seconds given in the Retry-After header before trying again.

Webhooks

Coming Soon. A webhook is a URL on your server that Avina calls when something happens. Webhooks that tell you when a search has finished are not available yet. Until they are, poll the search’s status as shown above.

Common mistakes

Stopping when has_more is false

Only stop when has_more is false and the search status is terminal. Otherwise you miss everything found after you caught up.

Starting from the beginning on every poll

Without after, each call returns the first page again. Always pass the last next_cursor.

Treating a client timeout as cancellation

If your process gives up waiting, the search keeps running and spending. Call cancel to stop it.

Discarding results from a failed search

A failed or cancelled search still has every result found before it stopped. Read them.

Search lifecycle: statuses and stop reasons
List company search results
List people search results

Get started

Concepts

Capabilities

Guides

The two signals

Wait, then read everything

Stream results as they arrive

Polling intervals

Webhooks

Common mistakes

​The two signals

​Wait, then read everything

​Stream results as they arrive

​Pagination reference

​Polling intervals

​Webhooks

​Common mistakes

​Related

The two signals

Wait, then read everything

Stream results as they arrive

Pagination reference

Polling intervals

Webhooks

Common mistakes

Related