Skip to main content
Thoth

Thoth API Usage Policy

Purpose

Thoth is community-led, not-for-profit open infrastructure. Our APIs exist so that anyone can find, reuse, and build on open metadata, and they will always be open to everyone.

We are delighted to see the use of our APIs growing. But more use also means more load on infrastructure the community pays for and more work to keep the 99.98% uptime that publishers and other users rely on.

This policy keeps that shared resource fair and sustainable. Exploring, querying, and harvesting the data is free and needs no permission, but if your service relies on our infrastructure day in, day out, we ask you to help keep it running.

Scope

This policy covers Thoth's public APIs: the GraphQL API (api.thoth.pub), the Export API (export.thoth.pub) and any other public data endpoint we operate.

It does not cover:

  • Publishers and their staff managing their own records through the Thoth app or authenticated API calls.
  • Services that Thoth runs for an organisation under a separate agreement, such as Thoth Obelisk, Sphinx or Pyramid. Usage generated by those services is governed by that agreement.
  • Data you have already downloaded. What you do with it is governed by its licence (all Thoth metadata is CC0), not by this policy.

Open data, shared infrastructure

The metadata is open; our servers are a shared resource. Metadata published through Thoth is released under CC0, and nothing in this policy restricts what you do with it once you have it. You may copy, mirror, index, redistribute and build services on it, commercially or otherwise.

This policy governs access to Thoth's live services (how often and how heavily you call them), not the data itself. If you need the data at volume, keeping a local copy that you refresh periodically is usually cheaper for both of us than repeated live queries.

Standard use

Anyone may use the APIs free of charge and without registering for standard use, such as:

  • Scripts and one-off jobs that look something up or pull a dataset.
  • Research, teaching, data analysis, and reporting.
  • Prototypes, tests, and proofs of concept.
  • Integrations that sync with Thoth occasionally, for example a nightly incremental update, and follow the good practice below.

Standard use must not degrade the service for other users.

Sustained and high-volume use

Use is sustained or high-volume when it places a regular, significant load on our infrastructure, whatever its purpose. Typical signs are:

  • A website, app, dashboard, or other service that queries Thoth live in response to its own users' traffic, rather than serving from its own copy or cache.
  • Polling at short intervals, or repeatedly re-harvesting whole catalogues instead of fetching only what has changed.
  • Regular traffic from one organisation above roughly 5,000 requests per day or 120 requests per minute, or queries that are consistently large or expensive to resolve.

If your use is sustained or high-volume, please contact us to arrange Sustaining Access (see below). We will also contact organisations whose usage we identify as falling into this category.

Good practice

Everyone using the APIs, at every access level, should recognise that this is a shared community resource and behave respectfully. Specifically we expect users to:

  1. Identify themselves. Send a User-Agent header naming your application and giving a contact email, e.g. MyCatalogue/1.0 (mailto:tech@example.org), so we can reach you before acting on your traffic.
  2. Cache responses and serve from your own copy where possible. Book metadata changes infrequently.
  3. Fetch incrementally. After an initial load, request only records updated since your last sync rather than downloading everything again.
  4. Ask only for what you need. Request only the fields you use, page through large result sets at reasonable page sizes, and avoid deeply nested queries that pull in large parts of the database.
  5. Use the Export API when you need records in a standard format such as ONIX or MARC, rather than rebuilding them from many GraphQL queries.
  6. Pace your requests. Avoid large parallel bursts, back off when you receive errors or a 429 (Too Many Requests) response, and honour any Retry-After header.
  7. Not circumvent limits by spreading traffic across multiple IP addresses, keys, or accounts.

Access levels

There are three access levels; most users need nothing beyond the first two.

LevelWho it's forHow to get itCostWhat you get
StandardAnyone: scripts, research, occasional integrationsNothing to doFreeShared capacity on a best-effort basis
IdentifiedRegular automated usersSend a User-Agent with a contact email (and, when available, register for an API key)FreePriority over anonymous traffic when we need to limit load; notice before we restrict your access; notice of breaking changes
SustainingProduction services with sustained or high-volume useAgreement with ThothAnnual contributionHigher capacity, priority support, advance notice of changes, and service levels set out in the agreement

Organisations with an existing agreement with Thoth (for example members, hosting clients or infrastructure partners) may already have API usage covered by that agreement; we will confirm this with you.

Contributing to costs

Sustaining Access is an annual contribution towards running and developing the APIs, scaled to the load you place on them. It is a contribution to shared infrastructure, not a licence fee: it buys capacity and support, not rights to the data, which stays open to everyone.

We may reduce or waive contributions for scholar-led and small not-for-profit presses, libraries, research projects with limited funding, and organisations in low- and middle-income countries. Please ask.

Limits and our rights

To keep the service available to everyone, we may at our discretion:

  • Limit the rate, size, or complexity of requests, for all users or for particular clients.
  • Throttle, suspend or block traffic that degrades the service, does not follow this policy, or cannot be attributed to a contactable user.
  • Ask organisations making sustained or high-volume use to move to Sustaining Access, and restrict their access if they do not.

Where we can identify you, we will contact you and give you 15 days' notice before restricting your access for usage reasons, unless urgent action is needed to protect the service.

You must not use the APIs to attempt unauthorised access, to probe or attack our systems, or to load-test them without our agreement.

Availability and changes

Standard and Identified access is provided as is, with no formal service levels or dedicated support. Service levels for Sustaining access are set out in the relevant agreement.

We may change the APIs, their limits and this policy as the service evolves.

Contact

For questions about this policy, Sustaining access or contribution waivers, email info@thoth.pub. To report a problem with the APIs, email support@thoth.pub.