The useful content on a lot of sites is not on a web page. It is in a PDF: the annual report, the product manual, the whitepaper, the price list.
The PDF call takes a URL that points to a PDF and returns its text. It goes through the same account, key and quota as everything else, so you do not need a second tool for the files linked from the pages you are already reading.
Typical flow
- Crawl or scrape a site and collect the links ending in
.pdf. - Send each PDF URL to the extract call.
- Feed the text into the same chunking and embedding step as your page content.
A limit worth knowing
Extraction reads the text layer in the PDF. A scan that is only a picture of text has no text layer, and needs OCR first. If most of your PDFs are scans, plan for that.
Why bother with it here
A one-off script is easy. A script that survives odd encodings, huge files and servers that serve PDFs with the wrong headers is not. Having it behind the same API keeps your pipeline to one integration, and an agent can reach it through the MCP server without custom code.
See LLM ingestion for how it fits a knowledge-base build, and pricing for what a page costs.
Finding the PDFs worth reading
Start by deciding which documents matter. A company site can link dozens of PDFs, from brochures and spec sheets to legal notices and old newsletters. Extracting all of them is rarely the goal. Collect the links first, using a crawl or a scrape of the pages that link them, then filter by name, by folder or by page before you send anything. Spec sheets, manuals and annual reports are usually worth the text. Duplicated price lists and expired promotions usually are not.
Cleaning the text before you use it
PDF text is not Markdown. The text layer comes out in reading order, but a document built for print has headers and footers on every page, page numbers, hyphenated line breaks and sometimes two columns that read across instead of down. Plan a cleanup step. Strip repeating headers and footers, join broken lines, and keep the page number if you want to cite a page later. If you are chunking for retrieval, split on headings or on paragraphs, not on a fixed number of characters, so a table row is not cut in half.
Scans, forms and other awkward files
Extraction reads text that is actually in the file. A scanned document is an image, and an image has no text to read, so you need OCR before the call can help. Forms may hold their values in fields that are not part of the page text. Password-protected files will not open. Very large files take longer, so treat them like any other long task and set a sensible timeout in your own code. When a result looks empty, open the file and check which of these you are looking at before you assume something is broken.
Keeping the source
Always store the PDF’s URL alongside the text you extract. When a model quotes a clause from a manual, the reader needs to be able to open the original, check the wording and see the date. Text without a source is hard to trust, and a PDF can change at the same address, so record when you read it too.
Rights and sensitivity
Read documents you are allowed to read. A public annual report is a fair target. A file that was uploaded by mistake, or that contains personal data, is not, and the acceptable use policy applies to PDFs the same way it applies to pages.