MAKE YOUR SERVICE THE ANSWER.Discuss your service 
SAG TECHNOLOGY FRONTIER
We design how search, answers and generative AI understand brands, and explain the architecture behind SAG in clear technical terms.
SAG / ARCHITECTURE NOTES
From discovery and evidence to data boundaries, reliable execution and delivery—see how each engineering choice works.
14 articles · 1 / 2
This is the process of turning a sitemap’s address list into crawl candidates within an approved domain. A URL list is not an analysis result. Unless language-specific duplicates, excluded pages, and other hosts are cleaned up, the workload grows while comparable evidence remains scarce.
Manage document identity by distinguishing the original link from the address used for comparison. If citations with different fragments are counted as new sources each time, the number of evidence pages is inflated. Conversely, deleting important query parameters can merge different products.
These controls prevent user-provided addresses from directing requests to internal services or networks that are not allowed. Even an address that looks legitimate can reach an internal address after a redirect or name resolution. Opening network boundaries for analytical convenience undermines trust across the entire tenant.
A design that keeps processing resources predictable by limiting the permitted size, entries, and paths of compressed inputs. Even a small upload can require substantial memory after decompression. Using filenames directly as paths can also risk affecting files outside the analysis system.
A way to manage images and text/DOM as distinct evidence types. A screen may show a product description, but it does not show meta tags or canonical tags. Treating a screenshot as equivalent evidence to an HTML diagnosis leads to conclusions about technical items that have not been verified.
A data model that stores content from the same address as records tied to each collection time. If specifications or policies change while the address stays the same, the evidence behind an earlier report may differ from what is currently displayed. Storing only the latest content makes it difficult to reproduce past decisions.
A pattern for distinguishing unchanged and changed material using content hashes and structural comparisons. The more frequently pages are collected, the more collection runs accumulate. Sending inputs with no actual changes to an expensive model every time increases costs and can weaken consistency in the results.
This process extracts titles, body text, links, and metadata signals from different HTML documents into a common structure. On sites with long menus and footers, repeated sentences may be captured more often than product descriptions. If structure is not distinguished, sentence counts can be misread as indicating that there is sufficient evidence about the product.
A collection method that reads a set of webpage HTML files through a restricted input path and turns them into documents needed for diagnosis. When a public site has changed or external access is restricted, it is difficult to verify previous source material using only the current page. An approved source bundle is the starting point for establishing what was analyzed.
A pattern for narrowing the scope of reanalysis to changed pages and the questions they affect. Full recrawling is simple, but even small edits incur the full cost again. Conversely, judging changes too narrowly can miss inconsistencies in related FAQs and comparison tables.