SAG / ARCHITECTURE NOTE
URL Identity and Deduplication: Does Removing the Hash Make It the Same Page?
Manage document identity by distinguishing the original link from the address used for comparison. If citations with different fragments are counted as new sources each time, the number of evidence pages is inflated. Conversely, deleting important query parameters can merge different products.
What Is URL Identity?
It is a way to manage document identity by distinguishing the original link from the address used for comparison. This note treats URL identity in terms of the responsibilities of inputs, transformations, and outputs, rather than as the name of a feature. To trust an analysis, it must be possible to trace what material was included, what was checked, and how far the conclusions can go.
Why Is This Technology Needed?
If citations with different fragments are counted as new sources each time, the number of evidence pages is inflated. Conversely, deleting important query parameters can merge different products.
Design Principles and Data Flow
Set normalization rules according to their purpose. Use hash removal from citation URLs and deduplication for source aggregation, while keeping the original URL traceable.
Original URL → Normalization by purpose → Deduplicated source aggregation
Each step must not rebrand the success of the previous step as the achievement of the next. Keeping the material's identifier, time period, and verification status linked in the record helps locate omissions and errors and determine what needs to be checked again.
Connection to the SAG Architecture
SAG citation reports validate safe URLs and do not count repeated instances of the same source more than once based on hash differences. This rule is not extended to removing the meaning of all search URLs.
SAG's operational value lies in connecting this relationship to pages and questions, comparison results, and improvement tasks. Rather than reading only numbers, customers can review both what needs to be improved and the basis for the assessment. Patterns that require additional application should be interpreted according to the scope of the relevant paragraph.
Illustrative Example and Criteria for Assessment
For illustration, if product#spec and product#service are both cited in one answer, the number of cited answers for the same page is one. Different URLs such as product?id=1 and product?id=2 cannot be merged in the same way.
The example above is provided to explain the structure and calculation; it is not a measured result for a particular customer. In an actual report, the selected period, scope, observation conditions, and original records must be linked so that the same assessment can be checked again.
Practical Verification Checklist
| Flow stage | What to check |
|---|---|
| Original URL | Check for fragment duplicates |
| Normalization by purpose | Preserve meaningful query parameters |
| Deduplicated source aggregation | Whether the original link can be restored |
Check that the same meaning is preserved not only for normal inputs, but also for missing material, duplicate material, and material with different conditions. Linking verification items to completion criteria can reduce the gap between feature descriptions and actual operations.
Limitations and Points to Consider When Applying
Canonical hints, actual content identity, and normalization for aggregation are separate judgments. Deduplication rules must be managed by version to keep comparison history stable.
Research and Official Documentation
- IETF HTTP Semantics RFC 9110 — A standard for understanding the meaning of HTTP requests, responses, and statuses.
External sources provide background on the design topic above; they do not certify every SAG implementation or customer result. The interpretation of this note's application and its illustrative examples are organized according to SAG's operational structure. Material checked: 2026-10-06.
Further Reading and Feature Details
- Related architecture note
- Try a service connected to URL identity
- Feature-specific FAQ
- Discuss implementation scope
How to Continue Reading About This Technology
Follow the path: HTML ZIP·sitemap → normalization → page version → evidence record.
SAG / KNOWLEDGE LINKS