Knowledge · Guide
Do Not Let Support AI Apply a New Policy to an Old Order
A support assistant can retrieve the newest policy and still give the wrong answer. During a policy change, identify the version that applies to the customer before generating a response. A policy retired for new orders may remain relevant to older ones. The following cutover check can be rehearsed with fictional records before a team relies on it.
Do Not Let Support AI Apply a New Policy to an Old Order
Original AAHouse explanatory video. AI-assisted conceptual visuals and neural narration; original music.
English transcript · Simplified Chinese Mandarin transcript · Taiwan Mandarin transcript · Hong Kong Cantonese transcript · Macao Cantonese transcript
Separate the two clocks
The business clock determines whom a rule covers. The system clock determines when knowledge becomes available to the assistant. A document's latest modification date cannot substitute for its applicability conditions. A review date serves another purpose. Zendesk documents article-verification rules that prompt owners to review content according to selected conditions and intervals. Do not assume a review reminder also disables retrieval; check that control separately. Zendesk documentation
Explanatory diagram redrawn from the approved article.
Work through two fictional orders
Imagine a fictional retailer whose documented rules specify a 30-day return-request window for orders placed before November 1, 2026, at midnight in Hong Kong, and a 14-day window for orders placed from that instant onward. This example demonstrates internal information routing. It does not determine whether terms are enforceable or what rights a customer has. In a real business, the controlling event might be a contract date, an effective date, or a service date. The responsible owner must confirm it. Set the test clock to noon on November 20. Order A was placed at noon on October 25, 26 days earlier. It should retrieve the older 30-day rule. Order B was placed at noon on November 2, 18 days earlier. It should retrieve the newer 14-day rule. Use the same timezone throughout. Assuming every other condition is satisfied, A remains inside the illustrative window and B is outside it. This result checks policy selection, not authority to approve or reject a return. Giving both customers the newest paragraph would misapply the fictional rules to A. If the order timestamp is missing, consult an authorized order record or ask for the missing information. Do not guess the version. If applicability conditions conflict, route the question to the responsible owner instead of blending two policies into a third rule.
Explanatory diagram redrawn from the approved article.
Explanatory diagram redrawn from the approved article.
Explanatory diagram redrawn from the approved article.
Explanatory diagram redrawn from the approved article.
Select the version before selecting the passage
For each policy version, keep its identifier, version, owner, scope, applicability start and end, and timezone. An inclusive start and exclusive end is a useful interval convention: the exact cutover instant belongs to one interval, not both. Ensure every searchable chunk inherits the fields, rather than leaving them only on a document cover page. Filter by the confirmed applicability conditions before selecting relevant passages. Azure AI Search supports narrowing vector queries with filterable nonvector fields. The search capability does not define the business meaning of those fields; maintaining the rules remains the team's responsibility. Microsoft documentation Retiring an older version from the new-order path does not necessarily mean deleting it immediately. Historical orders may still need it. Manage its permitted audience and retention separately under the business's established rules. Record the selected version and the reason it applied with the answer, so another employee can reconstruct the decision.
Explanatory diagram redrawn from the approved article.
A scheduled switch is not proof of a completed cutover
Intercom supports scheduled availability for specified content types, but its documentation says scheduled changes are applied by a background job running roughly hourly, rather than at the exact selected minute. For a strict cutover, verify actual availability instead of relying on a screenshot of the schedule. Intercom documentation The practical question is whether the intended version can actually be retrieved for the intended case. If the transition is incomplete, hold that answer for review rather than letting a plausible policy fill the gap.
Explanatory diagram redrawn from the approved article.
Run this handover checklist
□ Test one second before the boundary and the exact boundary; expect neither overlap nor a gap. □ Check that an older order retrieves its applicable older version and a newer order retrieves its applicable newer version. □ Route missing dates, conflicting scope, and incomplete synchronization to human verification. □ Repeat the question in a new conversation and an existing conversation; check that a previous answer is not carried forward blindly. □ Record the expected version, actual version, answer timestamp, and reviewer. Start with one rule and fictional orders. A useful acceptance test is whether each answer can explain why its version applies, not whether someone uploaded the latest file. Original editorial analysis. Sources checked October 30, 2026. All examples are fictional. These are workflow recommendations, not reported customer results, legal advice, or a hands-on product test.
Explanatory diagram redrawn from the approved article.