Your AI Feature Needs a Fallback That Is Not an Apology

I started at the apology, because that is where the symptom landed, and walked it backwards link by link until the sorry stopped looking innocent.

This is the thesis: an AI feature needs a fallback that does something useful, and the apology message does not do something useful, so the apology is not a fallback, it is an outage with better copy.

This is an opinion, and the constraint that makes it hard is real, and the rule of thumb at the end is where I land.

Why the apology is the default

The apology is the default because it is the cheapest thing to build, and the cheapest is what ships first, so the message becomes the fallback and the work stops there. The reasoning is that the model is the feature, and the reasoning is the error, because the model is the enhancement on a thing that already existed, and the thing that existed is the search, the list, the manual flow, the human, and those are the fallback, available and unused.

The apology feels honest, and the honesty is the appeal, because the message tells the truth about the model being down. The truth is not the fallback, because the truth does not help the user, and the not helping is the outage.

Why the fallback is the feature's job

The feature has a job, and the job is the user's task, and the task is the reason the feature exists, and the exists is the obligation, and the obligation is the job's completion, and the completion is the user's outcome, and the outcome is the point, and the point is not the model, the point is the outcome, and the outcome can be reached without the model, and the without is the fallback, and the fallback is the obligation's fulfilment, and the fulfilment is the feature's job.

The model is one path to the outcome, and the path is the fast one, and the fast is the enhancement, and the enhancement is the improvement, and the improvement is not the only path, and the not only is the fallback, and the fallback is the slow path, and the slow path reaches the outcome, and the reached outcome is the job done, and the done is the difference between the fallback and the apology, because the fallback finishes the task and the apology explains why it cannot.

This is the same distinction as the rollback where the CDN kept serving the old version, which is why the two fixes look alike.

The constraint that makes it hard

The honest constraint is that some features have no useful non-model path, because the feature is the model, and the model is the capability, and the capability did not exist before, so there is no history to fall back to, and the no history is the real gap, and the gap is the reason the apology exists, and the exists is the legitimate case.

The legitimate case is narrower than it looks, because the capability usually has a partial version, and the partial is the keyword search instead of the semantic search, the template instead of the generated text, the cached answer instead of the fresh one, the human queue instead of the instant reply, and the partials are the fallbacks, and the fallbacks are the degraded service, and the degraded is better than the absent, and the better is the argument, and the argument is the rule.

Where the partial genuinely does not exist, the apology is correct, and the correct apology is the honest one, and the honest is the rare case, and the rare is the exception, and the exception should be the exception rather than the default, and the default should be the partial, and the partial is the work, and the work is the fallback.

The thumb rule I use

The fallback should complete the user's task at reduced quality, and the reduced quality is the degradation, and the degradation is the contract, and the contract is the user's expectation, and the expectation is the honest one, because the user gets a worse answer rather than no answer, and the worse answer is the service, and the service is the feature still working.

Design the fallback before the model path. The fallback is the baseline, and the baseline is the feature without the enhancement, and the without is the shipping version, and the version is the fallback, and the fallback is the design, and the design is the order, because the fallback designed after is the apology, and the apology is the default.

Make the degradation visible. The user is told the feature is degraded, and the told is the honesty, and the honesty is the trust, and the trust is the reason the degraded answer is accepted, and the accepted is the fallback's success, and the success is the task completed with the caveat.

Cache the good answers. The cache is the fallback's supply, and the supply is the previous model's output, and the output is the answer, and the answer is the served response, and the served is the continuity, and the continuity is the fallback, because the cached answer is better than the apology, and the better is the fix.

Route to the human when the machine cannot. The queue is the fallback, and the queue is the slow path, and the slow path is the outcome, and the outcome is the job, and the job is the feature's obligation, and the obligation is met by the human, and the human is the fallback, and the fallback is the service.

Where I land

The apology is the correct response when no degraded path exists, and that case is rare, and the rare should stay rare. For everything else the fallback is the feature's obligation, because the user's task is the point and the model is one path to it, and the path's failure is not the task's cancellation.

The model went down and the feature apologised, and the apology was the outage with better copy, and the copy was the only improvement. Name that one property in review and this class of bug gets hard to ship.

Published on debugly.dev by Rohit Bhadani | Founder of Krova Cloud