Is Your Site's AI Feature Helping or Getting in the Way?

A developer I know shipped an AI assistant on a checkout page. Conversion dropped four percent in a week. The feature worked exactly as designed. It popped open when the page loaded, offered to help, and waited. Nobody had asked it to. People were trying to pay, and a chat panel kept sliding over the button they were reaching for.

The feature was not broken. It was just in the way. That distinction matters, because most teams audit AI features for whether they function, not for whether they respect the person trying to get something done.

Ben Friedman argued that most people do not actually want more AI in their lives. I read that less as anti-AI and more as a design brief. The AI should sit second, behind the task the user came to finish. When you frame it that way, "is this feature good" stops being a philosophical question and becomes a heuristic audit you can run on your own site this afternoon.

What actually breaks when an AI feature gets in the way?

The same three Nielsen heuristics, almost every time.

User control and freedom goes first. The AI opens a state the user did not choose, and now they have to spend effort closing it. Match between the system and the real world goes second, because the AI assumes a task the user was not doing. And flexibility and efficiency of use goes third, since the feature adds steps to a flow that already worked.

Notice what these have in common. None of them is about whether the model gives good answers. You can have a genuinely smart AI feature that fails all three, because the failure is about placement and control, not intelligence. This is why a Lighthouse score tells you nothing here. The page can load in 1.2 seconds and still shove a mode in front of someone who wanted a quiet form.

So the audit is not "is the AI accurate." The audit is "does the AI defer." Three questions get you most of the way there.

Where does the AI sit in your task flow?

Map your primary task first, then find where the AI intercepts it.

Take the top one or two things people come to your page to do: buy, book, read, search, sign up. Write out the shortest click path to completion. Now mark where the AI feature appears on that path. There are only two possibilities, and they are not equal.

If the AI sits beside the path, the user can complete the task without ever touching it: a "summarise this" button next to an article, a suggested-reply chip the user can ignore. That is the safe position.

If the AI sits on the path, the user cannot finish without passing through it: an AI step wedged between "add to cart" and "pay," a search box that only accepts natural-language questions and no longer takes a plain keyword. That is the dangerous position, and it is where almost every harmful AI feature lives.

Here is a concrete threshold. Count the taps or clicks to complete the primary task with the AI feature present, then count them with the feature hidden. If the number goes up when the AI is on screen, the AI is on your critical path and it is taxing every user, including the ones who never wanted it. That delta should be zero. A helper that costs the task nothing is a helper. A helper that adds a step is a tollbooth.

Can the user dismiss it in one action, and does it stay dismissed?

Dismissibility is the cleanest single test of whether you respect user control.

Try to close the feature. Count the actions it takes. One deliberate action, a single tap on a clearly labelled control, is the pass mark. If closing it requires hunting for a faint x, or a two-step "are you sure," or it has no close affordance at all, the feature is holding the user hostage to a mode they never entered on purpose.

Then reload the page and watch what happens. This is the part teams forget. A feature that reopens itself on every visit is not dismissible, it is merely pausable. If I close your AI panel and it greets me again tomorrow, and the day after, you have taught me that my choice does not persist. Respecting control means the dismissal sticks. Store the preference. A returning user who closed it once should not have to close it every session.

One more check for the attention it demands while open. Does the feature auto-expand, animate, or fire a notification badge on load? Motion and badges are attention claims. A background helper does not wave its arms when you enter the room. If your AI feature announces itself before the user has done anything, it has decided its own importance is higher than the task, and it is almost always wrong about that.

Does it do the boring work silently, or does it demand attention?

The best AI features remove work without asking to be noticed.

Think about what the feature is actually for. There are two kinds. The first quietly does something tedious the user would rather not do at all: auto-tagging an upload, pre-filling a form field from context, deduplicating a list, cleaning up pasted text. The user feels the result, not the mechanism. The second kind demands a conversation. It opens a panel, asks what you want, waits for a prompt, and makes the user do the work of directing it.

Neither is wrong in the abstract, but the second one has to earn its place, and most do not. If your AI feature requires the user to stop, switch into "talk to the AI" mode, phrase a request, and read a response, you have added a mode. Modes are expensive. The user has to remember they are in one, and remember how to leave. A feature that silently does the boring work adds no mode at all, which is exactly why people love it and never mention it.

A quick way to tell which you built. Ask whether the feature would still deliver value if it had no visible interface. If auto-tagging works invisibly, good, that is a silent helper. If the whole feature is the chat box, you have built something that needs the user's attention to function, and attention is the one resource your users came in with the least of.

How do you demote an AI-first feature back to a quiet helper?

Move it off the path, make it opt-in, and let it work in the background by default.

If your audit turned up an AI feature that is on the critical path, hard to dismiss, and attention-hungry, you do not have to delete it. You demote it. Specifically:

• Take it off the primary flow. The core task must be completable start to finish with the AI feature untouched. If cart-to-pay works without ever opening the assistant, you have passed.
• Make appearance a user choice, not a default. Replace auto-open with a calm, persistent entry point the user can reach when they want it. A labelled button beats an uninvited panel every time.
• Push the useful part into the background. Whatever the feature does that is genuinely boring and mechanical, run it silently and surface only the result. Keep the conversational layer as an option for the minority who want to direct it.
• Persist the dismissal. One close should mean closed, this session and the next.

This is the "AI-second" idea turned into edits you can make in a sprint. The user's task comes first. The AI sits behind it, quiet, reliable, doing the dull parts, available when summoned and invisible when not.

I run a version of this audit inside theuxbites, because these failures show up in the heuristic pass long before anyone files a complaint. The pattern is always the same. The metrics look fine, the model answers well, and the feature is still quietly costing you conversions because it forgot to defer. The fix is rarely a better model. It is a better position.

The honest test is uncomfortable but simple. Turn the AI feature off for a day and watch whether anything gets worse. If the task still gets done and nobody misses it, you did not build a helper. You built an interruption with good branding. And the users already knew.