ScaleBun

The Error With 500 Users Might Matter More Than the Error With 50,000

ObservabilityScaleBun5 min
ScaleBun blog cover for error-with-500-users-matters-moreScaleBun blog cover for error-with-500-users-matters-more

We had this problem recently while looking at production issues.

One error was happening constantly. Thousands of times. It looked terrible on the dashboard and naturally got everyone's attention.

Another error was much smaller.

A few hundred users.

The difference was that the smaller one happened during a critical flow.

People were getting all the way to the final step and then something failed. They tried again, got the same result, and left.

If you sort an error dashboard by occurrence count, you probably fix the first issue.

If you actually look at what users were doing when the errors happened, you fix the second one.

That's something I've started thinking about a lot with monitoring tools.

We measure technical severity really well.

Occurrences.

Affected users.

Stack traces.

Versions.

But an error doesn't exist in a vacuum.

An exception on a profile avatar and an exception on the Pay button shouldn't necessarily have the same priority just because they happened the same number of times.

Ideally I want to open an issue and see something like:

PaymentInitializationError

412 users hit it.

Most of them were inside checkout.

79% didn't complete the flow afterward.

It started becoming noticeable after the latest deployment.

And there are 180 sessions I can actually watch.

At that point the issue is no longer just an exception with a stack trace. I understand what it did to the product.

This applies to a browser just as much as it does to a mobile app.

On web it might be a JavaScript exception or a request that keeps returning 500.

On mobile it might be a crash.

I don't really care which runtime produced the signal when I'm deciding what the team should fix Monday morning.

I care about which broken thing is hurting people.

That's one of the reasons we're interested in keeping replay, errors and product events connected inside ScaleBun instead of making them three completely separate datasets.

Error monitoring is good at answering what broke.

I think it becomes much more useful when it can also answer:

Did anyone actually care that it broke?

See the full picture across web and mobile.

Connect application health, user behavior, attribution, and product context with ScaleBun.

Explore ScaleBun →