[Instrumenting] labels size

plievana

Hello,
We are deploying an API. Our endpoints have the format: /api/v<version>/<path>
For every request, we can get "customer_id" from session.

We would like to monitor our api (latency, response_code, response_size):

• by customer and by path

For example, we are looking for customers with higher latencies, endpoints with higher latencies, ....

Our initial plan was to have metrics like:

• http_request_duration_seconds
• http_requests_total
• ...

and have labels for:

• account_id
• path

for example:

http_request_duration_seconds{account="1234", path="/api/v1/user", method="GET"}

We have more than 500 customers (and continue growing).

After reading chapter about instrumentation, I think it is not a good idea (point number four says:

Avoid labels with an unbounded number of values (like user IP addresses). Every label value creates a new time series that needs to be tracked

What metrics and labels do you suggest to have for our use case?

Thanks and regards,
Pedro.