What are acceptable failure rates in load testing scenarios

I am doing a report on load testing for a client.

I am using a service called blitz IO.

He would like to handle 1000 concurrent users at 500ms.

Every time I have run a test like this there it has never come back 100%, invariably if i run the test for 10 minutes a few of the connections out of the thousands timeout, and some go over the 500ms threshold.

How do you manage client expectations, and what sort of thresholds do you set.

Do you say that 90% of connections do not timeout and 90% of connections are within 500ms?

Interested to hear how manage this.

Answer

If he wants to handle 1000 concurrent users at 500ms, then I would expect a zero error rate. However, it is common that the specification allow for some slow requests. I am used to specification like "95% of the requests will be completed in with 500ms with 1000 concurrent users". Dropping connections is generally not acceptable.