diff --git a/guide/performance_testing.md b/guide/performance_testing.md new file mode 100644 index 0000000000000000000000000000000000000000..945adc2649502d223887e6e2e6f459c21c4fd2a5 --- /dev/null +++ b/guide/performance_testing.md @@ -0,0 +1,33 @@ +## Notes on load testing + +### Checklist + +* [ ] I have created a test-plan with a hypothesis and documented my method so I can share it with the project team. +* [ ] I'm using a performance testing tool such as jMeter, LoadRunner or Gattling +* [ ] My environment is hosted in an isolated and repeatable environment +* [ ] I have extended or removed memory limits / quotas +* [ ] I have created my own function and understand the difference between AfterBurn and the default watchdog mode. +* [ ] I have turned off `write_debug` and `read_debug` so that the logs for the function are kept sparse +* [ ] I am monitoring / collecting logs from the core services and function under test +* [ ] I am monitoring the system for feedback through Prometheus and / or Grafana - i.e. throughput and 200/500 errors + +### Common mistakes for performance-testing a project: + +* Using an inappropriate method + +There is a differnce between performance testing and Denial of Service DoS attacks. Use tools which allow a gradual ramp-up and controlled conditions such as jMeter, LoadRunner or Gattling. + +* Choosing an inappropriate test environment + +Do not try to performance test OpenFaaS on your laptop within a VM - this carries an overhead of virtualisation and will likely cause contention. + +The test environment needs to replicate the production environment you are likely to use. Take note that most AWS virtual machines are subject to CPU throttling and a credits system which will make performance testing hard and unscientific. + +* Poor choice of test function + +There are several sample functions provided in the project, but that does not automatically qualify them for benchmarking or load-testing. It's important to create your own function and understand exactly what is being put into it so you can measure it efficiently. + +* Ignoring CPU / memory limits + +OpenFaaS enforces memory limits on core services. If you are going to perform a high load test you will want to extend these beyond the defaults or remove them completely. +