This could pose interesting problem in the field. If there is a deployment where multiple functions are used by large number of connections, there could be memory shortage and other resources shortage. This leads to error being returned when the resource is being allocated. If error detection, propagation and recovery is not taken care well by the software, this could lead to instability, leaks, crashes and lockups. It is tester job to ensure that these kinds of problems do not happen in the field. Typically testers simulate different conditions and ensure that system is stable. Yet times, it is not possible for testers to test all different combinations or simulate different conditions.
I believe testers should be able to simulate all possible combinations by simulating all kinds of error conditions. As part of it, testers should ask development team to provide facilities to inject the faults. In particular, testers should ask for facilities to inject faults for following.
- Memory allocation failures: Almost all functions in software would allocate memory either at the time of connection establishment or on packet basis or to queue the packets and control data etc... Testers should have ammunition to inject the memory fault for specific functions.
- Socket/File open failures
- Semaphore creation failures
- Thread/Tasklet creation failures
- Fault simulation of any other OS resource that gets allocated after software is completely initialized.
- Keep list of all functions and OS resource allocations they do.
- For each one of them, create a test case.
- Before running the test case, configure to inject fault.
- Run the test and ensure that system works as expected.
- Run the test without fault and ensure that system is stable.