What should I do when receiving a message:
at FlexiCaptureProcessorsPool.getProcessor() …
at FlexiCaptureProcessorsPool.GetProcessor() …
at WorkerThread.doWork() …
at ThreadHelper.ThreadStart_Context(Object state)
at ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state.Boolean ignoreSyncCtx)
at ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
The reported message is actually a timeout: one of the threads waited for a free instance of FlexiCaptureProcessor for more than the specified limit of 60 seconds. It is the expected behavior for the sample in the circumstances. This assert is to warn about the following situation.
The pool is designed for a "server scenario" when there is a constant inflow of requests, and we want to ensure that the number of waiting threads does not choke the expected constant throughput of the system so that any given request is served in a timely manner (to meet QoS requirements). The throughput is limited by the number of CPU cores and the average time it takes to process a single work item. If the input exceeds throughput the work begins to pile up, which is not good for a server. The requirement above is met when the number of working threads exactly matches the number of FlexiCaptureProcessor, so any working thread can have an instance of FlexiCaptureProcessor of its own at any time. If this number is greater than the actual number of CPU cores, the threads will wait together, but the optimal number of threads is somewhere close to the number of CPU cores.
To create 400 threads (i.e. work items) and expect them to be processed in parallel by a pool of 8 processors with a timeout of 60 seconds is impossible. What we can recommend:
- Remove the timeout: change 60000 /*ms*/ to -1 (infinity) in SampleProcessorPool. This will help but the general architecture of the solution will be far from ideal: creating 400 threads to process 400 work items in parallel is not a very good idea
- A better approach would be to create a queue of work items (paths to directories in your case), fill it with the 400 items, create N threads (N ~ number of CPU cores + 1) so that the threads pull work items from the queue one by one and process the items with N FlexiCaptureProcessors. Thus the number of working threads will never exceed the actual machine capabilities and the work will actually be done faster because the creation and management of 400 threads is a considerable strain on the system.