Symptoms

The following error message occurs during the processing:

"Assertion Failed
at FlexiCaptureProcessorsPool.getProcessor() …
at FlexiCaptureProcessorsPool.GetProcessor() …
at WorkerThread.doWork() …
at ThreadHelper.ThreadStart_Context(Object state)
at ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state.Boolean ignoreSyncCtx)
at ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at ThreadHelper.ThreadStart()"

Cause

The reported message is a timeout: one of the threads waited for a free instance of FlexiCaptureProcessor for more than the specified limit of 60 seconds. It is the expected behavior for the sample in some circumstances. This assertion is to warn about the following situation:

The pool is designed for a "server scenario" - a constant inflow of requests, where the requirement is to have the number of waiting threads not choke the expected constant throughput of the system so that any given request is served in a timely manner (to meet QoS requirements). The throughput is limited by the number of CPU cores and the average time it takes to process a single work item. If the input exceeds throughput the work begins to pile up, which is not good for a server. The requirement above is met when the number of working threads exactly matches the number of FlexiCaptureProcessor, so any working thread can have an instance of FlexiCaptureProcessor of its own at any time. If this number is greater than the actual number of CPU cores, the threads will wait together, but the optimal number of threads is somewhere close to the number of CPU cores.

Resolution

Unfortunately, it is not possible to create 400 threads (i.e. work items) and expect them to be processed in parallel by a pool of 8 processors with a timeout of 60 seconds. We recommend the following:

1. Remove the timeout: change 60000 /*ms*/ to -1 (infinity) in SampleProcessorPool. This should help but the general architecture of the solution will be far from ideal: creating 400 threads to process 400 work items in parallel is not advised
2. A more suitable approach would be to
   
   1. Create a queue of work items (paths to directories in this case)
   2. Fill it with the 400 items
   3. Create N threads (N ~ number of CPU cores + 1) so that the threads pull work items from the queue one by one
   4. Process the items with N FlexiCaptureProcessors.

This way the number of working threads will never exceed the actual machine's capabilities and the work will should be done faster because the creation and management of 400 threads is a considerable strain on the system.