The very concept of ABBYY Cloud OCR SDK service implies that there are many applications using it simultaneously, uploading new tasks and receiving the processing results in the same time. And each application’s tasks have to be completed within an acceptable period of time.

How is the processing power managed to ensure good processing speed for all applications? This article describes the scalability of ABBYY Cloud OCR SDK.

Cloud OCR SDK and Windows Azure

ABBYY Cloud OCR SDK service is powered by Microsoft Windows Azure, which provides a reliable way to scale the processing power to match the workload. Here are some of the technical details:

• Uploaded images/documents are stored as BLOBs — the data center can accept high loads and will securely store the distributed data.
• The submitted tasks are managed in the scalable, redundant Azure SQL database service.
• OCR processing of the tasks is executed on dedicated “Azure worker roles”, a lot of which can be started and used at once.

For more on Windows Azure scalability, see their website.

Cloud OCR SDK can process millions of pages per day

The processing capacity of the service is virtually unlimited. Several high-load experiments conducted by ABBYY (and some of our clients) failed to reach the technical limit of processing throughput.

Processing queue

All tasks submitted to the service are queued and then passed for processing according to the priority of the application. If the processing queue is getting longer, new instances are started to deal with growing workload – that is, the service is scaled up.

In general, the service aims to keep the average workload of one running instance within certain limits; new instances are started if for a certain period of time the average workload is too high. The average incoming volume is also taken into account; if it keeps growing, the service will not be scaled down.

As most applications submit new tasks at more or less random moments, the number of pages to be processed increases/decreases gradually, and the service capacity changes are unnoticed by clients. But ABBYY also monitors the trends in workload and can pre-scale the service if the amount of work is expected to increase, for example at the most popular time of day.

Dealing with large volumes

The scalable Cloud service can be used to the greatest advantage in the cases of one-time processing of a large volume of documents, e.g. converting the company's archive into digital format. To do this in-house large hardware expenses would be necessary, and the purchased servers would not be fully used once conversion is over. On the other hand, using Cloud OCR SDK service you would only need to pay for the actual amount of pages processed.

But is a large batch of documents processed quickly enough and does it interfere with the response time for other applications, processing smaller amounts?

Batch processing vs. Random moment uploads

Our service’s scalability logic is based on the following assumption: if small volumes are uploaded at random moments, they need a fast response; but if a large number of documents is processed, the overall time for the batch is more important than the response time for an individual page.

When one application uploads a large batch for processing, the processing queue grows, and the service begins to scale up. During the short period of time necessary for this, the pages of this batch will be processed quite slowly. But once scaling-up is completed, the processing reaches the normal speed. The total processing time, which is what matters for the large batch, will be in proportion to the results of small processing volumes.

Meanwhile, other applications will not be affected by the jump in workload, because during the scaling-up process the application which submitted the large batch receives lower priority, and pages from applications with small load will be still processed with the highest priority.

In fine, if you want to measure real processing speed for your large document batches, you have to load the service with about 5000 pages within the shortest possible period of time.

Example

The chart below illustrates the relationship between the processing load and the number of instances.

The units of measurement on the vertical scale are different for each graph line.

The periods of time that are marked with numbers on the chart illustrate the following processes:

1. When the service receives a large batch containing many pages for processing, average instance CPU usage rate (of the instances currently running) grows to almost 100%. Service’s scalability algorithm monitors this rate, and if it stays high for some period, the service begins to scale up. It takes some time to start new instances; the delay may vary between 5 and 10 minutes for each instance.
2. The load is still high, but average instance CPU usage rate is decreasing as the service is scaling up. The incoming volume and the number of instances are gradually balanced.
3. When the load goes down, average instance CPU usage rate decreases. If this rate stays low for some period, the service begins to scale down.

Note: This graph is intended to illustrate the general principles of scaling-up and scaling-down of the service. It is based on real service productivity data, but we cannot guarantee that the processing speed of your specific documents in the specific situation will be the same as suggested by this graph.

Cloud service vs. Local server

You may want to compare the efficiency of a local server with an installed OCR solution and ABBYY Cloud OCR SDK.

On small processing loads the local instance may well perform better. It will also display better speed in the beginning of large batch processing, because the service must be scaled up first, which takes more time than simply starting OCR processing on a server which has already been configured.

On the other hand, the local server obviously cannot be scaled up indefinitely and will eventually reach the physical limit of processing speed, while the Cloud service will provide stable results for any processing amount.

In making a comparison between a local server and the Cloud OCR SDK service it is also necessary to take into account the speed of your Internet connection, as for one page the amount of time used up by uploading the image and downloading the result can actually be larger than the time spent on processing the task.

Summary

ABBYY Cloud OCR SDK service is capable of processing very large document volume and at the same time ensures quick response for all applications.