CIB ocr technical manual (EN)

Tesseract is available for Linux, Windows and Mac OS X. Since version 3.00 Tesseract has supported output text formatting, hOCR positional information and page layout analysis. Support for a number of new image formats was added using the Leptonica library.  Tesseract is able to process a lot of different languages, German included. 
  

First a valid license is set and tracing is activated. To recognize the QR-code, the properties “BarcodeType” and “Recognize” have to be set accordingly. Then the  output filename and the output directory are set. CIB  ocr is called via the command “–oc ”, followed by the input filename of the .jpg  image that is to be processed.  

All properties used in this call are described in more detail in chapters 4 to 6. 

CIB runshell can be called directly via command line in the appropriate directory, or via a batch script.  

Cibrsh.exe LicenseCompany=“Example Company“ LicenseKey=“xxxx-xxxx-xxxxxxxx“ 
TraceFilename=”trace.log” BarcodeType=QR Recognize=BarcodeRecognizer 
BarcodeOutputFilename=”./output/CIB_web.jpg.txt” –oc “./input/CIB_web.jpg”    

Inputfile: 

This is the QR Code as JPG file:

BARCODE:QR;http://www.cib.de 

This example uses CIB ocr to extract text from a PDF file that is not readable, i.e.  the text is represented by a picture. To pass through CIB ocr properties in this CIB pdf toolbox call, “CibOcr” is added as a prefix to the CIB ocr properties described in this document.   

Again a valid license has to be set. Then the property “OutputFormat” is set to specify that a searchable PDF should be generated. It is also specified that in this case the image in the input PDF should be removed and the extracted text from that image shown instead in the output PDF  (properties “FormatSearchablePdfRemoveImage” and “FormatSearchablePdfShowText”).  

The language of the text in the PDF is set accordingly (“CibOcrOCRLanguage”) and the path to the necessary dictionary files is set (“CibOcrDataFolder”).  Also, a preprocessor is selected to improve the ocr result.  

Then cib pdf toolbox is called via “–fj ” (join) and the input and output files are specified. 

CibRsh.exe LicenseCompany=”Example Company” LicenseKey=”xxxx-xxxx-xxxxxxxx” 
OutputFormat=FormatSearchablePdf FormatSearchablePdfShowText=1 
FormatSearchablePdfRemoveImages=1 CibOcrDataFolder=”./tessdata” CibOcrOCRLanguage=”eng” 
CibOcrPreprocess=SauvolaThresholding –fj “./input/TextAsPicture.pdf” 
“./output/TextAsPicture.out.pdf”
    

In load-step inputfile is loaded into memory.  
In ocr-step first a valid license has to be set, “Tesseract” is the OCRlibrary to be used and OutputFormat is “FormatHocr”. Extracted text is in German ("OCRLanguage"=deu). 

In save-step in-memory-output is written into a file. 

Example: 

<?xml version="1.0" encoding="ISO-8859-1" ?>  
<root> 
   <Comod> 
      <defaults> 
         <properties command="job"> 
            <property name="OutputMode">XML</property> 
            <property name="UseInMemoryProcessing">1</property> 
         </properties> 
      </defaults> 
      <jobs> 
         <job name="tesseract" expected-result-code="404"> 
         <steps> 
            <step name="LoadStep" command="load"> 
            <properties> 
               <property name="InputFilename">./input/input.png</property> 
            </properties> 
            </step> 
            <step name="OcrStep" expected-result-code="1000" command="ocr"> 
            <properties> 
               <property name="LicenseCompany">CustomerLicensee</property> 
               <property name="LicenseKey">4444-cccc-88888888</property> 
               <property name="OCRLibraryName">Tesseract</property> 
               <property name="DataFolder">.</property> 
               <property name="OutputFormat">FormatHocr</property> 
               <property name="TraceFilename">OCR_trace.log</property> 
               <property name="OCRLanguage">deu</property> 
               <property name="TracePreprocessOutput">0</property> 
            </properties> 
            </step> 
            <step name="SaveStep" expected-result-code="0" command="save"> 
            <properties> 
               <property name="OutputFilename">./ocr_out.html</property> 
            </properties> 
            </step> 
         </steps> 
         </job> 
      </jobs> 
   </Comod> 
</root> 

− NEW: add new property "PageSelection" RELEASE 2.3.0 

− property “TraceRecognitionOutput” has been added; REL 2.1.0 

This property allows to specify methods for  preprocessing the inputfile in order to  get a better ocr-result. 

Syntax  

Preprocess: <preprocessnames> 
<preprocessnames>= <preprocessor> | <preprocessor> “+” <preprocessnames> 
<preprocessor>: NativeAdaptiveThresholding | PureMedianBlur | PureAdaptiveThresholding |
PureAdaptiveGaussianThresholding | MedianBlurAGT | MedianBlurAT | MedianBlurGAT | 
SauvolaThresholding | NiblackThresholding | WolfJolionThresholding | NickThresholding | 
FengThresholding | OtsuThresholding | BilateralFilter | ThinningZhangSuen | 
ThinningGuoHall | DeSkew | Invert | AutoInvert | Composite

No default

Example 

Preprocess = NativeAdaptiveThresholding 

Using a global value as threshold value  may not be good in all conditions where  an  image has different lighting conditions in different areas. In that case, we go for adaptive thresholding. Adaptive thresholding means that  the algorithm calculates the threshold for small regions of the image.  Thus we get different thresholds for different regions of the same image and this gives us better results for images with varying illumination. 

It has three ‘special’ input parameters and only one output argument.  

Adaptive Method - It decides how the  thresholding value is calculated. 

This property specifies a path to the Tesseract language package „tessdata“.  

If the property is empty, it is assumed the folder „tessdata“ is located in the currently used folder. 

Syntax 

DataFolder=<path> 

default=No input 
(current folder is used) 

Example 

DataFolder=C:\Test\Invoice 

(From CIB ocr version 2.3.1) 

Names of Tesseract config files. 

All Tesseract config files should be located in $(TESSDATA_PREFIX)\tessdata\configs\ 

Syntax 

OCRConfigs=config_name1[;config_name2[;config_name3...]] 

Examples 

OCRConfigs=hocr 

or 

OCRConfigs=hocr;debug 

This property specifies a rectangle on a page. 
This rectangle is used to define a scan-area to extract the text. That means all characters are ignored which are outside of this scan-area.  
A rectangle is defined by two basic points (left,top and right,bottom).  
Point of origin is the top-left corner of the page, the unit is mm.  

Syntax 

OCRRegion=<onerectangle> 
<onerectangle>: <left> ";" <top> ";" <right> ";" <bottom> 

default=No input 

The whole page is scanned if no input is set or if the rectangles given by the coordinates are empty. 

Example 

OCRRegion=5;5;15;20 

This property adds a horizontal padding to the rectangle determined by “OCRRegion” on a page. 
The main Use Case is Textrecognition with deeper on a line. This Property allows to further extend the OCRRegion in horizontal direction. This way context information in the image will not get lost.  

The unit is %. That means in case of a OCRRegion width of 100 and PaddingHorizontal of 10: The image is extended by 10 pixels left and 10 pixels right. The center of the OCRRegion and the padded rectangle remains. 

Syntax 

PaddingHorizontal =<integer_value> 

default=0 

Example 

PaddingHorizontal = 10 

This property adds a vertical padding to the rectangle determined by “OCRRegion” on a page. 
The main Use Case is Textrecognition with deeper on a line. This Property allows to further extend the OCRRegion in vertical direction. This way context information in the image will not get lost.  

The unit is %. That means in case of a OCRRegion height of 100 and PaddingVertical of 10: The image is extended by 10 pixels on top and 10 pixels at the bottom. The center of the OCRRegion and the padded rectangle remains. 

Syntax 

PaddingVertical=<integer_value> 

default=0 

Example 

PaddingVertical=10 

This property specifies the name of the outputfile.  
The property OutputFilename is optional, if it is empty – OutputTextLength and OutputText are used. 
The format/extension is described in the next property OutputFormat. 

Syntax 

OutputFilename=<name> 
<name>: name.ext  

default=No input, use of OutputTextLength and OutputText. 

Example 

OutputFilename=Rechnung.txt 

This property defines the format of the created outputfile. 

Syntax 

OutputFormat=<format> 
<format>: FormatText | FormatHocr | FormatHocrText 

default=FormatHocr 

Example 

OutputFormat=FormatHocr 

This property contains the result of text-recognition. 
If used, it is also required to define the size of the output buffer with the property OutputTextLength. 

Syntax of output: 

[textstring] 

Example 

Das ist der gelesene Text. 

This property contains the length of the output result and specifies the required size of the output buffer. 

Syntax of output: 

[integer] (string representation)  

Example 

1000 

This property defines whether output should be in memory or to a file. 
This property is automatically set depending on whether OutputFilename is set or not. If OutputFilename is set, then OutputType=File is automatically set, otherwise OutputType=Memory is set.  

Syntax 

OutputType=<type> 
<type>: File | Memory  

default=File 

Example 

OutputType=File 

The property RegionTemplate contains the name of the xfdf-file, where the OCRRegions are defined. 

Syntax 

RegionTemplate=<filename.xfdf> 

default=No input 

Example 

RegionTemplate=region.xfdf 

This Property is mandatory.  

It is not yet possible to use In-Memory-Processing for the input. 

This property sets the name of the outputfile to which values of all barcodes found in the inputfile are written. 
The property BarcodeOutputFilename is optional, if it is empty - BarcodeValueLength  and BarcodeValue are used . 
For format/extension look at OutputFormat  

The format of this file is given by “BarcodeOutputType ”. 

Syntax 

BarcodeOutputFilename=<filename> 
<filename>:name.ext 

default=No input, BarcodeValueLength and BarcodeValue are used . 

 Example 

BarcodeOutputFilename=barcodes.txt 

This property defines whether the output should  be in memory or to file. 
This property is automatically set depending on whether BarcodeOutputFilename is set  or not. If BarcodeOutputFilename is set ,  then BarcodeO utputType=File is set,  otherwise  BarcodeOutputType=Memory .   

Syntax 

BarcodeOutputType=<type> 
<type>: File | Memory  

default=File 

Example 

BarcodeOutputType=File 

With this property it is set that a search for barcodes is only applied on a defined rectangle. 

If a barcode is always situated on the same area of each page , performance is improved by limiting  the  search for barcodes to this area.  
Th is area (= rectangle) is defined by two basic points (left ,top and right,bottom).  
Point of origin is the top-left corner of the page,  the unit is  pixel.  

Syntax 

BarcodeRegion=<rectangle> 
<rectangle>: <left> ";" <top> ";" <right> ";" <bottom> 

default=No input 

The whole page is scanned if no input is done or rectangle given by coordinates is empty. 

Example 

BarcodeRegion=10.5;50;100.0;200.5 

When this property is set, checksum is not considered  when reading a barcode. 

The property can be used for barcodes Code39 , Code39Extended and Code128   

Syntax 

BarcodeRemoveChecksum=<value> 
<value>: 0 | 1 

default=0 
(Checksum is not cut off). 

Example 

BarcodeRemoveChecksum=1 

When this property is set, the page-number of the inputfile where this barcode was found is added to each output barcode from  property “BarcodeValue”. 

Syntax  

BarcodeShowPageNumber=<value> 
<value>: 0 | 1  

default=0 
(No output of page-number). 

Example 

BarcodeShowPageNumber=1 
Afterward property „BarcodeValue“ contains e.g. 
BARCODE:1;DATAMATRIX;00011122233344455566677788;BARCODE:2;DATAMATRIX;899374032904908

For Datamatrix only. 

Defines that search is stopped after retrieving N th barcode. 

Syntax 

BarcodeStopAfter=<value> 
<value>: 1 to N 

default=”” 
No value set means that search continues until the end of inputfile. 

Example 

BarcodeStopAfter=5 

Search stops after retrieving 5th barcode.

This property specifies the type of barcode which is to be searched. 
 

Syntax  

BarcodeType = < onebarcodetype>  

<onebarcodetype>= "DataMatrix" | "Code128" | "Code39" | "Code39Extended" | "QR" 

For Datamatrix only. 

This property  specifies the time (in milliseconds) when CIB  ocr stops looking for more barcode-candidates in the inputfile. 

Syntax 

BarcodeTimeout=<value>
<value>:1 to N  

default=The whole inputfile is processed  to find all barcode-candidates, neglecting the time it takes.  

Example 

BarcodeTimeout=50 

This property contains a list of all barcodes found in the  input-image-file. 
BarcodeValueLength is  necessary to define the  size of the output buffer.  

Syntax of output: 

[BARCODE :[pagenumber;]BarcodeType;TextValue;] 

Example 

BARCODE:DATAMATRIX;00011122233344455566677788 

This property contains the length of the output result and thus gives the required size of output buffer.  

Syntax of output: 

[integer] (string representation)  

Example 

1000 

default=10 

Example 

DatamatrixAngleDeviation=20 

For Datamatrix only. 

This property sets a factor for shrinking a high resolution image internally. 
This sometimes provides a dramatic performance benefit  as the amount of pixels of a page is minimized. It especially helps when  an image has  high resolution but blurry focus.  

Syntax  

DatamatrixShrinkingFactor=<value>
<value>: 1 to N  

Default: 1  
(no change of original resolution) 

Example 

DatamatrixShrinkingFactor=2 

Means resolution is divided in half. 

For Datamatrix only. 

This Property allows specifying the size of the gaps in the grid pattern (using pixels). 

Increasing the gaps (e.g. to 100) can improve performance, but  if the grid is too coarse it may cause that  the barcode is no t found at all. 

Syntax 

DatamatrixScanGap=<value>
<value>: 1 to N 

default=1 

Example 

DatamatrixScanGap=50

For Datamatrix only. 

Lowering the threshold can increase the number of features to be scanned, but thereby slows performance. But this  may be necessary if the image is blurry or has low contrast.  
Sometimes lowering the threshold will actually improve performance if thereby a good barcode candidate  is found more quickly than otherwise. 

Syntax 

DatamatrixThreshold=<value>
<value>: 1 to 100  

default=5 

Example 

DatamatrixThreshold=10 

Weak edges below threshold 10 are ignored. 

(From CIB ocr version 2.4.0) 

Property for tuning of barcode recognition (ZBar functionality). 

Syntax 

ZBarConfig
        =config_
        line1[;config
        _line2[;config_line3...]] 

Example 

ZBarConfig=code39.enable 

In order to use the WordRecognizer this property has to be set to “WordRecognizer”.  

Syntax 

Recognize=<Value> 
<Value>:BarcodeRecognizer | OcrRecognizer | WordRecognizer 

default=OcrRecognizer   

Example 

Recognize=WordRecognizer 

This property can also be defined within the WordRecognizerOptions. It is recommended to define it within  WordRecognizerOptions, as WordRecognizerOptions  overrules  this property.  

However this property has to be defined at least within this property or WordRecognizerOptions.  

Syntax 

DictionaryPath=<Value> 

No default value! It has to be set. 

Example 

DictionaryPath=".\\hunspell" 

In order to use the WordRecognizer this property has to be set to “WordRecognizer”.  

Syntax 

InputFormat=<Value> 
<Value>: HOCR | UTF8 | UTF16 | Unicode 

HOCR: input is a HOCR file which must be UTF8 encoded 
UTF8: input is plain text in UTF8 encoding (with or without UTF8 BOM ) 
UTF16: input is plain text in UTF-16 encoding. The BOM (FE FF or FF FE) must be present 

Example 

InputFormat=UTF8 

This property is defined as json-String and contains all the information that is needed in order to analyse the document by WordRecognizer. 

This property might look like this. A more detailed explanation for each component can be found below the example:  

Example: 

{ 
"DictionaryPath": "D:\\PROJEKTE-SVN\\products\\CIB ocr\\trunk\\src-test\\testdata\\hunspell",  
"Dictionaries": {"DE": {"Dictionaries": "de_DE_frami-UTF8", "StopwordFiles": "stopword_german.txt", "DigramScores": "de_digramscores.txt"}}, 
"InputFormat": "UTF8",  
"RecognizedWordsFilename": "recognized.log",  
"RejectedWordsFilename": "rejected.log",  
"StatisticsFilename": "statistics.log",  
"StatisticsOutputFormat": "FormatCsv"} 

Explanation of each component: 

Component “Dictionaries”: 

specification of <dictionary-object> (same as a few comments above): 

{<language-name>: <language>, ...}  

<language-name> = JSON-String: "..." (specifies a language name)  
<language> = JSON-String: "..." (specifies  a single dictionary for that language, no stopwords) 
<language> = JSON-Array: ["...","..."] (specifies one or more dictionaries for that language, no stopwords) 
<language> = JSON-Object: {"Dictionaries": <dictionaries>, "StopwordDictionaries:" <stopword-dicts>, "Stopwords": <stopwords>]} 
<dictionaries> = JSON-String: "..." (specifies a single dictionary for that language) 
<dictionaries> = JSON-Array: ["...","..."] (specifies one or more dictionaries for that language) 
<stopword-dicts> = JSON-String: "..." (specifies a single stopword dictionary for that language) 
<stopword-dicts> = JSON-Array: ["...","..."] (specifies one or more stopword dictionaries for that language) 
<stopwords> = JSON-Array: ["...", "..."] (specifies a list of stopwords (UTF-8 encoded)) 

Example 1: 

"Dictionaries": {"DE": ["de_DE-frami-UTF8", "de_user"], "EN": "en_US-UTF8"} 

Example 2: 

"Dictionaries": { 
"DE": {"Dictionaries": "de_DE-frami-UTF8", "StopwordDictionaries": "de_stopwords", "DigramScores": "de_digramscores.txt"}  
"EN": {"Dictionaries": "en_US-UTF8", "Stopwords": ["a", "an", "in"], "DigramScores": "de_digramscores.txt}
    }

Wordrecognizerresult

WordRecognizerResult = { 
   "DocumentStatistics": { 
      "AllLanguages": {"GlyphRatioLongWords": 80, ...}, 
      "Languages": { 
         "DE": {"GlyphRatioLongWords": 55, ...}, 
         "EN": {"GlyphRatioLongWords": 33, ...}  
      }, 
      "TextAccepted": false  
}, 
"PageStatistics": { 
   "1": { 
      "AllLanguages": {"GlyphRatioLongWords": 100, ...}, 
      "Languages": { 
         "DE": {"GlyphRatioLongWords": 100, ...}, 
         "EN": {"GlyphRatioLongWords": 16, ...}  
      }, 
      "TextAccepted": true  
   }, 
   "2": { 
      "AllLanguages": {"GlyphRatioLongWords": 55, ...}, 
      "Languages": { 
         "DE": {"GlyphRatioLongWords": 0, ...}, 
         "EN": {"GlyphRatioLongWords": 55, ...}  
      }, 
      "TextAccepted": false  
    }  
  }  
}