Extract Receipts in English

Extracts information from sales receipts using its prebuilt receipt model. It combines the powerful Optical Character Recognition (OCR) capabilities with receipt understanding deep learning models to extract key information from receipts in English. The Receipt API extracts key information from sales receipts in English, such as merchant name, transaction date, transaction total, line items, and more.

For this extraction we use Microsoft Azure Form Recognizer api.

To be able to use this activity, we need an Api Key and an Endpoint URl.

You can use the existing keys from a Microsoft Azure Form Recognizer resource(if any available), or you can create a new one. To create a new resource, once you have the azure subscription, create a Form Recognizer resource in the Azure portal to get your key and endpoint. After it deploys, click Go to resource. You will need the key and endpoint from the resource you create to connect your workflow to the Form Recognizer API.

You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Below is a screenshot of Form Recognizer resource. Just copy one of the keys to Api Key and the EndPoint to Endpoint URl.

Form Recognizer resource keysExtract receipts designer

Designer Properties#

  1. Api Key The Microsoft Azure api key for Form recognizer. Please check above how to get this key.
  2. Endpoint Url The Microsoft Azure Endpoint for Form recognizer. Please check above how to get this key.
  3. File Path The path to the file for which we want to extract the tables, words or lines.
  4. All String Fields All string fields extracted from the document for each receipt form. Represents a dictionary of fields grouped by field name(Dictionary(string, string)). If there are 2 fields with the same name, the last one will be used. Most of the time, we have only one receipt in a document, and we can use this property to access its fields as string. For all the possible keys please check https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-receipts#fields-extracted.
  5. All Receipt Fields All fields extracted from document for each receipt. Represents a dictionary of fields grouped by field name(Dictionary(string, Field)). If there are 2 fields with the same name, the last one will be used. Most of the time, we have only one receipt in a document, and we can use this property to access its fields as Field. For all the possible keys please check https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-receipts#fields-extracted.
    public class Field
    {
    public string Name { get; }
    public object Value { get; }
    public FieldType Type { get; }
    public List<Field> ValueList { get; }
    public Dictionary<string, Field> ValueDictionary { get; }
    }
    public enum FieldType
    {
    String,
    Date,
    TimeSpan,
    PhoneNumber,
    Float,
    Integer,
    List,
    Dictionary,
    }
  6. Output Tables Extracted tables by page number represented as a Dictionary(int, List(DataTable)), where the key is the page number while the value is a list of tables for that page.
  7. Merged Tables All the extracted data tables merged under one table and represented as a DataTable. This is useful when having only one table.

Properties#

Receipt extraction properties

Azure Properties#

  1. See Designer Properties above.
  2. Minimum Confidence Minimum confidence when accepting the parsed input content. By default, it is 0.5, but it can have any values between 0 and 1.

General Properties#

See General Properties.

Misc#

See Misc.

Out Error#

See Out Error.

Result#

  1. All String Fields See Designer Properties above.
  2. All Receipt Fields See Designer Properties above.
  3. Document Lines Extracted lines by page and line number represented as a Dictionary(int, Dictionary(int, string)), where the key is the page number while the value is a dictionary from line number to corresponding string on that line.
  4. Receipt Fields by Receipt The extracted fields for each receipt in the file represented as a List(Dictionary(string, Field)) where each item in the list represents the data for a receipt and each receipt has a dictionary of predefined keys to the corresponding extracted values. For more details about Field type please check above Field Class. For all the possible keys please check https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-receipts#example-output.
  5. Merged Tables See Designer Properties above.
  6. Output Tables See Designer Properties above.
  7. Output Words Extracted words by page and line number represented as a Dictionary(int, Dictionary(int, List(string))), where the key is the page number while the value is a dictionary from line number to the list of words on that line.
  8. String Fields by receipt The extracted fields for each receipt in the file represented as a List(Dictionary(string, string)) where each item in the list represents the data for a receipt and each receipt has dictionary of predefined keys to the corresponding extracted values. For all the possible keys please check https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-receipts#example-output.

Example#

Receipt Extraction

Sample receipt file

In this example, we extract all the available receipts fields from SampleReceipt and display them in a MessageBox separated by new line.

Please set the Api Key, Endpoint URl and file path using the above SampleReceipt.jpg file or any other file.