Data Requirements

By utilizing process intelligence, we are looking to analyze specific events in a chain of structured business activities. The events typically change the state of data and/or a product and generate some type of output.

Some examples of business processes include receiving orders, invoicing, shipping products, updating employee information, or servicing a customer. Business processes occur at all levels of an organization’s activities and include events that the customer sees and events that are hidden in information systems. The term also refers to the mix of all the separate steps toward the final business goal.

What kind of data does ABBYY Timeline require?

ABBYY Timeline assumes the existence of an event log where each event refers to a case, an activity, and a point in time. An event log can be seen as a collection of cases and a case can be seen as a trace/sequence of events.

Event data may come from a wide variety of sources:

  • a database system (e.g., patient data in a hospital)
  • a comma-separated values (CSV) file or spreadsheet
  • a transaction log (e.g., a trading system)
  • a business suite/ERP system (SAP, Oracle, etc.)
  • a message log (e.g., from IBM middleware)

Example Data Set

Datetime OrderID Workflow Step Employee Location Day Of Week Shift
2016-02-07 10:50:00+00 100012 Order Received Thomas Moore Philadelphia Thurs 3rd
2016-02-08 00:49:00+00 100012 Existing Customer Check Thomas Moore Philadelphia Thurs 3rd
2016-02-08 08:03:00+00 100012 Stock Check Thomas Moore Philadelphia Thurs 3rd
2016-02-09 11:40:00+00 100012 Order Hold Thomas Moore Philadelphia Thurs 3rd
2016-02-11 01:39:00+00 100012 Stock Packed Thomas Moore Philadelphia Thurs 3rd
2016-02-11 17:37:00+00 100012 Goods Shipped Thomas Moore Philadelphia Thurs 3rd
2016-02-13 14:30:00+00 100012 Goods Accepted Thomas Moore Philadelphia Thurs 3rd
2016-02-14 17:42:00+00 100012 Order Complete Thomas Moore Philadelphia Thurs 3rd
2016-01-14 21:14:00+00 100013 Order Received Thomas Moore San Francisco Mon 3rd
2016-01-15 09:39:00+00 100013 Existing Customer Check Thomas Moore San Francisco Mon 3rd
2016-01-15 17:16:00+00 100013 Stock Check Thomas Moore San Francisco Mon 3rd
2016-01-17 05:20:00+00 100013 Order Hold Thomas Moore San Francisco Mon 3rd
2016-01-21 07:35:00+00 100013 Stock Packed Thomas Moore San Francisco Mon 3rd
2016-01-22 00:46:00+00 100013 Goods Shipped Thomas Moore San Francisco Mon 3rd
2016-01-23 05:49:00+00 100013 Goods Accepted Thomas Moore San Francisco Mon 3rd
2016-01-24 09:02:00+00 100013 Order Complete Thomas Moore San Francisco Mon 3rd
2016-03-04 11:49:00+00 100014 Order Received Thomas Moore Los Angeles Sun 3rd
2016-03-05 01:22:00+00 100014 Existing Customer Check Thomas Moore Los Angeles Sun 3rd
2016-03-05 04:05:00+00 100014 Stock Check Thomas Moore Los Angeles Sun 3rd
2016-03-06 05:10:00+00 100014 Order Hold Thomas Moore Los Angeles Sun 3rd
2016-03-08 13:05:00+00 100014 Stock Packed Thomas Moore Los Angeles Sun 3rd
2016-03-08 15:41:00+00 100014 Goods Shipped Thomas Moore Los Angeles Sun 3rd
2016-03-10 01:50:00+00 100014 Goods Accepted Thomas Moore Los Angeles Sun 3rd
2016-03-11 06:29:00+00 100014 Order Complete Thomas Moore Los Angeles Sun 3rd
2016-03-10 19:01:00+00 100015 Order Received Mark Allen San Francisco Sat 3rd
2016-03-11 08:55:00+00 100015 Existing Customer Check Mark Allen San Francisco Sat 3rd
2016-03-11 17:03:00+00 100015 Stock Check Mark Allen San Francisco Sat 3rd
2016-03-12 22:34:00+00 100015 Order Hold Mark Allen San Francisco Sat 3rd
2016-03-15 18:45:00+00 100015 Stock Packed Mark Allen San Francisco Sat 3rd
2016-03-16 11:05:00+00 100015 Goods Shipped Mark Allen San Francisco Sat 3rd
2016-03-17 23:36:00+00 100015 Goods Accepted Mark Allen San Francisco Sat 3rd
2016-03-19 02:40:00+00 100015 Order Complete Mark Allen San Francisco Sat 3rd
2016-01-05 03:25:00+00 100016 Order Received Tori Stevens San Francisco Thurs 2nd
2016-01-05 16:15:00+00 100016 Existing Customer Check Tori Stevens San Francisco Thurs 2nd
2016-01-13 22:35:00+00 100016 Order Complete Tori Stevens San Francisco Thurs 2nd

Data File Requirements for TimelinePI

The most common mechanism to load your data into the ABBYY Timeline platform is via a CSV file. This document describes the required structure of the file, along with some tips and tricks related to the generation of the file.

File Structure

The data should be placed into a comma-separated file. Each row in the file represents an event – a record that something happened to a specific object at a particular time. The file must have three mandatory columns and can include any number of optional columns. All columns can have arbitrary names, as no naming rules are imposed, however no two columns should have the same name.

The mandatory columns are:
  • TimelineID – A column for some identifier of an object you want to track over time. This could be an Order ID, Claim ID, Patient Encounter Number, Support Ticket Number, and so on.
  • Timestamp – A column for the timestamp showing when something happened in the life of the object. This column generally contains a date and time (see format description below). If a date with no time is provided, midnight (12:00 AM, 00:00:00) will be used.
  • Event Name – A column describing what happened to the object at that time – Order Submitted, Patient Departed, Adjuster Assigned, Ticket Escalated etc.
Optional columns

In addition to the mandatory columns, the file can contain any number of additional columns which will be used as dimensional attributes. You can filter by these fields, group and break down by them, or use them as additional information when analyzing the processes.

Order of Records

The order of the records in the file doesn’t matter, with one exception. If several records, related to the same object, have the exact same timestamp, the application will keep them in the same order they were placed in the file.

File Sample

TimelineID Timestamp Event name Employee Location
A 1/16/2017 7:20:15 Student Applied John Boston
A 3/10/2017 16:54:10 Student Accepted Mary Boston
A 4/11/2017 15:04:00 Bill Generated Ann Charlotte
B 2/1/2017 9:15:00 Student Applied John Boston
B 3/2/2017 16:20:05 Student Accepted Mary Boston

Here, A and B are the identifiers of the traceable objects, in our example – Students. Event names are “Student Applied, Student Accepted” etc.

In a CSV file, it looks like this:

TimelineID,Timestamp,Event name,Employee,Location A,1/16/2017 7:20:15,Student Applied,John,Boston A,3/10/2017 16:54:10,Student Accepted,Mary,Boston A,4/11/2017 15:04:00,Bill Generated,Ann,Charlotte B,2/1/2017 9:15:00,Student Applied,John,Boston B,3/2/2017 16:20:05,Student Accepted,Mary,Boston

Common Issues

The most common issues with data files are a bad date/time format and a broken file format.

Date/Time Format Issues
  • 1/6/2017 7:20:15
  • 1/6/2017 7:20:15 AM
  • 2017-01-06 7:20:15
  • 2017-01-06T7:20:15Z

Make sure you save timestamps in the file in one of these formats. Keep in mind, the default Time Format in Excel doesn’t include seconds. In order to save seconds into a CSV file, switch to Custom Formats in Excel, select “m/d/yyyy h:mm” and change it to “m/d/yyyy h:mm:ss”.

File Format Issues

If the values in any field include commas, the format of the file may break. To avoid this, make sure to specify double quotes as the string qualifier. Excel does it automatically, however, some tools like MS SQL Export Wizard require manual settings like the ones seen below.

The file should be Locale English (United States) and US ASCII or UTF-8 encoded.

DataFileRequirements.png

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.

Recently viewed