Improving order management solution

An order management system has several key benefits for business including visibility at every step of the sales process, reduced risk of human error, time efficiency, and improved accuracy.

The current ordering process for our customer is solved through TOM (traditional order management). This process went through three phases::

1. Origins started with EDI, which worked for the most significant customers with regular substantial orders.
2. Something smaller - a need for a solution for smaller or one-time customers came to light, a small excel document which was inserting entries into a database.
3. Advanced solution - with the help of OCR, customers were able to create orders based on PDF scanning..

In case of smaller customers, the EDI approach was deemed too robust and non-flexible, a new approach was needed. This case study focuses on the challenges we had to face during the integration of this new solution.

First draft

The original goal of this tool was aimed only at small customers. The system received the invoice in pdf format via email, followed by the OCR process where the necessary fields were identified and mapped into an XML file. XML files were sent further to the webMethods integration tool via REST, and afterward, an Idoc in SAP was created, completing the order creation.

The tool simplifies the creation of orders if the field's reading process is done correctly. Many customers preferred this solution, it started to be so popular that it took over 80% of total orders created in the company, compared to the other two options. However, the solution functioned on slow, hard-to-support software prone to malfunction by design and needed human assistance in the OCR mapping process. The system's inability to learn on its own was one of the critical reasons for future improvement.

Integration pattern we chose for order creation:

OCR tool ---> SAP

1. The user sends a PDF/XLS/PNG/JPG order via email.
2. OCR System scans this folder on a scheduled basis.
3. OCR System downloads the PDF on the server.
4. The user creates teaching in the OCR system.
5. OCR System creates an XML based on this teaching.
6. OCR System pushes the XML to the webmethods endpoint.
7. Webmethods maps the XML into the canonical document.
8. Data from the canonical document is pushed to the SAP functional module.
9. SAP creates an IDOC order.
10. Notification is created to inform a customer.

To enable the OCR tool to get the data containing the necessary company information such as addresses, prices, quantities, and users, the data has to be fed into the system first. We decided to extract CSV files from the corresponding SAP components and transfer them to the OCR tool via SFTP transfers for the first solution.

Integration pattern we chose for data refresh:

SAP ---> OCR tool

1. Once per day, a list of users is generated from the Oracle database and pushed to middleware in the file.
2. Middleware transforms the file into CSV file format and moves it to the OCR tool via SFTP connection
3. SAP collects sales history data throughout the day and pushes it to middleware in a file.
4. Once per day middleware collects all records into a single CSV file and pushes it to OCR via SFTP connection.

Fire and Forget

Several months into the production support, the first significant cracks in the design started to show. The business started to complain that some of the orders were either not created, or the notifications were not working. Shortly after, we discovered the root cause.

The setup containing information about who should receive these notification emails was stored locally on the webMethods side which proved to be insufficient as sometimes the new setup was not updated by time order was sent. To add to this gap between OCR and SAP all data from SAP were synchronized with the OCR tool only once a day and were stored locally in the database. It was an on-premise solution running on a slow and unstable physical server. These issues became the starting point for the improvement of the original solution.

Staying ahead of the curve

The new solution offers field recognition using artificial intelligence, which will significantly facilitate manual learning. The system will learn on its own based on the previous orders. One significant addition to the process is the on-demand APIs used to retrieve live data from SAP, eliminating the need to synchronize and locally store them on the OCR server.

The system processes the received invoices in pdf format into XML and saves them on the disk, from which Active Transfer (integration tool) subsequently extracts them. One of the main advantages of this reengineering is the provision of an automatic backup of XMLs.

Each integration step is tracked, and periodical acknowledgments are sent back to the OCR application. In reality, this enables the customer to get up-to-date information on their order and its state in either webmethods or SAP. This also gives the customer an opportunity to reprocess the order and eliminates the need to create a new one in case of failure at some point in the order's life cycle. If an error occurs during processing, a notification with the current order status is sent to OCR. Once the potential problem in SAP is resolved, a new update is sent to OCR, and the process continues from the point of failure.

This new solution also integrates the OCR application in the cloud, using code optimization, while data from SAP is provided on demand using SOAP.

Integration pattern we chose:

1. The user sends a PDF/XLS Order via email.
2. OCR System scans this folder on a scheduled basis.
3. OCR System downloads the PDF on the server.
4. OCR system creates learning via Artificial Intelligence.
5. OCR System creates an XML based on this teaching.
6. OCR System stores this XML on the server's file system..
7. Webmethods picks up these XML files on a scheduled basis, using the Active Transfer tool while also creating a backup in case of any system malfunction.
8. Webmethods maps the XML into the canonical document.
9. Data from the canonical document is pushed to the SAP functional module.
10. SAP creates order IDOC.
11. Notification is created to inform a customer.

66% of all orders

Less human resource needed

API on demand

Error reduction

Removed unnecessary iterations

Increase in completed orders