Document Collector & Scan Management

This service consists of two components which take care of document uploading to the cloud or to the DMS directly.

Architecture

The first service (STP.IM.DocumentCollectorOnPremiseService)

This service will take care of the document processing. Once document is received in the working folder it got picked up from the service and processed accordingly.

In the working folder there are several folders that help with document tracking.

  • approved
  • error
  • info
  • originals
  • pending
  • processed
  • retained
  • temp
  • work

The second service (STP.IM.DocumentCollectorManagement)

This service is a visual component and will help with basec configurations, approvals etc.

Development

Prerequisites

  • Clone the repo to C:\Projects\

Local Setup

  • In App.config update example settings to connect to the corresponding tenant, User Management (on-premise) and LSB (on-premise).
  • For debugging purposes in Visual Studio Debug Properties, /ui needs to be added as a command line argument

Run

  • Update example folder configuration in C:\ProgramData\STP AG\STP.IM.DocumentCollectorOnPremiseService\ScanFolders
  • In folder C:\Users\Public\STP.Scanfolder create folder named as it is in your folder configuration

Collecting a document

Start the service. Copy your document to C:\Users\Public\STP.Scanfolder\<yourfolder> folder.

Settings

There are two places where you can configure settings for the Document Collector * General config, usually placed in C:\ProgramData\STP AG\STP.IM.DocumentCollectorOnPremiseService

Sample for a `config.json` config file:
```json
{
"CountUploadWorkers": 3,
"DurationToErrorQueueInMinutes": 60,
"EnableUploads": true,
"ErrorItemPath": "C:\\Users\\Public\\STP.Scanfolder\\ErrorItems",
"ImageRenderingSettings": {
    "ImageRenderingMode": 4
},
"SmtpMailSettings": {
    "DefaultSenderAddress": null,
    "DefaultSenderName": null,
    "Domain": null,
    "Password": null,
    "Port": 0,
    "Server": null,
    "SSL": false,
    "UseDefaultCredentials": false,
    "User": null
},
"UploadQueuePath": "C:\\Users\\Public\\STP.Scanfolder\\UploadQueue",
"WorkingModeSettings": {
    "WorkingMode": 0
},
"RetentionCleanerWorkerRunAt": 15
}
```

* `CountUploadWorkers` - How many workers will upload documents in parallel. Optional parameter of type `number`. Default value is `3`.
* `DurationToErrorQueueInMinutes` - Duration in minutes when the file will be moved from the upload folder to error folder. Optional parameter of type `number`. Default is `60` minutes.
* `EnableUploads` - This parameter enables start of the upload workers. Optional parameter of type `boolean` (true/false). Default is `true`.
* `ErrorItemPath` - The path where errorious documents will be moved. Required parameter of type `string`.
* `ImageRenderingSettings.ImageRenderingMode` - The quality that the images will be rendered. Optional parameter of type `number`. Default value is `4`. Possible options:
    * `High Quality` - 5
    * `Normal Quality` - 4
    * `No Image Correction` - 3
    * `Salt And Paper Filtering` - 2
    * `Median Smoothing` - 1
* `SmtpMailSettings` - Settings that will enable a mail notification in case of errorious documents.
* `UploadQueuePath` - The path where documents for upload will be put. Required parameter of type `string`.
* `WorkingModeSettings.WorkingMode` - The way Document collector handle incoming files in the input folder. Optional parameter of type `number`. Default value is `0`.  Possible options:
    * `Cloud Only` - 0 - The documents will be enqueued to be uploaded in IM cloud
    * `OnPremiseOnly` - 1 - The documents will be moved in configured folder, to be inported by DMS
* `RetentionCleanerWorkerRunAt` - The local time hour when the retention worker will fire and clean up older documents. Optional parameter of type `number`. Default value is `1`.
* `MaxUploadFileSizeInBytes` - The max file size that is allowed to be read and upload. Optional parameter of type `number`. Default value is `262144000` (250MB).

  • Folder specific configs (can be multiple files)

    Sample for a ScanFolders config file: json { "DirectoryPath": "C:\\Users\\Public\\STP.Scanfolder\\Input1", "ScanIntervalSeconds": 5, "Enabled": true, "UseUploadApproval": false, "AutomaticApproveAfter": null, "MasksToTargets": [ { "TargetLaneName": ".automation", "FileMask": "*.*", "ClassName": ".automation", "OverridesTargetLaneByCode": [ ] } ], "IsBatchFolder": true, "DocumentSeparatorIdentifier": "DOCUMENT_SEPARATOR", "OutGoingMailOnly": true, "DmsImportFolder": "C:\\Users\\Public\\STP.Scanfolder\\Input1\\temp", "DmsImportFolderFallback": "", "ServiceMode": true, "EmailRecipientsOnError": "info@test.de", "KeepSeparatorSheet": true, "DmsImportFilePattern": null, "MinimumFileAgeInSeconds": 60, "RemoveEmptyPagesEnabled": true, "RemoveEmptyPagesThreshold": 5.0, "RetentionPeriods": { "OriginalsRetentionPeriodInDays": 7, "ErrorsRetentionPeriodInDays": 0, "ProcessedRetentionPeriodInDays": 0 } }

    • DirectoryPath - The folder where documents are put for processing. Required parameter of type string.
    • ScanIntervalSeconds - How many seconds service will wait before picking a document. Required parameter of type number.
    • Enabled - Enables the folder scaner worker to be stated. Required parameter of type boolean.
    • UseUploadApproval - If true requires manual approval (works with DocumentCollectorManagement). Optional parameter of type boolean. Default is false.
    • AutomaticApproveAfter - How many seconds service will wait if UseUploadApproval is set to true (works only with DocumentCollectorManagement).
    • MasksToTargets.TargetLaneName - Input Management lane where the documents will be uploaded. Optional parameter of type string.
    • MasksToTargets.FileMask - Wildcard pattern that will be used in the scanned folder to pick up files (in the example *.* means all file types). Required parameter of type string.
    • MasksToTargets.ClassName - Input Management class where the documents will be classified. Optional parameter of type string.
    • IsBatchFolder - Determines if the document should be sent directly to IM (false) or processed by Document Collector and then sent to IM (true). Optional parameter of type boolean. Default is false.
    • DocumentSeparatorIdentifier - Document separator pattern that will be used to separate the document. Required parameter of type string.
    • OutGoingMailOnly - If true bypasses Input Management and sends documents to DMS folder if wz barcode is found. Default value is false
    • DmsImportFolder - Folder that DMS is scanning directrly for a documents that have wz barcodes.
    • DmsImportFolderFallback - Backup folder that DMS is scanning directrly for a documents that have wz barcodes (works with DocumentCollectorManagement).
    • ServiceMode - Works with DocumentCollectorManagement only. Optional parameter of type boolean. Default is false
    • EmailRecipientsOnError - List of recepients on error. Optional parameter of type string.
    • KeepSeparatorSheet - Will keep or delete the page with separator. Don’t use it with barcodes with content!. Optional parameter of type boolean. Default is false.
    • MinimumFileAgeInSeconds - The interval in seconds from which on, the file is old enough for processing. Optional parameter of type number. Default is 60.
    • RemoveEmptyPagesEnabled - Determine if the empty pages of a document will be removed. Optional parameter of type boolean. Default is false.
    • RemoveEmptyPagesThreshold - A threshold from below which the page will be marked as empty. Optional parameter of type double. Default is 0.0.
    • LexDmsImportFolder_OnPremiseOnly - The folder where the documents will be put when we configure WorkingModeSettings.WorkingMode to 1 in the config.json general config. Required if the general config option is set to 1 of type string.
    • SplitDocumentByFixedBatchSize - Separates the document by a fixed page number. 0 means it is turned off, and the other types of separators can apply, otherwise the document will be split by entered number no matter if it has separators inside. Optional parameter of type number. Default is 0.
    • RetentionPeriods - The system won’t keep documents forever. Every day the service will delete the documents from folders. The time of execution is configurable in config.json file in C:\ProgramData\STP AG\STP.IM.DocumentCollectorOnPremiseService folder. Retention period is set in days. O means documents won’t get deleted.
    • RetentionPeriods.OriginalsRetentionPeriodInDays - How many days the service will wait before automatically delete documents from folder originals. Optional parameter of type number. Default is 7.
    • RetentionPeriods.ErrorsRetentionPeriodInDays - How many days the service will wait before automatically delete documents from folder error. Optional parameter of type number. Default is 0.
    • RetentionPeriods.ProcessedRetentionPeriodInDays - How many days the service will wait before automatically delete documents from folder processed. Optional parameter of type number. Default is 0.

Outgoing Mail

Since version 24.08.15.2 the DocumentCollector supports the following combinations of p4 barcodes and separator sheets:

  • Separate outgoing mail documents batches using the winsolvenz.p4 barcode only.
  • Separate outgoing mail documents batches using the winsolvenz.p4 barcode and to separate their attachments using document separator sheets.
  • Separate outgoing mail documents batches using document separator sheets only.

There are a few changes compared to previous versions:

  • Document batches with X or Y barcodes (Code39) are scanned and processed based on the barcode on the first page of the document.
  • Barcodes on the pages of the documents are recognized, and the barcode page itself is not deleted.
  • When the barcode is recognized, it is written into the file name in mode <barcode><timestamp>-<document_number_grouped_by_barcode>.pdf, and transferred to the Document Management System (DMS) collector via the configured DmsImportFolder path.
  • If no barcode is detected, the documents are transferred to Input Management.
  • If there is no DocumentSeparatorIdentifier match, the document are processed based on barcodes.

⚠️ DocumentSeparatorIdentifier cannot have the same value as the barcodes anymore since it will be confused whether it has to separate a document or an attachments. If DocumentSeparatorIdentifier matches the barcode value, the page with the barcode will be deleted and rest of the document processed in Input Management.