Overview

The Default-Fetch-Worker is one possbile Fetch-Worker

Actions

When the Default-Fetch-Worker is started, the following actions take place:

A) Download Files

1. The Default-Fetch-Worker searches for storageentities with state = NEW and fetchduedate < now (or no fetchduedate)
2. It retrieves the datasource from the according entity
3. it creates sub-folders on the according storage, using the uuid of the storageentity, e.g. uuid = 59cc972a-9c7f-49be-822b-b2fb129d646a => created folders: ./59/cc/97/
4. it sets the state of the storageentity to IS_FETCHING
5. it downloads the file into the created folder
6. it saves the filesize , checksum and the fetched-at-date in the storageentity and sets its state to FETCHED
7. if download failed, the state of the storageentity is set to FETCHED_ERROR.

B) Copy Files

1. The Default-Fetch-Worker searches for storageentities with state = TO_BE_COPIED and fetchduedate < now (or no fetchduedate)
2. It retrieves the location of the file that should be copied by the customdata-field "copyfrom", which was set by the schedule-worker
3. it creates sub-folders on the according storage, using the uuid of the storageentity, e.g. uuid = 59cc972a-9c7f-49be-822b-b2fb129d646a => created folders: ./59/cc/97/
4. it sets the state of the storageentity to IS_COPYING. If the state of the source-storageentity is TO_BE_MOVED, this state is set to IS_MOVING.
5. it copies the file into the created folder
6. it saves the filesize, checksum and the fetched-at-date in the storageentity and sets its state to FETCHED. When the state of the source-storageentity is "IS_MOVING", this state is set to "MOVED"
7. if the copy-process fails, the state of the source-storageentity is set to MOVING_ERROR and the state of the storageentity is set to FETCHED_ERROR.