Skip to content

Update source table for a next execution

Once a source has finished processing successfully, the system should update the Source table with the necessary changes to prepare it for the next execution. For example, suppose we are running the UNICHEM process for the sources ChEMBL and ChEBI, and the current state of the Source table is as follows:

id name official_name last_released_version download_url
1 chembl ChEMBL 34 /Users/carlosm/Documents/UNICHEM/unichem-loader/tests/mock_cluster_folder/chembl_34.sdf
7 chebi ChEBI (Chemical Entities of Biological Interest). 221 /Users/carlosm/Documents/UNICHEM/unichem-loader/tests/mock_cluster_folder/chebi_lite.sdf.gz

By checking the download_url or last_released_version column, we can determine that the current versions being processed are ChEMBL_34 and ChEBI 221. If the UNICHEM process completes successfully for both sources, then the next execution should target ChEMBL_35 and ChEBI 222. Therefore, the Source table must be updated accordingly:

id name official_name last_released_version download_url
1 chembl ChEMBL 35 /Users/carlosm/Documents/UNICHEM/unichem-loader/tests/mock_cluster_folder/chembl_35.sdf
7 chebi ChEBI (Chemical Entities of Biological Interest). 222 /Users/carlosm/Documents/UNICHEM/unichem-loader/tests/mock_cluster_folder/chebi_lite.sdf.gz

Depending on the source, will be update one or two columns with the necessary information for the next execution.