Workers concurrently register execution for one step

[trihoangvo]

Bug Report

Description

In yorc logs, we see sometimes, two task executions are registered for one step at the same time. (See two duplicated entries "All previous steps of step" in the log below). As a result, workers concurrently work on the registered task execution. They create and remove ansible files from each other (e.g., overlay files) and cause unexpected error.

2022/09/20 17:05:26 [DEBUG] All previous steps of step:"Secrule_inbound_RDSMySQL_database_endpoint_install" are done, so it can be registered to be executed
2022/09/20 17:05:26 [DEBUG] All previous steps of step:"Secrule_inbound_RDSMySQL_database_endpoint_install" are done, so it can be registered to be executed
2022/09/20 17:05:26 [DEBUG] Register task execution with ID:"445ecfcc-da88-418e-ac35-1493d3c7a057", taskID:"89f0adf7-4b64-47bc-b76e-7b74e5c58905" and step:"Secrule_inbound_RDSMySQL_database_endpoint_install"
2022/09/20 17:05:26 [DEBUG] Will store runningExecutions with id "445ecfcc-da88-418e-ac35-1493d3c7a057" in txn for task "89f0adf7-4b64-47bc-b76e-7b74e5c58905"
2022/09/20 17:05:26 [DEBUG] Register task execution with ID:"d88fc3f0-6a59-4be1-81b8-20c800ed5b81", taskID:"89f0adf7-4b64-47bc-b76e-7b74e5c58905" and step:"Secrule_inbound_RDSMySQL_database_endpoint_install"
2022/09/20 17:05:26 [DEBUG] Will store runningExecutions with id "d88fc3f0-6a59-4be1-81b8-20c800ed5b81" in txn for task "89f0adf7-4b64-47bc-b76e-7b74e5c58905"

Expected behavior

One task execution for a given step is registered.

Actual behavior

Two duplicated task executions are registered.

Additional information you deem important

This issue happens only occasionally when two or more steps join into one step. For example:

Step 1 ------> Step 3
Step 2 -----------^

• Worker 1 completes Step 1.
• Worker 2 completes Step 2 and registers next Step 3.
• Worker 2 checks all previous steps of 3 are DONE, acquires lock for task, registers task execution A for step 3, and unlocks.
• Worker 1 checks all previous steps of 3 are DONE, acquires lock for task, registers task execution B for step 3, and unlocks.

The following lock:

https://github.com/ystia/yorc/blob/develop/tasks/workflow/step.go#L621

cannot prevent this issue.

Output of yorc version

develop

Priority

Low