Skip to content

Adds Coriolis integration tests#413

Merged
Dany9966 merged 5 commits into
cloudbase:masterfrom
claudiubelu:integration-tests
Apr 27, 2026
Merged

Adds Coriolis integration tests#413
Dany9966 merged 5 commits into
cloudbase:masterfrom
claudiubelu:integration-tests

Conversation

@claudiubelu
Copy link
Copy Markdown
Member

Introduces the skeleton for integration tests that exercise the full Coriolis migration pipeline without requiring RabbitMQ, Keystone, or Barbican.

  • eventlet.monkey_patch() needs to be called (also called in coriolis/cmd/__init__.py).
  • use fake:// messaging_transport_url, effectively using an in-process fake messaging queue, instead of relying on an external service.
  • use a no-auth middleware, removing the need for Keystone.
  • use a test APIRouter, which won't have the /project_id/ prefix.
  • use an in-process worker server, which will not spawn subprocesses. This is needed; otherwise Coriolis would hang waiting for workers to reply (the messaging transport url is fake).
  • Uses python-coriolisclient to send requests to Coriolis.
  • Adds a few smoke tests, which lightly exercises the Coriolis API, ensuring the harness works.
  • Adds a light integration tests TestProvider, which implements the base provider interfaces.
  • Adds replica transfer test, deployment test, transfer failure test.
  • Adds integration entry in tox (sudo tox -e integration)
  • Adds the integration tests GitHub Action.

@claudiubelu claudiubelu force-pushed the integration-tests branch 7 times, most recently from 89ed27b to 0f365df Compare April 22, 2026 12:59
Comment thread coriolis/tests/integration/providers/test_provider/__init__.py Outdated

def _make_replicator(self, pkey_path, event_mgr, volumes_info, repl_state):
pkey = paramiko.RSAKey.from_private_key_file(pkey_path)
conn_info = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we have a test provider that points to localhost instead of a minion vm. This is very likely to pollute the test environment.

Since we're planning to use a container or vm in a subsequent PR, I won't block it.

Comment thread coriolis/tests/integration/providers/test_provider/__init__.py Outdated
Comment thread coriolis/tests/integration/providers/test_provider/exp.py Outdated
Comment thread coriolis/tests/integration/providers/test_provider/exp.py
Comment thread coriolis/tests/integration/providers/test_provider/exp.py Outdated
Comment thread coriolis/tests/integration/harness.py
Comment thread coriolis/tests/integration/__init__.py Outdated
Comment thread coriolis/tests/integration/harness.py Outdated
# SQLite does not support bulk UPDATE ... FROM ... for
# joined-table-inheritance models; replace the production function
# with a per-object alternative for the lifetime of this process.
db_api._delete_transfer_action = _sqlite_delete_transfer_action
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using quite a few hacks in order to run the Coriolis services manually, without an amqp broker or mysql database (using sqlite). It also skips the standard service launch commands, which would go untested.

On the other hand, it allows us to run the integration tests with a minimal configuration.

I'd say it's a reasonable compromise.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Running with other services might be interesting as well, unintended interactions / issues could be caught doing so. Maybe some time in the future we'd set those up as well. But for the time being, these are sufficient, it allows us to test the project internals.

Though, the sqlite issue could eventually be addressed, if we'd want to add support for that, but it's beyond the scope of this PR.

Introduces the skeleton for integration tests that exercise the full
Coriolis migration pipeline without requiring RabbitMQ, Keystone, or
Barbican.

- eventlet.monkey_patch() needs to be called (also called in
  coriolis/cmd/__init__.py).
- use `fake://` messaging_transport_url, effectively using an in-process
  fake messaging queue, instead of relying on an external service.
- use a no-auth middleware, removing the need for Keystone.
- use a test APIRouter, which won't have the /project_id/ prefix.
- use an in-process worker server, which will not spawn subprocesses.
  This is needed; otherwise Coriolis would hang waiting for workers to
  reply (the messaging transport url is fake).
- Uses python-coriolisclient to send requests to Coriolis.
- Adds a few smoke tests, which lightly exercises the Coriolis API,
  ensuring the harness works.
- Adds integration entry in tox (sudo tox -e integration)
Adds TestExportProvider and TestImportProvider.

Implements BaseReplicaExportProvider / BaseReplicaExportValidationProvider
backed by a local scsi_debug block device.

Implements BaseReplicaImportProvider backed by pre-allocated scsi_debug block
devices.
Adds shared utilities for the scsi_debug-backed test provider.
Adds Replica transfer tests, including incremental replica transfer.
Adds Replica deployment test.
Adds transfer failure test.
Adds `CORIOLIS_TEST_SSH_KEY_PATH` configuration option for the
integration tests.
Copy link
Copy Markdown
Member

@petrutlucian94 petrutlucian94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@Dany9966 Dany9966 merged commit 5b8e722 into cloudbase:master Apr 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants