Description
Our test harness is very janky and it should be fixed before testing can be enforced properly. I've been band-aiding our test harness, but I think it needs a full touch-up before more people contribute and the bad patterns I've introduced should be fixed.
Issues
- Reliance on seeding the database multiple times when running a full test suite.
- Reliance on mock data being actually inserted into the database.
- Test effects leak between tests, so test order can change results.
- Database sessions sometimes used the one defined in
integration/conftest.py, but sometimes they use the one defined for the application.
- We have markers declared in
pyproject.toml, but they're not used.
- The way fixtures are scoped need to be made more exact.
Proper test flow
Session scoped
- Application started in test mode.
- Test database's schema is prepared and the database is emptied.
Function scoped
- Each test creates a database transaction/session.
- Each test will seed their transaction with data.
- Clients are created per test.
- Test runs.
- Test effects are cleaned up.
Changes needed
This only applies to the integration test suite, the unit tests will be looked at another time.
Loading the Test Database
Fixtures
test_database session/module scope
└── db_connection function scope + outer transaction
├── db_session function scope
└── client function scope
└── FastAPI creates sessions using db_connection
Clients
These are used to emulate users contacting the web server.
Tests
We'll need to check our test suite and update them.
Description
Our test harness is very janky and it should be fixed before testing can be enforced properly. I've been band-aiding our test harness, but I think it needs a full touch-up before more people contribute and the bad patterns I've introduced should be fixed.
Issues
integration/conftest.py, but sometimes they use the one defined for the application.pyproject.toml, but they're not used.Proper test flow
Session scoped
Function scoped
Changes needed
This only applies to the integration test suite, the unit tests will be looked at another time.
Loading the Test Database
load_test_db.py.Fixtures
join_transaction_mode=create_savepoint.load_test_db.py.db_session.flush()instead ofdb_session.commit().pytestmark = [pytest.mark.integration, pytest.mark.asyncio(loop_scope="session")]at the module level.pytest tests/integration -m integrationClients
These are used to emulate users contacting the web server.
Tests
We'll need to check our test suite and update them.