Replace sgemm() panics with Result-based error handling

[arrayka]

Summary

Refactors diskann_linalg::sgemm() to return Result<(), SgemmError> instead of panicking on invalid inputs. This change improves error handling and adds overflow protection for matrix dimension calculations.

Principle: DiskANN must not panic in production for recoverable errors; panics are reserved for catastrophic failures indicating undefined behavior.

Changes

1. New Error Type (SgemmError)

• Added SgemmError enum with two variants:
  • InvalidMatrixDimensions: Matrix has incorrect dimensions
  • DimensionOverflow: Multiplication would overflow usize
• Implemented Display and std::error::Error traits for proper error reporting

2. Updated sgemm() Function Signature

Before:

pub fn sgemm(...) { ... }

After:

pub fn sgemm(...) -> Result<(), SgemmError> { ... }

3. Overflow Protection

Replaced unsafe multiplication with checked_mul() for all dimension calculations:

• m * k validation (matrix a)
• k * n validation (matrix b)
• m * n validation (matrix c)

This prevents integer overflow vulnerabilities that could lead to undefined behavior.

4. Updated Call Sites

Updated all callers of sgemm() to handle the Result:

• diskann-quantization: Propagates errors through TransformFailed::SgemmError variant
• diskann-disk: Converts errors to ANNError with descriptive messages
• diskann-linalg tests: Added wrapper function for compatibility

5. Comprehensive Test Coverage

Added 6 new unit tests:

• test_sgemm_invalid_matrix_a_dimensions: Validates dimension check for matrix a
• test_sgemm_invalid_matrix_b_dimensions: Validates dimension check for matrix b
• test_sgemm_invalid_matrix_c_dimensions: Validates dimension check for matrix c
• test_sgemm_m_times_k_overflow: Tests overflow detection for m * k
• test_sgemm_k_times_n_overflow: Tests overflow detection for k * n
• test_sgemm_m_times_n_overflow: Tests overflow detection for m * n

Review Order

1. Core Changes (5 min)

📁 diskann-linalg/src/lib.rs

• Lines 15-50: New SgemmError type
• Lines 155-202: Overflow protection with checked_mul()

2. Tests (3 min)

📁 diskann-linalg/src/lib.rs (lines 318-461)

• 6 new tests covering all error variants
• Verify error messages are tested exactly

3. Call Sites (2 min)

📁 diskann-disk/src/utils/math_util.rs (lines 164-204)

• Three .map_err() conversions to ANNError

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.89119% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.58%. Comparing base (767879f) to head (c8cf3e1).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
diskann-quantization/src/spherical/quantizer.rs	0.00%	3 Missing ⚠️
diskann-linalg/src/lib.rs	99.39%	1 Missing ⚠️
diskann-linalg/src/reference.rs	94.73%	1 Missing ⚠️
...ation/src/algorithms/transforms/random_rotation.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
+ Coverage   89.48%   89.58%   +0.09%     
==========================================
  Files         448      459      +11     
  Lines       84095    85196    +1101     
==========================================
+ Hits        75250    76320    +1070     
- Misses       8845     8876      +31     

Flag	Coverage Δ
miri	89.58% <96.89%> (+0.09%)	⬆️
unittests	89.42% <96.89%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-disk/src/utils/math_util.rs	98.81% <100.00%> (+<0.01%)	⬆️
...nn-quantization/src/algorithms/transforms/utils.rs	90.24% <ø> (ø)
diskann-linalg/src/lib.rs	98.49% <99.39%> (+5.37%)	⬆️
diskann-linalg/src/reference.rs	97.14% <94.73%> (+0.26%)	⬆️
...ation/src/algorithms/transforms/random_rotation.rs	99.25% <0.00%> (-0.75%)	⬇️
diskann-quantization/src/spherical/quantizer.rs	95.06% <0.00%> (-0.22%)	⬇️

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR refactors diskann_linalg::sgemm() to return Result<(), SgemmError> instead of panicking, adds overflow-safe dimension validation via checked_mul(), and updates downstream call sites/tests to handle the new error path.

Changes:

• Introduces SgemmError and converts sgemm()’s dimension checks from assert_eq! to Result-based validation with overflow protection.
• Updates callers (notably diskann-disk and diskann-quantization’s linalg-backed transform) to handle sgemm() errors.
• Adds unit tests in diskann-linalg covering invalid dimensions and overflow cases.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
diskann-linalg/src/lib.rs	Adds SgemmError, changes sgemm() to return Result, adds overflow checks and new unit tests.
diskann-disk/src/utils/math_util.rs	Converts 3 sgemm() call sites to propagate errors as ANNError.
diskann-quantization/src/algorithms/transforms/utils.rs	Adds TransformFailed::SgemmError (cfg’d on linalg) and removes unconditional Copy derive.
diskann-quantization/src/algorithms/transforms/random_rotation.rs	Propagates sgemm() errors via ? from transform_into().
diskann-quantization/src/spherical/quantizer.rs	Adds match arms for TransformFailed::SgemmError, but currently handles it with panic!.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[hildebrandmw]

Thanks Alex - just a few general error handling comments from my end.