feat: add indexSize and usedIndexSize to IndexStats - #1275
feat: add indexSize and usedIndexSize to IndexStats#1275AshSgDe29071999 wants to merge 1 commit into
Conversation
Expose the Meilisearch v1.53.0 per-index size fields on the Python SDK IndexStats model.
📝 WalkthroughWalkthrough
ChangesIndex statistics fields
Estimated code review effort: 1 (Trivial) | ~2 minutes Possibly related issues
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@meilisearch/models/index.py`:
- Around line 39-40: Update the index-stats model fields near index_size and
used_index_size to expose raw_document_db_size with type int | str | None,
removing the unsupported indexSize and usedIndexSize fields. Update or add
response fixtures and tests to cover both integer and human-readable string
rawDocumentDbSize values.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8763a415-a838-49ae-907b-5bf2fd4865f3
📒 Files selected for processing (1)
meilisearch/models/index.py
| index_size: int | None = None | ||
| used_index_size: int | None = None |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- candidate files ---'
git ls-files 'meilisearch/models/index.py' 'meilisearch/index.py' '*test*' '*tests*' | head -200
printf '%s\n' '--- index model outline ---'
ast-grep outline meilisearch/models/index.py || true
printf '%s\n' '--- relevant model source ---'
cat -n meilisearch/models/index.py | sed -n '1,100p'
printf '%s\n' '--- stats construction and size_format usages ---'
rg -n -C 4 'IndexStats|index_size|used_index_size|size_format|sizeFormat' meilisearch tests test* 2>/dev/null | head -300 || true
printf '%s\n' '--- dependency and test configuration ---'
rg -n -C 3 'pydantic|pytest|python_requires' pyproject.toml setup.py setup.cfg requirements* 2>/dev/null | head -200 || trueRepository: meilisearch/meilisearch-python
Length of output: 28058
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- all-stats implementation ---'
cat -n meilisearch/client.py | sed -n '340,405p'
printf '%s\n' '--- index stats tests ---'
cat -n tests/index/test_index_stats_meilisearch.py | sed -n '1,90p'
printf '%s\n' '--- client stats tests ---'
cat -n tests/client/test_client_stats_meilisearch.py | sed -n '1,100p'
printf '%s\n' '--- stats response-shape references ---'
rg -n -C 5 'databaseSize|indexSize|usedDatabaseSize|usedIndexSize|indexes' tests meilisearch README.md docs 2>/dev/null | head -400 || true
printf '%s\n' '--- installed Pydantic runtime ---'
python3 - <<'PY'
try:
import pydantic
print("pydantic_version=", pydantic.__version__)
except Exception as exc:
print("pydantic_unavailable=", repr(exc))
PY
printf '%s\n' '--- standalone Pydantic probe ---'
python3 - <<'PY'
try:
from pydantic import BaseModel, ValidationError
except Exception as exc:
print("probe_unavailable=", repr(exc))
else:
class Stats(BaseModel):
index_size: int | None = None
used_index_size: int | None = None
class FlexibleStats(BaseModel):
index_size: int | str | None = None
used_index_size: int | str | None = None
for model in (Stats, FlexibleStats):
print(model.__name__)
for payload in (
{"index_size": 123, "used_index_size": 456},
{"index_size": "1.5 GiB", "used_index_size": "2 MiB"},
{"index_size": None, "used_index_size": None},
):
try:
result = model.model_validate(payload)
print(" accepted:", payload, "=>", result.model_dump())
except ValidationError as exc:
print(" rejected:", payload, "=>", exc.errors())
PYRepository: meilisearch/meilisearch-python
Length of output: 33419
🌐 Web query:
Meilisearch API stats sizeFormat human indexSize usedDatabaseSize all stats response
💡 Result:
In Meilisearch, the /stats endpoint provides metrics regarding the instance and its indexes [1]. When querying this endpoint, you can use the optional sizeFormat parameter to control how database sizes are reported [1]. The sizeFormat parameter accepts the following values [1]: - raw (default): formats sizes as a number of bytes. - human: formats sizes as a human-readable string with an appropriate unit (e.g., KiB, MiB). Regarding key database metrics returned in the response [1][2]: - databaseSize: Represents the total disk space claimed by the Meilisearch instance. Because the underlying storage engine (LMDB) does not immediately free space after documents are deleted, this value may not decrease after deletions [2]. - usedDatabaseSize: Represents the actual space used by the data stored in Meilisearch. Unlike databaseSize, this value excludes unused space claimed by the engine and will decrease when documents are removed [2]. Additionally, you can use the showInternalDatabaseSizes query parameter (set to true) to include more granular internal database size metrics in the response [1][3].
Citations:
- 1: https://meilisearch.com/docs/reference/api/stats/get-stats-of-all-indexes
- 2: Add
usedDatabaseSizetostatsroute meilisearch#5318 - 3: https://www.meilisearch.com/docs/reference/api/indexes/get-stats-of-index
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- official stats documentation excerpts ---'
for url in \
'https://www.meilisearch.com/docs/reference/api/indexes/get-stats-of-an-index' \
'https://www.meilisearch.com/docs/reference/api/stats/get-stats-of-all-indexes'
do
echo "URL: $url"
curl -L --max-time 15 -sS "$url" |
tr '\n' ' ' |
sed 's/<[^>]*>/ /g; s/[[:space:]]\+/ /g' |
grep -oE '.{0,180}(indexSize|usedDatabaseSize|databaseSize|sizeFormat|internalDatabaseSizes).{0,260}' |
head -30 || true
done
printf '%s\n' '--- model boundary and response keys ---'
python3 - <<'PY'
from pathlib import Path
import ast
model = ast.parse(Path("meilisearch/models/index.py").read_text())
index_stats = next(
node for node in model.body
if isinstance(node, ast.ClassDef) and node.name == "IndexStats"
)
fields = [
node.target.id
for node in index_stats.body
if isinstance(node, ast.AnnAssign)
and isinstance(node.target, ast.Name)
]
print("IndexStats_fields=", fields)
index_source = Path("meilisearch/index.py").read_text()
print("direct_IndexStats_construction=", "return IndexStats(**stats)" in index_source)
sample_raw = {
"numberOfDocuments": 31,
"isIndexing": False,
"fieldDistribution": {"genre": 11},
"indexSize": 1610612736,
"usedDatabaseSize": 805306368,
}
sample_human = {
**sample_raw,
"indexSize": "1.5 GiB",
"usedDatabaseSize": "768 MiB",
}
for label, payload in (("raw", sample_raw), ("human", sample_human)):
print(label, "size_types=", type(payload["indexSize"]).__name__, type(payload["usedDatabaseSize"]).__name__)
PYRepository: meilisearch/meilisearch-python
Length of output: 7084
🌐 Web query:
site:meilisearch.com/docs/reference/api/indexes/get-stats-of-an-index "indexSize" "sizeFormat"
💡 Result:
The Meilisearch endpoint is:
GET /indexes/{index_uid}/statsindexSize is not the current response field. Size-related fields include rawDocumentDbSize; newer versions also support:
GET /indexes/{index_uid}/stats?sizeFormat=humansizeFormat=human returns sizes with units such as MiB or GiB; the default raw format returns bytes. (meilisearch.com)
Citations:
🏁 Script executed:
#!/bin/bash
set -eu
url='https://www.meilisearch.com/docs/reference/api/indexes/get-stats-of-index'
tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
curl -L --max-time 20 -sS "$url" > "$tmp"
printf '%s\n' '--- index-stats field contexts ---'
python3 - "$tmp" <<'PY'
from pathlib import Path
import re
import sys
text = Path(sys.argv[1]).read_text(errors="replace")
text = re.sub(r"<[^>]+>", " ", text)
text = text.replace(""", '"').replace("&`#x27`;", "'").replace("<", "<").replace(">", ">")
text = re.sub(r"\s+", " ", text)
for key in ("indexSize", "usedIndexSize", "rawDocumentDbSize", "databaseSize",
"usedDatabaseSize", "internalDatabaseSizes", "sizeFormat"):
print(f"\nKEY {key}")
matches = list(re.finditer(re.escape(key), text))
for match in matches[:5]:
start = max(0, match.start() - 220)
end = min(len(text), match.end() + 420)
print(text[start:end])
PY
printf '%s\n' '--- local naming and alias evidence ---'
rg -n -C 5 'class CamelBase|camel_converter|alias_generator|populate_by_name|raw_document_db_size|used_index_size|index_size|usedIndexSize|rawDocumentDbSize' . \
-g '*.py' -g '*.toml' -g '*.yaml' -g '*.yml' | head -350 || true
printf '%s\n' '--- focused model annotations ---'
python3 - <<'PY'
from pathlib import Path
import ast
tree = ast.parse(Path("meilisearch/models/index.py").read_text())
cls = next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == "IndexStats")
for node in cls.body:
if isinstance(node, ast.AnnAssign):
print(ast.unparse(node))
PYRepository: meilisearch/meilisearch-python
Length of output: 13730
Model the documented index-stats size field.
The API returns rawDocumentDbSize as an integer or a human-readable string. It does not return indexSize or usedIndexSize. Replace these fields with raw_document_db_size: int | str | None and add raw and human response tests.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@meilisearch/models/index.py` around lines 39 - 40, Update the index-stats
model fields near index_size and used_index_size to expose raw_document_db_size
with type int | str | None, removing the unsupported indexSize and usedIndexSize
fields. Update or add response fixtures and tests to cover both integer and
human-readable string rawDocumentDbSize values.
1 participant
Summary
index_size/used_index_sizefields onIndexStatsfor Meilisearch v1.53.0.Closes #1274
Summary by CodeRabbit