Skip to content

fix(candidates+labels): tighten date pattern so a ZIP + partial date is not one 'date' - #763

Open
bosd wants to merge 1 commit into
invoice-x:masterfrom
bosd:fix/date-candidate-and-label-precision
Open

bosd wants to merge 1 commit into
invoice-x:masterfrom
bosd:fix/date-candidate-and-label-precision

Conversation

@bosd

@bosd bosd commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reported downstream (OCA/edi template builder): Guided Suggest for the DATE field captured a ZIP + partial date as a single "date" value, e.g. 18503 04/04 on an invoice where the Date label sat one line below the address block.

Two date regexes were too loose:

1. labels._DATE = r'\d[\d /.\-]{6,12}\d'

Any 8-14 char run of digits + space/dot/slash/dash bounded by digits. That's happy to span from inside a ZIP into the first half of an actual date, so a nearby Date label anchor picks up the wrong span. Replace with alternatives that each require a real numeric or month-name separator between parts:

\d{1,4}[-/.]\d{1,2}[-/.]\d{1,4}                    # 2024-05-12, 12/05/2024, 06/09/26
| \d{1,2}[-/. ]+[A-Za-z]{3,9}\.?,?[-/. ]+\d{2,4}   # 6 sept 2026, 6-Sept-2026
| [A-Za-z]{3,9}\.?[-/. ]+\d{1,2},?[-/. ]+\d{2,4}   # Sept 6, 2026

2. candidates._DATE_RE third alt: \b[A-Za-z]{3,9}\.?\s+\d{1,2},?\s+\d{2,4}\b

The letter class accepts any 3-9 letter word, so on a line like Date 12.05.2024, "Date 12.05" matched as a "date" and swallowed the digits — preventing the real 12.05.2024 from being detected. Replace [A-Za-z]{3,9} with an actual month-name alternation (case-insensitive, English full names + English/Dutch/German/French/Spanish/Italian common short forms):

(?i: jan(uary)? | feb(ruary)? | mar(ch|z)? | mrt | apr(il)? | may
   | mai | mei | jun[ei]? | jul[yi]? | aug(ust)? | sep(t(ember)?)?
   | oct(ober)? | okt | nov(ember)? | dec(ember)? | dez | dic )

The same alternation is used in the dd mon yyyy (dash/dot separator ok) and mon dd, yyyy alts, so real-world formats like 6-sept-2026, 6.sep.2026, 06/09/26, May 12, 2024 all detect cleanly, while a label like Date cannot masquerade as a month.

Tests

  • test_date_label_does_not_grab_zip_and_partial_date — the reproduction from the downstream report.
  • test_date_label_accepts_month_name_with_dashes — four format variants each go through find_labeled_fields.
  • test_find_dates_recognises_month_name_with_dash and test_find_dates_recognises_short_year_slash — candidate-side coverage.

All 22 candidate/label/template_builder tests pass locally.

Test plan

  • pytest tests/test_candidates.py tests/test_labels.py tests/test_template_builder.py — 22/22
  • ruff check / ruff format --check clean
  • CI: full matrix

…is not one 'date'

Reported by the downstream Odoo template builder: Guided Suggest for
DATE captured '18503 04/04' (a ZIP-code + first half of a real date) on
an invoice where the Date label sat one line below the address
block.

Root cause: two date regexes were loose.

- labels._DATE = r'\d[\d /.\-]{6,12}\d' — any 8-14 char run of
  digits + space/dot/slash/dash bounded by digits. Matched 'ZIP + rest
  of the day+month' as one 'date' value; Date label then anchored
  onto that span. Replace with a set of alternatives that each require
  a real numeric / month-name separator between parts.
- candidates._DATE_RE third alt \b[A-Za-z]{3,9}\.?\s+\d{1,2},?
  \s+\d{2,4}\b (and my in-progress expansion) matched 'Date 12.05' as
  a 'date' because [A-Za-z]{3,9} accepts any 3-9 letter word. That
  swallowed adjacent digits and prevented the real '12.05.2024' from
  becoming a date candidate. Replace the letter class with a proper
  month-name alternation (case-insensitive, English + common
  Dutch/German/French/Spanish/Italian short forms).

Also broadens the accepted date shapes so real-world dash-separated
month-name variants ('6-sept-2026', '6-Sep-26', '6.sep.2026') go
through, alongside the existing '2024-05-12' / '12/05/2024' / '06/09/26'
and 'May 12, 2024' forms.

Tests:
- test_date_label_does_not_grab_zip_and_partial_date — regression.
- test_date_label_accepts_month_name_with_dashes — four format
  variants each go through find_labeled_fields.
- test_find_dates_recognises_month_name_with_dash and
  test_find_dates_recognises_short_year_slash — candidate-side
  coverage.

All 22 candidate/label/template_builder tests pass locally.
@bosd
bosd force-pushed the fix/date-candidate-and-label-precision branch from c6fb101 to 66f524f Compare September 6, 2026 17:35
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/invoice2data
  __main__.py
  src/invoice2data/extract
  candidates.py
  labels.py
  src/invoice2data/extract/plugins
  tables.py
  src/invoice2data/input
  pdftotext.py
  src/invoice2data/output
  to_csv.py
  tests
  test_candidates.py
  test_labels.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant