CRUD for automations in the API and UI

[Flix6x]

Description

Automations can now be created, updated and deleted through the API and the UI, not only from the
CLI.

API. [POST] /assets/(id)/automations, [PATCH] /assets/(id)/automations/(automation_id) and
[DELETE] /assets/(id)/automations/(automation_id). They require the same rights as deleting the
asset — account admins and consultants — which matches the automation's own access rules: whoever
may read an asset may read its automations, and whoever may delete it may change them. A PATCH
covers the name, the recurrence, the timezone and the activation status; the parameters are
deliberately not editable, so the sensors an automation involves stay the ones its creator was
checked against.

UI. The asset's Automations page gains a creation modal and per-row actions for those users.

Only sensors the creator can access. An automation administered this way may only involve
sensors its creator can access themselves: read access to the sensors it reads from, and
create-children on the sensors it writes to, which is the permission the API already requires for
recording data on a sensor. A refused request gets a 403 naming the sensor and the action. This
sits behind a flag that the API passes; the CLI creates automations without a user and stays
unrestricted.

Which sensors those are depends on the type:

• Forecasts — the forecaster reports its own input and output sensors, derived from the very
  config and parameters it will run with, so a regressor that filters on sources counts too even
  though it is a sensor reference rather than a plain sensor.
• Schedules — the scheduler resolves its flex config first, so sensors inherited from the asset
  tree are included, and the outputs are then taken from the fields that name where results are
  recorded: a device's power sensor, its state of charge, consumption and production sensors, and
  the flex-context's aggregates. Everything else the parameters refer to counts as an input.

Timezone through the API. An automation carries the timezone its recurrence is interpreted in.
Creation and update now accept it, so automations administered through the API or the UI are no
longer stuck on the server's timezone. Changing it rebases the scheduling cursor, exactly as the CLI
does.

Refusals leave nothing behind. The data source holding a forecaster's configuration is set up
only once the automation is allowed, so a refused request adds nothing. The check on where a
forecast may be recorded runs after the access check, so a sensor the caller may not read is refused
as forbidden rather than described as being outside the asset.

Consolidation. Creating, updating and deleting live in flexmeasures/data/services/automations.py,
so the CLI and the API share one implementation rather than each building automations inline.

• Added changelog item in documentation/changelog.rst

Look & Feel

An account admin gets a New automation button and per-row Edit, Activate /
Deactivate and Delete actions:

The same page, for the same asset, as a plain user of the same organisation. The automations and
their details are still readable; nothing that would change them is offered:

Creating one asks for the type, the recurrence and the timezone it is interpreted in, and the
parameters — forecast parameters, or a schedule trigger message:

How to test

See the manual test walkthrough in the PR comments.

pytest \
  flexmeasures/api/v3_0/tests/test_automations_api.py \
  flexmeasures/api/v3_0/tests/test_automations_api_fresh_db.py \
  flexmeasures/cli/tests/test_automations.py \
  flexmeasures/data/tests/test_automations_fresh_db.py \
  flexmeasures/ui/tests/test_asset_crud.py

Coverage includes a forecast on another organisation's sensor, a forecast whose source-filtered
regressor is another organisation's sensor, a schedule aggregated onto another organisation's
sensor, and the timezone roundtrip through creation and update. Each was verified to fail without
the check it covers.

Further improvements

• Which sensors a schedule would be recorded on is worked out by resolving the flex config and then
  reading the fields that name where results go. That agrees with the scheduler's own resolution for
  a device's sensor, consumption and production, and errs towards reporting more rather than
  fewer. Checking outputs against what a scheduler actually returns at run time is tracked in Check the sensors a scheduler actually writes to, instead of predicting them when an automation is created #2421.
• The parameters of an existing automation cannot be edited. Changing what an automation computes
  means deleting it and creating a new one, which keeps the access check honest but is blunt.

Related items

Closes #2372. Part of the automations story #2334. Stacked on #2293.

---

Sign-off

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on code under GPL or another incompatible license.

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #34335451 | 📁 Comparing e8f5bc1 against latest (60cfda8)

  🔍 Preview build  

139 files changed · + 24 added · ± 115 modified

+ Added

• _autodoc/COUPLING.html
• _autodoc/COUPLING_COEFFICIENT.html
• _autodoc/GROUP.html
• _autodoc/INFLEXIBLE_CONSUMPTION.html
• _autodoc/INFLEXIBLE_PRODUCTION.html
• _autodoc/OPERATION_MODES.html
• _autosummary/flexmeasures.api.common.rate_limiting.html
• _autosummary/flexmeasures.api.v3_0.utils.html
• _autosummary/flexmeasures.data.models.automations.html
• _autosummary/flexmeasures.data.models.planning.devices.html
• and 14 more...

± Modified

• changelog.html
• configuration.html
• genindex.html
• getting-started.html
• http-routingtable.html
• index.html
• py-modindex.html
• _autodoc/COMMITMENTS.html
• _autodoc/CONSUMPTION_CAPACITY.html
• _autodoc/INFLEXIBLE_DEVICE_SENSORS.html
• and 105 more...

[Flix6x]

@BelhsanHmida — Felix reviewed the automations work and asked for one thing to land here: an automation administered through the UI (and hence the API) may only involve sensors that its creator can access. Could you review these commits, and then take the PR over again?

What we contributed:

• create_automation now works out which sensors an automation would read from and write to, and requires read access to the former and create-children on the latter — the same permission the API requires for recording data on a sensor. A refused request gets a 403 naming the sensor and the action, through the existing api_message mechanism.
• This is behind a check_permissions flag that the API passes; the CLI creates automations without a user and stays unrestricted.
• Forecasts: the sensor to forecast plus its regressors are inputs, the sensor to save to (the same sensor by default) is the output.
• Schedules: schedulers hand their results to make_schedule as (sensor, data) pairs, so the outputs are not only the flex-model's device sensors — a schedule is also recorded on a device's state-of-charge, consumption and production sensors, and on the flex-context's aggregate-consumption and aggregate-production. We derive the outputs from all of those fields at any depth, since the flex-model and flex-context are deserialized by the scheduler itself and their sensor references are still raw at this point. Everything else the parameters refer to (price sensors, inflexible-device sensors — these can live on the flex-context too) counts as an input.
• Tests cover both a forecast on another account's sensor and a schedule aggregated onto another account's sensor; both were verified to fail without the check. Docs, changelog and API change log updated.

Two things to be aware of:

1. This branch is ~108 commits behind its base, so it does not yet have the input_sensors / output_sensors properties we added to DataGenerator in Automations - first roundtrip for forecasts #2290. We kept the sensor resolution inside create_automation rather than duplicating those properties, so syncing the stack should be a small edit rather than a conflict — but the two should be collapsed into one implementation then.
2. Deriving output sensors statically is an approximation of what a scheduler actually returns at run time. If a scheduler starts writing somewhere else, this check will not know about it, so it is worth revisiting whenever a new output field is added.

🤖 Generated with Claude Code

[BelhsanHmida]

Manual test walkthrough

Covers what CRUD adds over #2290 and #2293: administering automations through the API and the UI,
and the rule that you may only involve sensors you can access yourself. The responses below were
captured against a development database on this branch.

You need two logins to see the interesting half:

• an account admin (or consultant) on the asset — can create, edit and delete automations
• a plain user of the same organisation — can read automations, and is offered nothing that
  would change them

The examples use asset 242 (Campus, owned by the Campus Demo organisation) and its power sensor
913.

1. The page gains controls, for the right people

/assets/242/automations

As an account admin the listing gains a New automation button and per-row Edit,
Activate / Deactivate and Delete actions. As a plain user of the same organisation the
listing and its Details are unchanged, and none of those controls appear. Both are shown in the
PR description.

2. Create one from the UI

New automation asks for the name, the type, the recurrence, the timezone it is interpreted in,
and the parameters — forecast parameters for a forecast automation, or a schedule trigger message
for a schedule one. Create a forecast automation on sensor 913, daily at 06:00, in
Europe/Amsterdam, and it appears in the listing with its recurrence in words and its timezone.

3. The same over the API

# create
POST /api/v3_0/assets/242/automations
{
  "name": "Daily campus forecast",
  "type": "forecasts",
  "cronstr": "0 6 * * *",
  "timezone": "Europe/Amsterdam",
  "parameters": {"sensor": 913}
}
# → 201, with the automation, its timezone and its scheduling cursor

# change the recurrence, the timezone and the activation status
PATCH /api/v3_0/assets/242/automations/<N>
{"cronstr": "30 7 * * *", "timezone": "Asia/Seoul", "active": false}
# → 200

# delete
DELETE /api/v3_0/assets/242/automations/<N>
# → 204

The timezone is worth checking specifically: an automation created through the API used to be stuck
on the server's timezone, with only the CLI able to change it. Changing it also rebases the
scheduling cursor, so the automation does not immediately catch up on occurrences that only exist in
the new timezone's past.

Note that parameters cannot be patched. What an automation computes is fixed when it is created,
which is what keeps the access check below meaningful — the sensors it involves stay the ones its
creator was checked against.

4. You may only automate sensors you can access

This is the substance of the PR. As an account admin of one organisation, try to forecast a sensor
belonging to another. Sensor 426 below belongs to a different organisation than asset 242:

POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
 "parameters": {"sensor": 426}}

HTTP 403
{
  "message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
  "result": "Rejected",
  "status": "INVALID_SENDER"
}

A regressor that filters on sources is a sensor reference rather than a plain sensor, and is checked
just the same:

POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
 "parameters": {"sensor": 913},
 "config": {"regressors": [{"sensor": 426, "source-types": ["forecaster"]}]}}

HTTP 403
{
  "message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
  "result": "Rejected",
  "status": "INVALID_SENDER"
}

Three things are worth confirming while you are here:

• Nothing is created. The automation does not appear in the listing, and no data source is left
  behind for the forecaster it would have used — the data source is set up only once the automation
  is allowed.
• The message names the id, not the name. Refusing access should not disclose what a sensor you
  cannot read is called.
• Writing is checked separately from reading. Reading a sensor's data needs read access;
  recording data on one needs the same permission the API requires for recording data by hand.

For schedules the same applies to whatever the schedule would be recorded on, including sensors the
scheduler inherits from the asset tree rather than ones written out in the trigger message.

5. The CLI stays unrestricted

flexmeasures add automation --asset 242 --name "From the CLI" --cron "0 6 * * *" --sensor 913

The CLI runs without a user, so it is trusted and not subject to the check above. That is deliberate:
the restriction is about what a user may set up through the API or the UI.

6. Clean up

Delete anything you created, from the listing's Delete action or with
flexmeasures delete automation --id N --force.