Validate emissions with lower bound

[danielhuppmann]

This PR adds validation of non-negativity (lower bound = 0) to all emissions species except CO2 and Kyoto GHG, and enforces non-negative values for gross emission and gross removals of CO2.

FYI @jkikstra @IAMconsortium/common-definitions-emissions @phackstock @dc-almeida

Closes #339

[phackstock]

Looks good to me, one small question in line.
Did you check that the replacement of the lower_bound values works and a number is actually read instead of a string? Does the lower_bound replacement also work for the None values and is the validator (or lack thereof in this case) then applied correctly?

[phackstock]

The notes should be different to the ones for Gross Emissions|CO2, no?

[danielhuppmann]

Don't understand what you mean, the equation "Net Emissions = Gross Emissions - Gross Removals" should always be true. And I think it's less confusing (and less error-prone) to always use exactly the same ordering of terms.

[danielhuppmann]

Did you check that the replacement of the lower_bound values works and a number is actually read instead of a string?

Yes

Does the lower_bound replacement also work for the None values and is the validator (or lack thereof in this case) then applied correctly?

Yes. (It has to be None in the yaml-sense of no value, not the string "None")

[flohump]

In general I agree with the sign change — the +/− inconsistency should be fixed.
One concern before it lands:

@danielhuppmann — could we check the ScenarioMIP impact before merging? The
ScenarioMIP submission workflow (iiasa/ssp-submission-workflow) picks up changes
from this repo's main automatically — there's no fixed version. So merging the
sign change would immediately change validation for submissions already in: most
models reported Gross Removals|CO2|AFOLU as negative and would suddenly fail the
new lower_bound: 0 check. Could we freeze the workflow to a fixed version before
this merges, and update it once ScenarioMIP submissions are final?
Also tagging @laurinks here

(There's a related structural question — whether the Intentional/Unintentional
split should stay at all — but that belongs in #224, where I've commented.)

[gidden]

One additional note - in work we did with @tgasser, a small number of assessed scenarios had negative values for carbon removal on land due to the flux definitions applied.

In principle, I agree this is counter intuitive and it may be best to just have a check that all gross removals are strictly positive. But based on how land carbon flux is defined, it is possible (but unlikely) to provide a valid negative flux for this variable.

[jkikstra]

ScenarioMIP submission workflow (iiasa/ssp-submission-workflow) ... Could we freeze the workflow to a fixed version before this merges, and update it once ScenarioMIP submissions are final?

@phackstock would you be able to take care of that?

[tgasser]

One additional note - in work we did with @tgasser, a small number of assessed scenarios had negative values for carbon removal on land due to the flux definitions applied.

In principle, I agree this is counter intuitive and it may be best to just have a check that all gross removals are strictly positive. But based on how land carbon flux is defined, it is possible (but unlikely) to provide a valid negative flux for this variable.

Just want to insist: for the land sector, in some cases, it is possible to get a negative removal, i.e. a flux to the atmosphere. I'd be careful with the validation, here.

[phackstock]

ScenarioMIP submission workflow (iiasa/ssp-submission-workflow) ... Could we freeze the workflow to a fixed version before this merges, and update it once ScenarioMIP submissions are final?

@phackstock would you be able to take care of that?

@jkikstra, the validation is not automatically applied anyway, so merging this would not cause any uploads to fail without explicitly setting the validator in the workflow. This should be fine to merge without having to pin common-definitions in the ssp-submission-workflow.

Never mind, as @danielhuppmann pointed out, the validation would be automatically applied, we'll apply the pin once this PR gets merged to avoid any issues.