Skip to content

Track nested-lambda closure captures during flat-expression construction#546

Merged
dadhi merged 6 commits intomasterfrom
copilot/map-parameter-usage-in-lambdas
May 5, 2026
Merged

Track nested-lambda closure captures during flat-expression construction#546
dadhi merged 6 commits intomasterfrom
copilot/map-parameter-usage-in-lambdas

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 4, 2026

FlatExpression was indexing lambda/block/control-flow nodes, but it was not carrying forward the closure-shaping information that TryCollectInfo derives for nested lambdas. This change makes flat-expression construction record captured outer parameters/variables directly, so nested-lambda closure metadata is available without a separate collect pass.

  • Add closure-capture metadata to ExprTree

    • Introduce LambdaClosureParameterUsage and ExprTree.LambdaClosureParameterUsages
    • Record, per lambda, which outer parameter/variable identities are used from outside the lambda’s own parameter list
  • Collect captures during flat construction

    • Run capture analysis when building lambdas via the direct ExprTree.Lambda(...) API
    • Run the same analysis when flattening existing expression trees via ExprTree.FromExpression(...)
    • Handle the same scoping shapes that matter for closure creation:
      • nested lambdas
      • block-local variables
      • catch variables / filters
      • propagated captures through multiple nesting levels
  • Preserve identity-based mapping

    • Store the shared parameter/variable identity (ExprNode.ChildIdx) so closure analysis stays aligned with the existing flat-tree identity model
    • Deduplicate repeated uses of the same captured parameter within a lambda
  • Add focused coverage

    • Direct flat construction: captured outer parameter
    • Multi-level nesting: propagated capture into intermediate lambdas
    • Captured block variable
    • ToFlatExpression() conversion from System.Linq.Expressions

Example:

var fe = default(ExprTree);
var x = fe.ParameterOf<int>("x");

var inner = fe.Lambda<Func<int>>(x);
fe.RootIndex = fe.Lambda<Func<int, Func<int>>>(inner, x);

// inner lambda captures outer x
var usage = fe.LambdaClosureParameterUsages[0];
Console.WriteLine((usage.LambdaIndex, usage.ParameterId));

Copilot AI changed the title [WIP] Fix FlatExpression to track parameter usage in nested lambdas Track nested-lambda closure captures during flat-expression construction May 4, 2026
Copilot AI requested a review from dadhi May 4, 2026 12:21
Copilot finished work on behalf of dadhi May 4, 2026 12:21
/// <summary>Maps a lambda node to a captured outer parameter/variable identity used for closure creation.</summary>
public struct LambdaClosureParameterUsage
{
public int LambdaIndex;
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The possible range of indexes is 2 bytes. Please use it here for indexes as well. Use explicit zyruct kayout to avoid padding.
Add xml doc comments for the public fields. The ci is failing.
Use the consistent naming for indexes - if we are using Idx in FE nodes, use Idx here indtead of index.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7e61241 with the compact ushort/Idx update, and followed up in 772ea38 with the XML docs and explicit note about the shared 16-bit FE index range.

if (children.Count == 0)
return;

SmallList<int, Stack8<int>, NoArrayPool<int>> lambdaParameterIds = default;
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use shorts or better ushort for all stacks/lists with indexes here and below. Why are we wating 2 bytes per index?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7e61241 by switching the new capture-analysis index stacks/lists to ushort, with the follow-up range/documentation cleanup in 772ea38.

@@ -197,6 +213,12 @@ public struct ExprTree
/// enabling callers to locate all try regions without a full tree traversal.</summary>
public SmallList<int, Stack16<int>, NoArrayPool<int>> TryCatchNodes;
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ushort instead of int for each index operating list.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7e61241 by using ushort for the new index-tracking lists and compact capture records, with the final range/documentation touch-up in 772ea38.

Copilot finished work on behalf of dadhi May 5, 2026 08:27
Copilot AI requested a review from dadhi May 5, 2026 08:27
@dadhi dadhi marked this pull request as ready for review May 5, 2026 09:25
@dadhi
Copy link
Copy Markdown
Owner

dadhi commented May 5, 2026

@copilot please fix this warning from ci:

Warning: D:\a\FastExpressionCompiler\FastExpressionCompiler\test\FastExpressionCompiler.IssueTests\Issue500_IndexOutOfRangeException_with_value_objects_implicit_conversions.cs(27,85): warning CS8632: The annotation for nullable reference types should only be used in code within a '#nullable' annotations context. [D:\a\FastExpressionCompiler\FastExpressionCompiler\test\FastExpressionCompiler.IssueTests\FastExpressionCompiler.IssueTests.csproj::TargetFramework=net8.0]

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 5, 2026

@copilot please fix this warning from ci:

Warning: D:\a\FastExpressionCompiler\FastExpressionCompiler\test\FastExpressionCompiler.IssueTests\Issue500_IndexOutOfRangeException_with_value_objects_imp...

Fixed in c388121 by enabling nullable context in the shared Issue500 test file so the ? annotations no longer produce CS8632 across the issue-test target frameworks.

Copilot finished work on behalf of dadhi May 5, 2026 10:09
@dadhi dadhi merged commit b186a36 into master May 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FlatExpression: map parameter usage for closures in nested lambdas

2 participants