Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'must be reducible node' when aggregating over a join #5812

Closed
koenbeuk opened this issue Jun 20, 2016 · 6 comments
Closed

'must be reducible node' when aggregating over a join #5812

koenbeuk opened this issue Jun 20, 2016 · 6 comments
Assignees
@maumar
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone
1.1.0

Comments

@koenbeuk
Copy link

Using EF Core version: 1RC2

Given this model:

public class Fee { 
    public Guid Id { get; set; }
    public Foo Foo { get; set; }
    public ICollection<Foo> Foos { get; set; }
}
public class Foo {
    public Guid Id { get; set; }
    public ICollection<Bar> Bars { get; set; }
}
public class Bar {
    public Guid Id { get; set; }
}

A simple query like this works fine:
context.Set<Foo>().Where(x => x.Bars.Any()).ToList();

Similarly, this query will also work (albeit it using multiple queries):
context.Fees.Select(x => x.Foo).Where(x => x.Bars.Any()).ToList();

However this query fails with a 'must be reducible node' exception (see stacktrace below):
context.Fees.SelectMany(x => x.Foos).Where(x => x.Bars.Any()).ToList();

Exception message: must be reducible node
Stack trace:

   at System.Linq.Expressions.Expression.ReduceAndCheck()
   at System.Linq.Expressions.Expression.ReduceExtensions()
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteBinaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLambdaExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLambdaExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Compiler.LambdaCompiler.Compile(LambdaExpression lambda)
   at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateExecutorLambda[TResults]()
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult](QueryModel queryModel)
@rowanmiller rowanmiller added this to the 1.1.0 milestone Jun 20, 2016
@rowanmiller rowanmiller removed the pri0 label Jul 6, 2016
@bajicdusko
Copy link

bajicdusko commented Jul 13, 2016 •
edited
Loading

I did not want to open new issues, since i have identical exception. My problem is more closer to this example here: Must be reducible node exception when performing Join

I am using EF Core 1.0.0.

These are my models:

ProjectUser

        public ProjectUser()
        {
            TimeEntry = new HashSet<TimeEntry>();
        }

        public Guid Id { get; set; }
        public Guid ProjectId { get; set; }
        public Guid UserId { get; set; }
        public bool IsTimekeeper { get; set; }
        public bool IsSupervisor { get; set; }
        public bool IsAdmin { get; set; }

        public virtual ICollection<TimeEntry> TimeEntry { get; set; }
        public virtual Project Project { get; set; }
        public virtual User User { get; set; }

User

        public User()
        {
            OrganizationUser = new HashSet<OrganizationUser>();
            ProjectUser = new HashSet<ProjectUser>();
        }

        public Guid Id { get; set; }
        public string Email { get; set; }
        public string DisplayName { get; set; }
        public bool IsSystem { get; set; }
        public bool IsRoot { get; set; }
        public bool IsActive { get; set; }

        public virtual ICollection<OrganizationUser> OrganizationUser { get; set; }
        public virtual ICollection<ProjectUser> ProjectUser { get; set; }

And this is the query that i am executing:

from u in DbContext.User
join pu in DbContext.ProjectUser.Where(filter) on u.Id equals pu.UserId into pus
from pu in pus.DefaultIfEmpty()
select new {
     IsActive = pu != null
     ProjectUser = pu,
     User = u
}

Exception message 'must be reducable node':

   at System.Linq.Expressions.Expression.ReduceAndCheck()
   at System.Linq.Expressions.Expression.ReduceExtensions()
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMemberExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteConditionalExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.MemberAssignmentRewriter..ctor(MemberAssignment binding, StackSpiller spiller, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.BindingRewriter.Create(MemberBinding binding, StackSpiller spiller, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMemberInitExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteNewExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Expression`1.Accept(StackSpiller spiller)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLambdaExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Expression`1.Accept(StackSpiller spiller)
   at System.Linq.Expressions.Compiler.LambdaCompiler.Compile(LambdaExpression lambda, DebugInfoGenerator debugInfoGenerator)
   at System.Linq.Expressions.Expression`1.Compile()
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateExecutorLambda[TResults]()
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult](QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](QueryModel queryModel)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass19_0`1.<CompileQuery>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQuery[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Remotion.Linq.QueryableBase`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Ticks.Api.Controllers.ProjectUsersController.Query(Nullable`1 id, String query, Boolean me, Nullable`1 projectId) in E:\Repositories\Codaxy\ticks\Source\Ticks.Api\Controllers\ProjectUsersController.cs:line 59
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.Execute(Object instance, Object[] arguments)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)

As AlexTo commented here by adding ToList() to DbContext.ProjectUser.Where(filter) query is working.

Thank you.

@rguerreiro
Copy link

rguerreiro commented Aug 13, 2016 •
edited
Loading

I'm getting the exact same error than @bajicdusko with a very similar query. Also using EF Core v1.0.0

My query:

var query = from publisher in Db.MyPublishers
            join sr in (
                from serviceReport in Db.MyServiceReports
                where serviceReport.ServiceDate == currentServiceDate && serviceReport.EntryServiceDate <= currentServiceDate
                select serviceReport
            ) on publisher.Id equals sr.Publisher.Id into serviceReports
            from serviceReport in serviceReports.DefaultIfEmpty()
            where publisher.Removed == false && (publisher.PublisherStartDate == null || publisher.PublisherStartDate <= currentCalendarServiceDate)
            select new ServiceReportPublisherViewModel
            {
                PublisherId = publisher.Id,
                PublisherName = publisher.DisplayName,
                PublisherStatusId = serviceReport != null ? serviceReport.PublisherStatusId : publisher.PublisherStatusId,
                ServiceGroupId = serviceReport != null ? serviceReport.ServiceGroupId : publisher.ServiceGroupId,
                CurrentServiceDate = serviceReport == null ? currentServiceDate : serviceReport.ServiceDate,
                CurrentAlreadyReported = serviceReport != null,
                ReportedMonths = (from s in publisher.ServiceReports where s.ServiceYear == currentServiceDate.Year && s.ServiceDate < currentServiceDate && s.EntryServiceDate <= currentServiceDate select s.ServiceDate.Month),
                PublisherStartDate = publisher.PublisherStartDate,
            };

return await query.ToListAsync();

maumar added a commit that referenced this issue Aug 30, 2016
@maumar
Fix to #5812 - 'must be reducible node' when aggregating over a join
565df08 
Problem was that when we do SelectMany, we should later re-map all subsequent references to the original query source, to point to the new query source (added during SelectMany nav rewrite). However we were not doing that for query sources inside a subquery. This caused incorrect query sources to be embedded in the query, which could not be translated and hence the 'must be reducible node' exception was thrown.

Fix is to also replace query sources inside a subquery.
@maumar
Copy link
Contributor

maumar commented Aug 30, 2016

Fixed in 565df08

@maumar maumar closed this as completed Aug 30, 2016
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Aug 30, 2016
@ranouf
Copy link

ranouf commented Jul 18, 2017

Hi,

I m using the version 1.1.2 and I still have the bug:

        var query = _userFavoriteRepository.GetAll();

        if (user.Role == Role.SuperAdmin || user.Role == Role.Manager)
        {
            query = query
                .Where(uf => uf.Shop != null
                    ? uf.Shop.AllowedUserPermissions.Any(aup => aup.AllowedUserId == user.Id)
                    : true
                )
                .Where(uf => uf.ShopChain != null
                    ? uf.ShopChain.AllowedUserPermissions.Any(aup => aup.AllowedUserId == user.Id)
                    : true
                )
                .Where(uf => uf.ShopLocation != null
                    ? uf.ShopLocation.AllowedUserPermissions.Any(aup => aup.AllowedUserId == user.Id)
                    : true
                );
        }
        return query;

UserFavorite:
public Guid UserId { get; private set; }
public virtual User User { get; set; }

    public Guid? ShopId { get; private set; }
    public virtual Shop Shop { get; set; }

    public Guid? ShopChainId { get; private set; }
    public virtual ShopChain ShopChain { get; set; }

    public Guid? ShopLocationId { get; private set; }
    public virtual ShopLocation ShopLocation { get; set; }

AllowedUserPermissions
[ForeignKey("AllowedUser")]
public Guid AllowedUserId { get; private set; }
public virtual User AllowedUser { get; set; }

    public Guid? ShopId { get; private set; }
    public virtual Shop Shop { get; set; }

    public Guid? ShopChainId { get; private set; }
    public virtual ShopChain ShopChain { get; set; }

    public Guid? ShopLocationId { get; private set; }
    public virtual ShopLocation ShopLocation { get; set; }

    [ForeignKey("User")]
    public Guid? UserId { get; private set; }
    public virtual User User { get; set; }

Shop, ShopChain, ShopLocation:
[...]
public virtual ICollection AssignedOfferPermissions { get; } = new List();

[...]

Is there a solution to fix it?

@maumar
Copy link
Contributor

maumar commented Jul 18, 2017

@ranouf this looks like a different case - "must be reducible node" exception appears for lot of cases when we fail to correctly translate the query. Can you file a new issue, so we can track/triage is better?

@ranouf
Copy link

ranouf commented Jul 18, 2017

Hi @maumar
#9199
Done

@ajcvickers ajcvickers modified the milestones: 1.1.0-preview1, 1.1.0 Oct 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maumar maumar

Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Milestone
1.1.0
Development

No branches or pull requests
7 participants
@rguerreiro @bajicdusko @ajcvickers @maumar @ranouf @koenbeuk @rowanmiller