Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query: compilation error for queries with navigations that also compare same navigation to null #7207

Closed
alaatm opened this issue Dec 7, 2016 · 9 comments
Assignees
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone

Comments

@alaatm
Copy link
Contributor

alaatm commented Dec 7, 2016

Exception message:
System.ArgumentException: must be reducible node

Stack trace:
   at System.Linq.Expressions.Expression.ReduceAndCheck()
   at System.Linq.Expressions.Expression.ReduceExtensions()
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteVariableAssignment(BinaryExpression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteAssignBinaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteBlockExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteUnaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLogicalBinaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLogicalBinaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteBinaryExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Expression`1.Accept(StackSpiller spiller)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteLambdaExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.Add(Expression node)
   at System.Linq.Expressions.Compiler.StackSpiller.ChildRewriter.AddArguments(IArgumentProvider expressions)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteMethodCallExpression(Expression expr, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
   at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
   at System.Linq.Expressions.Expression`1.Accept(StackSpiller spiller)
   at System.Linq.Expressions.Compiler.LambdaCompiler.Compile(LambdaExpression lambda)
   at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateExecutorLambda[TResults]()
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult](QueryModel queryModel)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](Expression query, INodeTypeProvider nodeTypeProvider, IDatabase database, ILogger logger, Type contextType)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass19_0`1.<CompileQuery>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Remotion.Linq.QueryableBase`1.GetEnumerator()
   at Program.Main(String[] args) in C:\PersonalProjects\ef-test\src\Program.cs:line 24

Steps to reproduce

Please see this repo for a repro: https://github.com/alaatm/EF-MustBeReducibleNode

Further technical details

EF Core version: 1.1.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10
IDE: Visual Studio Code

@maumar
Copy link
Contributor

maumar commented Dec 8, 2016

Still repros on current dev

@divega divega added this to the 1.1.1 milestone Jan 3, 2017
@divega divega added the type-bug label Jan 3, 2017
@divega
Copy link
Contributor

divega commented Jan 3, 2017

Assigning to 1.1.1 in case this is a regression that needs fixing. We can reevaluate based on the repro.

@maumar
Copy link
Contributor

maumar commented Jan 5, 2017

Verified that this was already broken in 1.0.2, moving out of 1.1.1

@maumar maumar modified the milestones: 2.0.0, 1.1.1 Jan 5, 2017
@Thenuwara
Copy link

Thenuwara commented Feb 2, 2017

IQueryable<Employee> employees =*******************
List<DTO> result = (from emp in empoyees*******
*****
select new DTO(){
Id = emp.Object ==null? 0: emp.Object.Attribute //int value

i had Exception Like This ------------------------------------------------------------------------------------

System.ArgumentException: must be reducible node
at System.Linq.Expressions.Expression.ReduceAndCheck()
at System.Linq.Expressions.Expression.ReduceExtensions()
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteVariableAssignment(BinaryExpression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteAssignBinaryExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteBlockExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExtensionExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteConditionalExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteUnaryExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.MemberAssignmentRewriter..ctor(MemberAssignment binding, StackSpiller spiller, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.BindingRewriter.Create(MemberBinding binding, StackSpiller spiller, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteMemberInitExpression(Expression expr, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpression(Expression node, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteExpressionFreeTemps(Expression expression, Stack stack)
at System.Linq.Expressions.Compiler.StackSpiller.Rewrite[T](Expression`1 lambda)
at System.Linq.Expressions.Compiler.StackSpiller.RewriteLambdaExpression(Expression expr, Stack stack)



more


SOLUTION 👍

> IQueryable<Employee> employees =*****************

change As

List<Employee> employees =*********************

@maumar
Copy link
Contributor

maumar commented Apr 15, 2017

Still repros in current bits. Simplified repro:

    class Program
    {
        static void Main(string[] args)
        {
            using (var ctx = new MyContext())
            {
                ctx.Database.EnsureDeleted();
                ctx.Database.EnsureCreated();

                var query = ctx.Fingerprints
                    .Where(p => !p.Person.Examined || p.Person == null)
                    .Select(p => p.Id);

                var result = query.ToList();
            }
        }
    }

    public class MyContext : DbContext
    {
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer(@"Server=.;Database=Repro7207;Trusted_Connection=True;MultipleActiveResultSets=True");
        }

        public DbSet<Person> Persons { get; set; }
        public DbSet<Fingerprint> Fingerprints { get; set; }
    }

    public class Person
    {
        public int Id { get; set; }
        public bool Examined { get; set; }
        public Fingerprint Fingerprint { get; set; }
    }

    public class Fingerprint
    {
        public int Id { get; set; }
        public int? PersonId { get; set; }
        public Person Person { get; set; }
    }

query plan:

(QueryContext queryContext) => IEnumerable<int> _InterceptExceptions(
    source: IEnumerable<int> _Select(
        source: IEnumerable<ValueBuffer> _Where(
            source: IEnumerable<ValueBuffer> _ShapedQuery(
                queryContext: queryContext, 
                shaperCommandContext: SelectExpression: 
                    SELECT [p].[PersonId], [p].[Id]
                    FROM [Fingerprints] AS [p], 
                shaper: ValueBufferShaper), 
            predicate: (ValueBuffer p) => !( ---> [p.Person] <--- ?.Examined) || (Nullable<bool>) !(p.IsEmpty) ? Nullable<int> TryReadValue(p, 0, Fingerprint.PersonId) : null == null == True), 
        selector: (ValueBuffer p) => int TryReadValue(p, 1, Fingerprint.Id)), 
    contextType: Repro7207.MyContext, 
    logger: InterceptingLogger<Query>, 
    queryContext: queryContext)

@smitpatel
Copy link
Contributor

It looks like that nav rewrite is not expanding through not expression.

@maumar maumar changed the title ArgumentException when filtering Query: compilation error for queries with navigations that also compare same navigation to null Apr 19, 2017
maumar added a commit that referenced this issue Apr 19, 2017
…that also compare same navigation to null

Problem was in the optimization that compared navigation to null, specifically:
o.Customer == null => c.CustomerId == null

If we detected that this optimization could be performed, we assumed that the navigation [o.Customer] is no longer needed, and it was being removed from the navigation joins list.
However, if that navigation was also used for another nav access, then it should not have been removed.

Fix is to add counter to the NavigationJoins list, and only remove given navigation join from the list, if it's not referenced anywhere else in the query.

Also removed some dead code and fixed an issue where we would not apply:
c == o.Customer => c.Id == o.CustomerId

if CustomerId was optional (i.e. clr type was different than the PK on the other side)
maumar added a commit that referenced this issue Apr 19, 2017
…that also compare same navigation to null

Problem was in the optimization that compared navigation to null, specifically:
o.Customer == null => c.CustomerId == null

If we detected that this optimization could be performed, we assumed that the navigation [o.Customer] is no longer needed, and it was being removed from the navigation joins list.
However, if that navigation was also used for another nav access, then it should not have been removed.

Fix is to add counter to the NavigationJoins list, and only remove given navigation join from the list, if it's not referenced anywhere else in the query.

Also removed some dead code and fixed an issue where we would not apply:
c == o.Customer => c.Id == o.CustomerId

if CustomerId was optional (i.e. clr type was different than the PK on the other side)
maumar added a commit that referenced this issue Apr 19, 2017
…that also compare same navigation to null

Problem was in the optimization that compared navigation to null, specifically:
o.Customer == null => c.CustomerId == null

If we detected that this optimization could be performed, we assumed that the navigation [o.Customer] is no longer needed, and it was being removed from the navigation joins list.
However, if that navigation was also used for another nav access, then it should not have been removed.

Fix is to add counter to the NavigationJoins list, and only remove given navigation join from the list, if it's not referenced anywhere else in the query.

Also removed some dead code and fixed an issue where we would not apply:
c == o.Customer => c.Id == o.CustomerId

if CustomerId was optional (i.e. clr type was different than the PK on the other side)
@ajcvickers ajcvickers modified the milestones: 2.0.0-preview1, 2.0.0 Apr 19, 2017
@maumar maumar closed this as completed in 9d9c29b Apr 19, 2017
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Apr 19, 2017
@maumar maumar modified the milestones: 2.0.0-preview1, 2.0.0 Apr 19, 2017
@divega divega added closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. and removed closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. labels May 10, 2017
@vladislav-karamfilov
Copy link

vladislav-karamfilov commented May 17, 2017

Is there a known workaround for this in version 1.1.2? (Except materializing the data in memory before making the projection)

@maumar
Copy link
Contributor

maumar commented May 17, 2017

You can try comparing a property on an entity to null, rather than the entity itself. E.g.: instead of doing
where p.Person.Name == "Foo" || p.Person == null

Try:
where p.Person.Name == "Foo" || p.Person.Name == null

This should circumvent the bug, but you might hit some other issues. Worth to try though.

@vladislav-karamfilov
Copy link

Wow, that worked! Thanks a lot! 👌

@ajcvickers ajcvickers modified the milestones: 2.0.0-preview1, 2.0.0 Oct 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Development

No branches or pull requests

8 participants